2023-07-13T21:44:17.755 INFO:root:teuthology version: 0.0.1.dev143+g073e6eb 2023-07-13T21:44:17.756 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} --name rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268 --verbose -- /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/orig.config.yaml 2023-07-13T21:44:17.809 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-13T21:44:17.895 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268 branch: wip-rishabh-2023Jul13 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7336268' kernel: client: branch: testing flavor: default kdb: true sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi117.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 073e6eb1231451146b4a01d0b85b4f490f71d266 timestamp: 2023-07-13_21:35:13 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-13T21:44:17.896 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa; will attempt to use it 2023-07-13T21:44:17.896 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3fc0089cc1082313a98853d48b892acdc1385b8b/qa/tasks 2023-07-13T21:44:17.897 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-13T21:44:17.898 INFO:teuthology.task.internal:Checking packages... 2023-07-13T21:44:17.926 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '3fc0089cc1082313a98853d48b892acdc1385b8b' 2023-07-13T21:44:17.927 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-13T21:44:17.927 INFO:teuthology.packaging:ref: None 2023-07-13T21:44:17.928 INFO:teuthology.packaging:tag: None 2023-07-13T21:44:17.928 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13 2023-07-13T21:44:17.928 INFO:teuthology.packaging:sha1: 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T21:44:17.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13 2023-07-13T21:44:18.159 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4757.g3fc0089c 2023-07-13T21:44:18.159 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-13T21:44:18.166 INFO:teuthology.task.internal:no buildpackages task found 2023-07-13T21:44:18.167 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-13T21:44:18.198 INFO:teuthology.task.internal:Saving configuration 2023-07-13T21:44:18.218 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-13T21:44:18.231 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T21:44:18.261 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.079125', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-13T21:44:18.288 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.081778', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-13T21:44:18.314 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.080855', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-13T21:44:18.314 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-13T21:44:18.321 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-13T21:44:18.321 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-13T21:44:18.322 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-13T21:44:18.322 INFO:teuthology.run_tasks:Running task console_log... 2023-07-13T21:44:18.459 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd13533f4c0>, signals=[15]) 2023-07-13T21:44:18.461 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-13T21:44:18.468 INFO:teuthology.task.internal:Opening connections... 2023-07-13T21:44:18.468 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T21:44:18.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:44:18.542 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2023-07-13T21:44:18.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:44:18.611 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-07-13T21:44:18.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:44:18.676 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-13T21:44:18.685 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2023-07-13T21:44:18.706 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2023-07-13T21:44:18.706 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2023-07-13T21:44:18.766 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Red Hat Enterprise Linux" 2023-07-13T21:44:18.766 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="8.6 (Ootpa)" 2023-07-13T21:44:18.766 INFO:teuthology.orchestra.run.smithi026.stdout:ID="rhel" 2023-07-13T21:44:18.766 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="fedora" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="8.6" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el8" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.redhat.com/" 2023-07-13T21:44:18.767 INFO:teuthology.orchestra.run.smithi026.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-13T21:44:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-13T21:44:18.769 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-07-13T21:44:18.808 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2023-07-13T21:44:18.828 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2023-07-13T21:44:18.828 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2023-07-13T21:44:18.887 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Red Hat Enterprise Linux" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="8.6 (Ootpa)" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:ID="rhel" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="fedora" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="8.6" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el8" 2023-07-13T21:44:18.888 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.redhat.com/" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-13T21:44:18.889 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-13T21:44:18.890 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-13T21:44:18.890 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-13T21:44:18.890 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2023-07-13T21:44:18.915 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-07-13T21:44:18.931 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-07-13T21:44:18.932 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-07-13T21:44:18.986 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2023-07-13T21:44:18.986 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.6 (Ootpa)" 2023-07-13T21:44:18.986 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.6" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-13T21:44:18.987 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-13T21:44:18.988 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-13T21:44:18.989 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-13T21:44:18.989 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-07-13T21:44:19.016 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-13T21:44:19.026 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-13T21:44:19.056 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-13T21:44:19.057 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2023-07-13T21:44:19.059 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2023-07-13T21:44:19.061 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-07-13T21:44:19.078 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-13T21:44:19.090 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-13T21:44:19.090 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2023-07-13T21:44:19.118 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2023-07-13T21:44:19.121 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-07-13T21:44:19.150 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-13T21:44:19.233 INFO:teuthology.run_tasks:Running task kernel... 2023-07-13T21:44:19.249 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a440c79eee0258ae07e4988d8690fe6187b8b7a3'} 2023-07-13T21:44:19.250 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-13T21:44:19.250 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-13T21:44:19.251 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-13T21:44:19.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:44:19.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-13T21:44:19.520 DEBUG:teuthology.task.kernel:sha1 for client.0 is a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:44:19.520 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "a440c79eee0258ae07e4988d8690fe6187b8b7a3"... 2023-07-13T21:44:19.521 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-07-13T21:44:19.541 INFO:teuthology.orchestra.run.smithi026.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-13T21:44:19.541 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:44:19.541 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-13T21:44:19.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum upgrade -y linux-firmware 2023-07-13T21:44:20.923 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-07-13T21:44:20.923 INFO:teuthology.orchestra.run.smithi026.stdout:Unable to read consumer identity 2023-07-13T21:44:25.043 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T21:44:26.810 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-07-13T21:44:30.435 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 417 kB/s | 24 kB 00:00 2023-07-13T21:44:31.234 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-07-13T21:44:31.235 INFO:teuthology.orchestra.run.smithi026.stdout:Nothing to do. 2023-07-13T21:44:31.235 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-07-13T21:44:31.366 INFO:teuthology.task.kernel:Downloading kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-13T21:44:31.367 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a440c79eee0258ae07e4988d8690fe6187b8b7a3 2023-07-13T21:44:31.590 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ 2023-07-13T21:44:31.591 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-13T21:44:31.593 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2023-07-13T21:44:45.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13 21:44:45 URL:https://2.chacra.ceph.com/r/kernel/testing/a440c79eee0258ae07e4988d8690fe6187b8b7a3/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_ga440c79eee02-1.x86_64.rpm [380684764/380684764] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-13T21:44:45.911 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2023-07-13 21:44:45-- 2023-07-13T21:44:45.911 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 14s 2023-07-13T21:44:45.911 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 363M in 14s (26.0 MB/s) 2023-07-13T21:44:45.912 INFO:teuthology.task.kernel:Installing kernel a440c79eee0258ae07e4988d8690fe6187b8b7a3 on client.0... 2023-07-13T21:44:45.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-13T21:44:47.573 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying... ######################################## 2023-07-13T21:44:47.633 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing... ######################################## 2023-07-13T21:44:47.759 INFO:teuthology.orchestra.run.smithi026.stdout:Updating / installing... 2023-07-13T21:45:28.467 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-6.5.0_rc1_ga440c79eee02-1 ######################################## 2023-07-13T21:45:52.805 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-13T21:45:52.806 DEBUG:teuthology.orchestra.run.smithi026:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-13T21:45:52.853 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:52.853 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/config-6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/vmlinuz-6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto 2023-07-13T21:45:52.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-13T21:45:52.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-13T21:45:52.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu 2023-07-13T21:45:52.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/arch/x86/kvm/kvm.ko 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ansi_cprng.ko 2023-07-13T21:45:52.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/anubis.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/arc4.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_memcpy.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_pq.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_tx.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/async_xor.ko 2023-07-13T21:45:52.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/async_tx/raid6test.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authenc.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/authencesn.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blake2b_generic.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_common.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/blowfish_generic.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/camellia_generic.ko 2023-07-13T21:45:52.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast5_generic.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast6_generic.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cast_common.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/ccm.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cmac.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crc32_generic.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_engine.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/crypto_user.ko 2023-07-13T21:45:52.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/cts.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/des_generic.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/dh_generic.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/echainiv.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/essiv.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/fcrypt.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/khazad.ko 2023-07-13T21:45:52.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/lz4hc.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/md4.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/michael_mic.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcbc.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/pcrypt.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/rmd160.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/seed.ko 2023-07-13T21:45:52.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/serpent_generic.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tcrypt.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/tea.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_common.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/twofish_generic.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/vmac.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/wp512.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xcbc.ko 2023-07-13T21:45:52.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xor.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/crypto/xxhash_generic.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/acpi_pad.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/custom_method.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/ec_sys.ko 2023-07-13T21:45:52.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/platform_profile.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbs.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/sbshc.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/acpi/video.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/acard-ahci.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ahci_platform.ko 2023-07-13T21:45:52.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/ata_generic.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/libahci_platform.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_acpi.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ali.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_amd.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_artop.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atiixp.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_atp867x.ko 2023-07-13T21:45:52.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd640.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cmd64x.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_cypress.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_efar.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt366.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt37x.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-13T21:45:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it8213.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_it821x.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_jmicron.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_marvell.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_mpiix.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_netcell.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ninja32.ko 2023-07-13T21:45:52.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87410.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_ns87415.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_oldpiix.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_opti.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_optidma.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pcmcia.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-13T21:45:52.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_piccolo.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_rdc.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sch.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_serverworks.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sil680.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sis.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_sl82c105.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_triflex.ko 2023-07-13T21:45:52.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pata_via.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/pdc_adma.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_inic162x.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_mv.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_nv.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_promise.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_qstor.ko 2023-07-13T21:45:52.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sil24.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sis.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_svw.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_sx4.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_uli.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_via.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ata/sata_vsc.ko 2023-07-13T21:45:52.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/auxdisplay/ks0108.ko 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/bcma/bcma.ko 2023-07-13T21:45:52.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/aoe/aoe.ko 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/brd.ko 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/drbd/drbd.ko 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/floppy.ko 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/loop.ko 2023-07-13T21:45:52.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/nbd.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/null_blk/null_blk.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/pktcdvd.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/rbd.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/virtio_blk.ko 2023-07-13T21:45:52.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/xen-blkfront.ko 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/block/zram/zram.ko 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hangcheck-timer.ko 2023-07-13T21:45:52.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/via-rng.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-13T21:45:52.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/lp.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/mwave/mwave.ko 2023-07-13T21:45:52.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/ppdev.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tlclk.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-13T21:45:52.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/uv_mmtimer.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/char/virtio_console.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-13T21:45:52.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/ccp/ccp.ko 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel 2023-07-13T21:45:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-aes.ko 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/padlock-sha.ko 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio 2023-07-13T21:45:52.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dca/dca.ko 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw 2023-07-13T21:45:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/amd64_edac.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/e752x_edac.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/edac_mce_amd.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3000_edac.ko 2023-07-13T21:45:52.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i3200_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5100_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i5400_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7300_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i7core_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/i82975x_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/ie31200_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/sb_edac.ko 2023-07-13T21:45:52.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/edac/x38_edac.ko 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/edd.ko 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-ich.ko 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid 2023-07-13T21:45:52.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-appleir.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-aureal.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-axff.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-dr.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elecom.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-elo.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-emsff.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gaff.ko 2023-07-13T21:45:52.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gt683r.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-gyration.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-holtekff.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-hyperv.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-icade.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-keytouch.ko 2023-07-13T21:45:52.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-kye.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lcpower.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-led.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-lenovo.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-multitouch.ko 2023-07-13T21:45:52.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-ortek.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-penmount.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-petalynx.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-picolcd.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-pl.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-plantronics.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-primax.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-rmi.ko 2023-07-13T21:45:52.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-common.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-13T21:45:52.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-roccat.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-saitek.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-samsung.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sjoy.ko 2023-07-13T21:45:52.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sony.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-speedlink.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-steelseries.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-sunplus.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tivo.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-tmff.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-topseed.ko 2023-07-13T21:45:52.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-twinhan.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-uclogic.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-waltop.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-wiimote.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-xinmo.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zpff.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/hid-zydacron.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/uhid.ko 2023-07-13T21:45:52.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hid/wacom.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_balloon.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_utils.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hv/hv_vmbus.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/abituguru3.ko 2023-07-13T21:45:52.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7414.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ad7418.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adc128d818.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1025.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1026.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1029.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm1031.ko 2023-07-13T21:45:52.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adm9240.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ads7828.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7410.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7411.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7462.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7470.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7475.ko 2023-07-13T21:45:52.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/adt7x10.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/amc6821.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/applesmc.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asb100.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asc7621.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/atxp1.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/coretemp.ko 2023-07-13T21:45:52.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/dme1737.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds1621.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ds620.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc1403.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/emc6w201.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71805f.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f71882fg.ko 2023-07-13T21:45:52.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/f75375s.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fam15h_power.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/fschmd.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g760a.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/g762.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl518sm.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/gl520sm.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-13T21:45:52.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5500_temp.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/i5k_amb.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmaem.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ibmpex.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina209.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ina2xx.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/it87.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k10temp.ko 2023-07-13T21:45:52.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/k8temp.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lineage-pem.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm63.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm73.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm75.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm77.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm78.ko 2023-07-13T21:45:52.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm80.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm83.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm85.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm87.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm90.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm92.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm93.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95234.ko 2023-07-13T21:45:52.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95241.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/lm95245.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc2945.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4151.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4215.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4222.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4245.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4260.ko 2023-07-13T21:45:52.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/ltc4261.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max16065.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1619.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max1668.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max197.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6639.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6650.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/max6697.ko 2023-07-13T21:45:52.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/mcp3021.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6683.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775-core.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct6775.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/nct7802.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87360.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pc87427.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pcf8591.ko 2023-07-13T21:45:52.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-13T21:45:52.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/powr1220.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5627.ko 2023-07-13T21:45:52.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch5636.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht15.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sht21.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/shtc1.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/sis5595.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47b397.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m1.ko 2023-07-13T21:45:52.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/smsc47m192.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/thmc50.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp102.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp103.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp401.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/tmp421.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via-cputemp.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/via686a.ko 2023-07-13T21:45:52.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt1211.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/vt8231.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627ehf.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83627hf.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83781d.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83791d.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83792d.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83793.ko 2023-07-13T21:45:52.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83795.ko 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l785ts.ko 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/hwmon/w83l786ng.ko 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses 2023-07-13T21:45:52.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-13T21:45:52.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-13T21:45:52.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-13T21:45:52.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-dev.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-mux.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-smbus.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/i2c/i2c-stub.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-13T21:45:52.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_core.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4 2023-07-13T21:45:52.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp 2023-07-13T21:45:52.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-13T21:45:52.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/gameport.ko 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/lightning.ko 2023-07-13T21:45:52.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/gameport/ns558.ko 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/joydev.ko 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/appletouch.ko 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/bcm5974.ko 2023-07-13T21:45:52.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/cyapatp.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/sermouse.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-13T21:45:52.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/altera_ps2.ko 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/arc_ps2.ko 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/serio/serio_raw.ko 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/input/sparse-keymap.ko 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd 2023-07-13T21:45:52.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-blinkm.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lm3530.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-lp3944.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/leds-ss4200.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-13T21:45:52.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md 2023-07-13T21:45:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/bcache/bcache.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-bio-prison.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache-smq.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-cache.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-crypt.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-delay.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-flakey.ko 2023-07-13T21:45:52.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-log-userspace.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-multipath.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-queue-length.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-raid.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-round-robin.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-service-time.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-switch.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-thin-pool.ko 2023-07-13T21:45:52.928 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/dm-verity.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/faulty.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/linear.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/multipath.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid0.ko 2023-07-13T21:45:52.929 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid1.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid10.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/md/raid456.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptbase.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptctl.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptfc.ko 2023-07-13T21:45:52.930 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptlan.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptsas.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptscsih.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/message/fusion/mptspi.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_ich.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/lpc_sch.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/mfd-core.ko 2023-07-13T21:45:52.931 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/sm501.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/viperboard.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/vx855.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/mfd/wl1273-core.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds9802als.ko 2023-07-13T21:45:52.932 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/apds990x.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/bh1770glc.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/cb710/cb710.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/at24.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-13T21:45:52.933 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/eeprom/max6875.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/enclosure.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/hpilo.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29003.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/isl29020.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-13T21:45:52.934 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-me.ko 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei-txe.ko 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/mei/mei.ko 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-13T21:45:52.935 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_7xx1.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tifm_core.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/tsl2550.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_balloon.ko 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci 2023-07-13T21:45:52.936 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/bonding/bonding.ko 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-13T21:45:52.937 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/dummy.ko 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/eql.ko 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-13T21:45:52.938 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-13T21:45:52.939 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-13T21:45:52.940 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-13T21:45:52.941 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom 2023-07-13T21:45:52.942 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-13T21:45:52.943 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-13T21:45:52.944 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic 2023-07-13T21:45:52.945 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-13T21:45:52.946 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/dnet.ko 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-13T21:45:52.947 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ethoc.ko 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/fealnx.ko 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-13T21:45:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-13T21:45:52.949 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/jme.ko 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell 2023-07-13T21:45:52.950 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-13T21:45:52.951 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-13T21:45:52.952 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines 2023-07-13T21:45:52.953 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-13T21:45:52.954 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker 2023-07-13T21:45:52.955 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-13T21:45:52.956 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-13T21:45:52.957 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti 2023-07-13T21:45:52.958 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-13T21:45:52.959 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/geneve.ko 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-13T21:45:52.960 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ifb.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvlan.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/macvtap.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-13T21:45:52.961 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/mii.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/net_failover.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/netconsole.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/nlmon.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-13T21:45:52.962 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/amd.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/bcm87xx.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/broadcom.ko 2023-07-13T21:45:52.963 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/cicada.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/davicom.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/dp83640.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/et1011c.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/icplus.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/lxt.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/marvell.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/micrel.ko 2023-07-13T21:45:52.964 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/national.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/phylink.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/qsemi.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/realtek.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/smsc.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/ste10Xp.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/phy/vitesse.ko 2023-07-13T21:45:52.965 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_async.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppoe.ko 2023-07-13T21:45:52.966 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pppox.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/ppp/pptp.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slhc.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/slip/slip.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/sungem_phy.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tap.ko 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team 2023-07-13T21:45:52.967 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_random.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/tun.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/veth.ko 2023-07-13T21:45:52.968 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/virtio_net.ko 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/vxlan/vxlan.ko 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/net/xen-netfront.ko 2023-07-13T21:45:52.969 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ntb/ntb.ko 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme-core.ko 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/nvme/host/nvme.ko 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport.ko 2023-07-13T21:45:52.970 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_cs.ko 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_pc.ko 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/parport/parport_serial.ko 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-13T21:45:52.971 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pci/xen-pcifront.ko 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/i82092.ko 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/pd6729.ko 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-13T21:45:52.972 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/acerhdf.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-13T21:45:52.973 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/hdaps.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-13T21:45:52.974 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/intel_ips.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-13T21:45:52.975 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/platform/x86/wmi.ko 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients 2023-07-13T21:45:52.976 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/clients/pps_parport.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/pps/pps_core.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ptp/ptp_kvm.ko 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc 2023-07-13T21:45:52.977 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-13T21:45:52.978 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-em3027.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-13T21:45:52.979 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-max6900.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-13T21:45:52.980 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/rtc/rtc-x1205.ko 2023-07-13T21:45:52.981 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-9xxx.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-sas.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/3w-xxxx.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/BusLogic.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/a100u2w.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-13T21:45:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/advansys.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/am53c974.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/atp870u.ko 2023-07-13T21:45:52.983 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bfa/bfa.ko 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-13T21:45:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ch.ko 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-13T21:45:52.985 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dc395x.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/dmx3191d.ko 2023-07-13T21:45:52.986 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/esp_scsi.ko 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/fnic/fnic.ko 2023-07-13T21:45:52.987 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hpsa.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hptiop.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/hv_storvsc.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/initio.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ipr.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ips.ko 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci 2023-07-13T21:45:52.988 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/isci/isci.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libfc/libfc.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/libsas/libsas.ko 2023-07-13T21:45:52.989 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid.ko 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas 2023-07-13T21:45:52.990 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/mvumi.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/pmcraid.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla1280.ko 2023-07-13T21:45:52.991 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/raid_class.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_debug.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-13T21:45:52.992 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/ses.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/st.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/stex.ko 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2 2023-07-13T21:45:52.993 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/virtio_scsi.ko 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/scsi/wd719x.ko 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ssb/ssb.ko 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi 2023-07-13T21:45:52.994 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_file.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_iblock.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_mod.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_pscsi.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/target_core_user.ko 2023-07-13T21:45:52.995 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-13T21:45:52.996 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-13T21:45:52.997 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-13T21:45:52.998 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_gsm.ko 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/n_hdlc.ko 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/nozomi.ko 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/arc_uart.ko 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm 2023-07-13T21:45:52.999 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/tty/synclink_gt.ko 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio 2023-07-13T21:45:53.000 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_aec.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_cif.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_pci_generic.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/uio/uio_sercos3.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-acm.ko 2023-07-13T21:45:53.001 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usblp.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/class/usbtmc.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/mdc800.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/image/microtek.ko 2023-07-13T21:45:53.002 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/adutux.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/appledisplay.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi26.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/emi62.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ezusb.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/idmouse.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/iowarrior.ko 2023-07-13T21:45:53.003 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/ldusb.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/legousbtower.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usb3503.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usblcd.ko 2023-07-13T21:45:53.004 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/uss720.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/misc/yurex.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/phy/phy-generic.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/aircable.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ark3116.ko 2023-07-13T21:45:53.005 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ch341.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cp210x.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cyberjack.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/empeg.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-13T21:45:53.006 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/io_ti.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipaq.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ipw.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ir-usb.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-13T21:45:53.007 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mct_u232.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7720.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/mos7840.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/navman.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/omninet.ko 2023-07-13T21:45:53.008 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/opticon.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/option.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/oti6858.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/pl2303.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcaux.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/qcserial.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/quatech2.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/safe_serial.ko 2023-07-13T21:45:53.009 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/sierra.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ssu100.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/symbolserial.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_debug.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-13T21:45:53.010 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/visor.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/whiteheat.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/uas.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-13T21:45:53.011 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-karma.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-13T21:45:53.012 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/storage/usb-storage.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio 2023-07-13T21:45:53.013 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-13T21:45:53.014 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_net.ko 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/vhost/vhost_scsi.ko 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/apple_bl.ko 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/lcd.ko 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio 2023-07-13T21:45:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_balloon.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_mmio.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/virtio/virtio_ring.ko 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1 2023-07-13T21:45:53.016 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2482.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/masters/ds2490.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-13T21:45:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/w1/wire.ko 2023-07-13T21:45:53.018 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/hpwdt.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/i6300esb.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-13T21:45:53.019 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ib700wdt.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ibmasr.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/it87_wdt.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/machzwd.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/nv_tco.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-13T21:45:53.020 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/softdog.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/via_wdt.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-13T21:45:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/wdt_pci.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/watchdog/xen_wdt.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-evtchn.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntalloc.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-gntdev.ko 2023-07-13T21:45:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xen-privcmd.ko 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/9p/9p.ko 2023-07-13T21:45:53.023 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/affs/affs.ko 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/befs/befs.ko 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/binfmt_misc.ko 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/btrfs/btrfs.ko 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles 2023-07-13T21:45:53.024 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cachefiles/cachefiles.ko 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ceph/ceph.ko 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/coda/coda.ko 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/cramfs/cramfs.ko 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm 2023-07-13T21:45:53.025 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/dlm/dlm.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/f2fs/f2fs.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/fat.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/msdos.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fat/vfat.ko 2023-07-13T21:45:53.026 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fscache/fscache.ko 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/cuse.ko 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/fuse/fuse.ko 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/gfs2/gfs2.ko 2023-07-13T21:45:53.027 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfs/hfs.ko 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/hfsplus/hfsplus.ko 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/isofs/isofs.ko 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/jfs/jfs.ko 2023-07-13T21:45:53.028 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/lockd/lockd.ko 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/minix/minix.ko 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/netfs/netfs.ko 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout 2023-07-13T21:45:53.029 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfs.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv3.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs/nfsv4.ko 2023-07-13T21:45:53.030 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/grace.ko 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfs_common/nfs_acl.ko 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nfsd/nfsd.ko 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nilfs2/nilfs2.ko 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls 2023-07-13T21:45:53.031 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-celtic.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-centeuro.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-croatian.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-cyrillic.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-gaelic.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-greek.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-iceland.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-inuit.ko 2023-07-13T21:45:53.032 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-roman.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-romanian.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/mac-turkish.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1250.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1251.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp1255.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp737.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp775.ko 2023-07-13T21:45:53.033 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp850.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp852.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp855.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp857.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp860.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp861.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp862.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp863.ko 2023-07-13T21:45:53.034 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp864.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp865.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp866.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp869.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp874.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp932.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp936.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp949.ko 2023-07-13T21:45:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_cp950.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_euc-jp.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-1.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-13.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-14.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-15.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-2.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-3.ko 2023-07-13T21:45:53.036 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-4.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-5.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-6.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-7.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_iso8859-9.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-r.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-ru.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_koi8-u.ko 2023-07-13T21:45:53.037 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/nls/nls_utf8.ko 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-13T21:45:53.038 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/overlayfs/overlay.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/pstore/ramoops.ko 2023-07-13T21:45:53.039 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/reiserfs/reiserfs.ko 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/romfs/romfs.ko 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/client/cifs.ko 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common 2023-07-13T21:45:53.040 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_arc4.ko 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/smb/common/cifs_md4.ko 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/squashfs/squashfs.ko 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/sysv/sysv.ko 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/udf/udf.ko 2023-07-13T21:45:53.041 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/ufs/ufs.ko 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/fs/xfs/xfs.ko 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/rcu/rcutorture.ko 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/torture.ko 2023-07-13T21:45:53.042 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/asn1_encoder.ko 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc-itu-t.ko 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crc8.ko 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libarc4.ko 2023-07-13T21:45:53.043 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/crypto/libdes.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/libcrc32c.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lru_cache.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4_compress.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/lz4/lz4hc_compress.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/math/cordic.ko 2023-07-13T21:45:53.044 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/raid6/raid6_pq.ko 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_bm.ko 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_fsm.ko 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/ts_kmp.ko 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd 2023-07-13T21:45:53.045 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/lib/zstd/zstd_compress.ko 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/mm/hwpoison-inject.ko 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/6lowpan.ko 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_dest.ko 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_fragment.ko 2023-07-13T21:45:53.046 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_hop.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_mobility.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_routing.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/6lowpan/nhc_udp.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/garp.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/mrp.ko 2023-07-13T21:45:53.047 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/p8022.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/psnap.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/802/stp.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/8021q/8021q.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_fd.ko 2023-07-13T21:45:53.048 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_rdma.ko 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/9p/9pnet_virtio.ko 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/batman-adv/batman-adv.ko 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/br_netfilter.ko 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/bridge.ko 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter 2023-07-13T21:45:53.049 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-13T21:45:53.050 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-13T21:45:53.051 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/ebtables.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-13T21:45:53.052 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ceph/libceph.ko 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/failover.ko 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/core/pktgen.ko 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp.ko 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_diag.ko 2023-07-13T21:45:53.053 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv4.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dccp/dccp_ipv6.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dns_resolver/dns_resolver.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/dsa_core.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_brcm.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_dsa.ko 2023-07-13T21:45:53.054 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_none.ko 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/dsa/tag_trailer.ko 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154.ko 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4 2023-07-13T21:45:53.055 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ah4.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/esp4.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/fou.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/gre.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/inet_diag.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_gre.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_tunnel.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ip_vti.ko 2023-07-13T21:45:53.056 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipcomp.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/ipip.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-13T21:45:53.057 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-13T21:45:53.058 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-13T21:45:53.059 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_bic.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_dctcp.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_diag.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_highspeed.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_htcp.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_hybla.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_illinois.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_lp.ko 2023-07-13T21:45:53.060 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_scalable.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_vegas.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_veno.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_westwood.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tcp_yeah.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/tunnel4.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_diag.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/udp_tunnel.ko 2023-07-13T21:45:53.061 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ah6.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/esp6.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/fou6.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_tunnel.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ip6_vti.ko 2023-07-13T21:45:53.062 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/ipcomp6.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-13T21:45:53.063 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-13T21:45:53.064 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-13T21:45:53.065 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/sit.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/tunnel6.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/key/af_key.ko 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp 2023-07-13T21:45:53.066 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_core.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_eth.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ip6.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_netlink.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/l2tp/l2tp_ppp.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/llc/llc.ko 2023-07-13T21:45:53.067 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mac802154/mac802154.ko 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/mpls/mpls_gso.ko 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set.ko 2023-07-13T21:45:53.068 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-13T21:45:53.069 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-13T21:45:53.070 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-13T21:45:53.071 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conncount.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack.ko 2023-07-13T21:45:53.072 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-13T21:45:53.073 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_log_syslog.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat.ko 2023-07-13T21:45:53.074 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_irc.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_sip.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nf_tables.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink.ko 2023-07-13T21:45:53.075 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_log.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_chain_nat.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_compat.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_ct.ko 2023-07-13T21:45:53.076 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fib.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_hash.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_limit.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_log.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_masq.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_nat.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_numgen.ko 2023-07-13T21:45:53.077 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_queue.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_quota.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_redir.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_reject_inet.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_socket.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/nft_tunnel.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_AUDIT.ko 2023-07-13T21:45:53.078 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_CT.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_DSCP.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HL.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_HMARK.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-13T21:45:53.079 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LED.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_LOG.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NETMAP.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFLOG.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_RATEEST.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-13T21:45:53.080 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_SECMARK.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TEE.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TPROXY.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_TRACE.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_addrtype.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_bpf.ko 2023-07-13T21:45:53.081 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cgroup.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cluster.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_comment.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connbytes.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlabel.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connlimit.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_connmark.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_conntrack.ko 2023-07-13T21:45:53.082 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_cpu.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dccp.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_devgroup.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_dscp.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ecn.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_esp.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hashlimit.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_helper.ko 2023-07-13T21:45:53.083 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_hl.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipcomp.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_iprange.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_ipvs.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_l2tp.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_length.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_limit.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mac.ko 2023-07-13T21:45:53.084 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_mark.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_multiport.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nat.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_nfacct.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_osf.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_owner.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_physdev.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_pkttype.ko 2023-07-13T21:45:53.085 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_policy.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_quota.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_rateest.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_realm.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_recent.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_sctp.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_set.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_socket.ko 2023-07-13T21:45:53.086 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_state.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_statistic.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_string.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_tcpmss.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_time.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netfilter/xt_u32.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/netlink/netlink_diag.ko 2023-07-13T21:45:53.087 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/nsh/nsh.ko 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/openvswitch.ko 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-geneve.ko 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-gre.ko 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/openvswitch/vport-vxlan.ko 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet 2023-07-13T21:45:53.088 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/packet/af_packet_diag.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_rdma.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/rds/rds_tcp.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_csum.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_gact.ko 2023-07-13T21:45:53.089 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_ipt.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_mirred.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_nat.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_pedit.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_police.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_simple.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_skbedit.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/act_vlan.ko 2023-07-13T21:45:53.090 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_basic.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_bpf.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_flow.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_fw.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_route.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/cls_u32.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_cmp.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_ipset.ko 2023-07-13T21:45:53.091 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_meta.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_nbyte.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_text.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/em_u32.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_choke.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_codel.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_drr.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_fq.ko 2023-07-13T21:45:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_gred.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hfsc.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_hhf.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_htb.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_ingress.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_mqprio_lib.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_multiq.ko 2023-07-13T21:45:53.093 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_netem.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_pie.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_plug.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_prio.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_qfq.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_red.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfb.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_sfq.ko 2023-07-13T21:45:53.094 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_tbf.ko 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sched/sch_teql.ko 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp.ko 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sctp/sctp_diag.ko 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-13T21:45:53.095 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/sunrpc.ko 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/diag.ko 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/tipc/tipc.ko 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix 2023-07-13T21:45:53.096 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/unix/unix_diag.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm 2023-07-13T21:45:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/security/keys/trusted-keys/trusted.ko 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt 2023-07-13T21:45:53.098 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/kernel/virt/lib/irqbypass.ko 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.alias.bin 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.bin 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.builtin.modinfo 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep 2023-07-13T21:45:53.099 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.dep.bin 2023-07-13T21:45:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.devname 2023-07-13T21:45:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.order 2023-07-13T21:45:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.softdep 2023-07-13T21:45:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols 2023-07-13T21:45:53.100 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.5.0-rc1-ga440c79eee02/modules.symbols.bin 2023-07-13T21:45:53.101 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:53.101 DEBUG:teuthology.orchestra.run.smithi026:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-13T21:45:53.129 INFO:teuthology.orchestra.run.smithi026.stdout:postinstall scriptlet (using /bin/sh): 2023-07-13T21:45:53.129 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 -a -r /boot/System.map-6.5.0-rc1-ga440c79eee02 ]; then 2023-07-13T21:45:53.129 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:45:53.129 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/System.map-6.5.0-rc1-ga440c79eee02 /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-ga440c79eee02 /boot/System.map-6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/installkernel 6.5.0-rc1-ga440c79eee02 /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-ga440c79eee02-rpm /boot/.System.map-6.5.0-rc1-ga440c79eee02-rpm 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-13T21:45:53.130 INFO:teuthology.orchestra.run.smithi026.stdout:new-kernel-pkg --remove 6.5.0-rc1-ga440c79eee02 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-ga440c79eee02.img 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-install remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:53.131 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2023-07-13T21:45:53.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2023-07-13T21:45:53.221 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2023-07-13T21:45:53.221 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2023-07-13T21:45:53.221 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.02 2023-07-13T21:45:53.221 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 123.el8 2023-07-13T21:45:53.221 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Group : System Environment/Base 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 9265450 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-13T21:45:53.222 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Relocations : (not relocatable) 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : Red Hat, Inc. 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : Red Hat, Inc. 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2023-07-13T21:45:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T21:45:53.224 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2023-07-13T21:45:53.225 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-13T21:45:53.225 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 6.5.0-rc1-ga440c79eee02 2023-07-13T21:45:53.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-13T21:45:57.792 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2023-07-13T21:45:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:done 2023-07-13T21:45:59.800 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2023-07-13T21:45:59.818 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.qZd6sjxeQZ 2023-07-13T21:45:59.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qZd6sjxeQZ 2023-07-13T21:45:59.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.qZd6sjxeQZ 2023-07-13T21:46:00.056 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.qZd6sjxeQZ is 6KB 2023-07-13T21:46:00.076 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.qZd6sjxeQZ 2023-07-13T21:46:00.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2023-07-13T21:46:00.169 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-13T21:46:00.169 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-13T21:46:00.169 INFO:teuthology.orchestra.run.smithi026.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02.conf 2023-07-13T21:46:00.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-ga440c79eee02 2023-07-13T21:46:00.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2023-07-13T21:46:30.328 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-13T21:46:30.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T21:46:30.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:46:48.816 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-07-13T21:46:57.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T21:46:57.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:47:16.208 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-07-13T21:47:28.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T21:47:28.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T21:47:30.905 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-07-13T21:47:31.371 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T21:47:31.372 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-ga440c79eee02"... 2023-07-13T21:47:31.372 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-07-13T21:47:31.415 INFO:teuthology.orchestra.run.smithi026.stdout:6.5.0-rc1-ga440c79eee02 2023-07-13T21:47:31.415 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-ga440c79eee02 vs 6.5.0-rc1-ga440c79eee02 2023-07-13T21:47:31.416 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-13T21:47:31.416 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-13T21:47:32.417 DEBUG:teuthology.parallel:result is None 2023-07-13T21:47:32.418 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-13T21:47:32.428 INFO:teuthology.task.internal:Creating test directory... 2023-07-13T21:47:32.429 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-13T21:47:32.434 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-13T21:47:32.437 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-13T21:47:32.474 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-13T21:47:32.482 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-13T21:47:32.546 INFO:teuthology.task.internal:Creating archive directory... 2023-07-13T21:47:32.546 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-13T21:47:32.551 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-13T21:47:32.553 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-13T21:47:32.605 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-13T21:47:32.613 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-13T21:47:32.613 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-13T21:47:32.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:47:32.645 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-13T21:47:32.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:47:32.665 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-13T21:47:32.669 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-13T21:47:32.687 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-13T21:47:32.742 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.748 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.759 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.766 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.780 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.812 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-13T21:47:32.816 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-13T21:47:32.831 INFO:teuthology.task.internal:Configuring sudo... 2023-07-13T21:47:32.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-13T21:47:32.836 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-13T21:47:32.838 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-13T21:47:32.901 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-13T21:47:32.917 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-13T21:47:32.918 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-13T21:47:32.922 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-13T21:47:32.924 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-13T21:47:32.960 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.156 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.298 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T21:47:33.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-13T21:47:33.396 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.447 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.494 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.623 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T21:47:33.623 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-13T21:47:33.692 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-13T21:47:33.792 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-13T21:47:33.925 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T21:47:33.926 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-13T21:47:33.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2023-07-13T21:47:33.999 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2023-07-13T21:47:34.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-07-13T21:47:34.070 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T21:47:34.092 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T21:47:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T21:47:34.593 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-13T21:47:34.602 INFO:teuthology.task.internal:Starting timer... 2023-07-13T21:47:34.602 INFO:teuthology.run_tasks:Running task pcp... 2023-07-13T21:47:34.650 INFO:teuthology.run_tasks:Running task selinux... 2023-07-13T21:47:34.681 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-13T21:47:34.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2023-07-13T21:47:34.827 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2023-07-13T21:47:34.831 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2023-07-13T21:47:34.891 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2023-07-13T21:47:34.892 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2023-07-13T21:47:34.954 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2023-07-13T21:47:34.955 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-13T21:47:34.956 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2023-07-13T21:47:35.008 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2023-07-13T21:47:35.009 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2023-07-13T21:47:35.030 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2023-07-13T21:47:35.030 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2023-07-13T21:47:35.049 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2023-07-13T21:47:35.050 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2023-07-13T21:47:35.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T21:47:35.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:47:35.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T21:47:35.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:47:35.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T21:47:35.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T21:47:35.177 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-13T21:47:35.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2023-07-13T21:47:35.248 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2023-07-13T21:47:35.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2023-07-13T21:47:35.304 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-13T21:47:35.315 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-13T21:47:35.320 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-13T21:47:35.453 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-13T21:47:35.473 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-13T21:47:35.475 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi117.front.sepia.ceph.com,smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-13T22:00:15.239 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2023-07-13T22:00:15.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T22:00:15.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T22:00:15.334 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-07-13T22:00:15.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-07-13T22:00:15.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-07-13T22:00:15.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T22:00:15.517 DEBUG:teuthology.orchestra.run.smithi117:> true 2023-07-13T22:00:15.588 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-07-13T22:00:15.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-13T22:00:15.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-13T22:00:15.659 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-07-13T22:00:15.736 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-07-13T22:00:15.737 INFO:teuthology.run_tasks:Running task clock... 2023-07-13T22:00:15.754 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-13T22:00:15.755 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-13T22:00:15.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:00:15.768 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-13T22:00:15.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:00:15.772 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-13T22:00:15.773 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:00:15.804 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-13T22:00:15.813 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-13T22:00:15.822 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-13T22:00:15.828 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-13T22:00:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-13T22:00:15.874 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-13T22:00:15.902 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2023-07-13T22:00:15.903 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2023-07-13T22:00:15.913 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2023-07-13T22:00:15.915 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2023-07-13T22:00:15.933 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-13T22:00:15.933 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-13T22:00:15.944 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-13T22:00:15.945 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-13T22:00:15.971 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2023-07-13T22:00:16.002 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2023-07-13T22:00:16.012 INFO:teuthology.orchestra.run.smithi117.stderr:bash: ntpq: command not found 2023-07-13T22:00:16.013 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.022 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.038 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-13T22:00:16.074 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-13T22:00:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-07-13T22:00:16.181 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-07-13T22:00:16.182 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.185 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:00:16.186 INFO:teuthology.run_tasks:Running task pexec... 2023-07-13T22:00:16.208 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-13T22:00:16.209 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T22:00:16.209 DEBUG:teuthology.orchestra.run.smithi026:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-13T22:00:16.210 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi117.front.sepia.ceph.com 2023-07-13T22:00:16.213 DEBUG:teuthology.orchestra.run.smithi117:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-13T22:00:16.214 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi121.front.sepia.ceph.com 2023-07-13T22:00:16.214 DEBUG:teuthology.orchestra.run.smithi121:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-13T22:00:16.589 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:16.590 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:16.734 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:18.390 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:23 ago on Thu 13 Jul 2023 09:54:54 PM UTC. 2023-07-13T22:00:18.453 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:05:31 ago on Thu 13 Jul 2023 09:54:47 PM UTC. 2023-07-13T22:00:18.810 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:01:58 ago on Thu 13 Jul 2023 09:58:20 PM UTC. 2023-07-13T22:00:20.105 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2023-07-13T22:00:20.106 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2023-07-13T22:00:20.106 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2023-07-13T22:00:20.106 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2023-07-13T22:00:20.106 INFO:teuthology.orchestra.run.smithi117.stdout:Resetting modules: 2023-07-13T22:00:20.106 INFO:teuthology.orchestra.run.smithi117.stdout: container-tools 2023-07-13T22:00:20.107 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:20.107 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2023-07-13T22:00:20.107 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2023-07-13T22:00:20.107 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:20.187 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-07-13T22:00:20.187 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-07-13T22:00:20.187 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-07-13T22:00:20.187 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-07-13T22:00:20.187 INFO:teuthology.orchestra.run.smithi121.stdout:Resetting modules: 2023-07-13T22:00:20.188 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools 2023-07-13T22:00:20.188 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:20.188 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-07-13T22:00:20.188 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-07-13T22:00:20.188 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:20.862 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-07-13T22:00:20.865 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-07-13T22:00:20.865 INFO:teuthology.orchestra.run.smithi026.stdout: Package Architecture Version Repository Size 2023-07-13T22:00:20.865 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-07-13T22:00:20.865 INFO:teuthology.orchestra.run.smithi026.stdout:Resetting modules: 2023-07-13T22:00:20.865 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools 2023-07-13T22:00:20.866 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:20.866 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-07-13T22:00:20.866 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2023-07-13T22:00:20.866 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:21.048 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:26 ago on Thu 13 Jul 2023 09:54:54 PM UTC. 2023-07-13T22:00:21.146 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:05:33 ago on Thu 13 Jul 2023 09:54:47 PM UTC. 2023-07-13T22:00:22.141 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:01 ago on Thu 13 Jul 2023 09:58:20 PM UTC. 2023-07-13T22:00:29.494 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2023-07-13T22:00:29.964 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:31.400 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-07-13T22:00:31.513 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:37 ago on Thu 13 Jul 2023 09:54:54 PM UTC. 2023-07-13T22:00:31.594 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-07-13T22:00:32.019 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:32.064 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:33.368 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2023-07-13T22:00:33.460 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================================ 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================================ 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout:Installing group/module packages: 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-13T22:00:33.461 INFO:teuthology.orchestra.run.smithi117.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-13T22:00:33.462 INFO:teuthology.orchestra.run.smithi117.stdout:Installing module profiles: 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout: container-tools/common 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling module streams: 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout: container-tools rhel8 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================================================ 2023-07-13T22:00:33.463 INFO:teuthology.orchestra.run.smithi117.stdout:Install 10 Packages 2023-07-13T22:00:33.464 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:33.464 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 18 M 2023-07-13T22:00:33.465 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 65 M 2023-07-13T22:00:33.465 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2023-07-13T22:00:33.563 INFO:teuthology.orchestra.run.smithi117.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 935 kB/s | 94 kB 00:00 2023-07-13T22:00:33.588 INFO:teuthology.orchestra.run.smithi117.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 198 kB/s | 25 kB 00:00 2023-07-13T22:00:33.614 INFO:teuthology.orchestra.run.smithi117.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 270 kB/s | 41 kB 00:00 2023-07-13T22:00:33.674 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:05:46 ago on Thu 13 Jul 2023 09:54:47 PM UTC. 2023-07-13T22:00:33.706 INFO:teuthology.orchestra.run.smithi117.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 347 kB/s | 49 kB 00:00 2023-07-13T22:00:33.739 INFO:teuthology.orchestra.run.smithi117.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.4 MB/s | 524 kB 00:00 2023-07-13T22:00:33.773 INFO:teuthology.orchestra.run.smithi117.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-13T22:00:33.815 INFO:teuthology.orchestra.run.smithi117.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.8 MB/s | 213 kB 00:00 2023-07-13T22:00:33.893 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:13 ago on Thu 13 Jul 2023 09:58:20 PM UTC. 2023-07-13T22:00:33.957 INFO:teuthology.orchestra.run.smithi117.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 23 MB/s | 8.0 MB 00:00 2023-07-13T22:00:34.015 INFO:teuthology.orchestra.run.smithi117.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-13T22:00:34.115 INFO:teuthology.orchestra.run.smithi117.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-07-13T22:00:34.116 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:00:34.116 INFO:teuthology.orchestra.run.smithi117.stdout:Total 28 MB/s | 18 MB 00:00 2023-07-13T22:00:34.317 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2023-07-13T22:00:34.340 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2023-07-13T22:00:34.341 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2023-07-13T22:00:34.516 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2023-07-13T22:00:34.520 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2023-07-13T22:00:35.026 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2023-07-13T22:00:35.199 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-13T22:00:35.368 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:35.389 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-07-13T22:00:35.484 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================================ 2023-07-13T22:00:35.484 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-07-13T22:00:35.484 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================================ 2023-07-13T22:00:35.484 INFO:teuthology.orchestra.run.smithi121.stdout:Installing group/module packages: 2023-07-13T22:00:35.484 INFO:teuthology.orchestra.run.smithi121.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-13T22:00:35.485 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-07-13T22:00:35.486 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-13T22:00:35.486 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-13T22:00:35.486 INFO:teuthology.orchestra.run.smithi121.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout:Installing module profiles: 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools/common 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout:Enabling module streams: 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout: container-tools rhel8 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:35.487 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-07-13T22:00:35.488 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================================ 2023-07-13T22:00:35.488 INFO:teuthology.orchestra.run.smithi121.stdout:Install 10 Packages 2023-07-13T22:00:35.488 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:35.489 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 18 M 2023-07-13T22:00:35.489 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 65 M 2023-07-13T22:00:35.489 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-07-13T22:00:35.571 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-13T22:00:35.642 INFO:teuthology.orchestra.run.smithi121.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 160 kB/s | 25 kB 00:00 2023-07-13T22:00:35.667 INFO:teuthology.orchestra.run.smithi121.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 226 kB/s | 41 kB 00:00 2023-07-13T22:00:35.692 INFO:teuthology.orchestra.run.smithi121.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 457 kB/s | 94 kB 00:00 2023-07-13T22:00:35.762 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-13T22:00:35.769 INFO:teuthology.orchestra.run.smithi121.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 394 kB/s | 49 kB 00:00 2023-07-13T22:00:35.818 INFO:teuthology.orchestra.run.smithi121.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.4 MB/s | 524 kB 00:00 2023-07-13T22:00:35.884 INFO:teuthology.orchestra.run.smithi121.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-07-13T22:00:35.926 INFO:teuthology.orchestra.run.smithi121.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.9 MB/s | 213 kB 00:00 2023-07-13T22:00:35.958 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-07-13T22:00:36.060 INFO:teuthology.orchestra.run.smithi121.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-07-13T22:00:36.063 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-07-13T22:00:36.063 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-07-13T22:00:36.063 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-07-13T22:00:36.063 INFO:teuthology.orchestra.run.smithi026.stdout:Installing group/module packages: 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-13T22:00:36.064 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-13T22:00:36.065 INFO:teuthology.orchestra.run.smithi026.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-13T22:00:36.065 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-07-13T22:00:36.065 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-13T22:00:36.065 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-13T22:00:36.065 INFO:teuthology.orchestra.run.smithi026.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-13T22:00:36.067 INFO:teuthology.orchestra.run.smithi026.stdout:Installing module profiles: 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools/common 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout: container-tools rhel8 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-07-13T22:00:36.068 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================================ 2023-07-13T22:00:36.069 INFO:teuthology.orchestra.run.smithi026.stdout:Install 10 Packages 2023-07-13T22:00:36.069 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:36.069 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 18 M 2023-07-13T22:00:36.070 INFO:teuthology.orchestra.run.smithi026.stdout:Installed size: 65 M 2023-07-13T22:00:36.070 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-07-13T22:00:36.114 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-13T22:00:36.135 INFO:teuthology.orchestra.run.smithi121.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-13T22:00:36.192 INFO:teuthology.orchestra.run.smithi026.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 326 kB/s | 41 kB 00:00 2023-07-13T22:00:36.226 INFO:teuthology.orchestra.run.smithi026.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 156 kB/s | 25 kB 00:00 2023-07-13T22:00:36.236 INFO:teuthology.orchestra.run.smithi121.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-07-13T22:00:36.236 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:00:36.236 INFO:teuthology.orchestra.run.smithi121.stdout:Total 24 MB/s | 18 MB 00:00 2023-07-13T22:00:36.251 INFO:teuthology.orchestra.run.smithi026.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 509 kB/s | 94 kB 00:00 2023-07-13T22:00:36.327 INFO:teuthology.orchestra.run.smithi026.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 368 kB/s | 49 kB 00:00 2023-07-13T22:00:36.393 INFO:teuthology.orchestra.run.smithi026.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.1 MB/s | 524 kB 00:00 2023-07-13T22:00:36.427 INFO:teuthology.orchestra.run.smithi026.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.5 MB/s | 149 kB 00:00 2023-07-13T22:00:36.437 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-07-13T22:00:36.456 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-07-13T22:00:36.456 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-07-13T22:00:36.510 INFO:teuthology.orchestra.run.smithi026.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-07-13T22:00:36.627 INFO:teuthology.orchestra.run.smithi026.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 21 MB/s | 8.0 MB 00:00 2023-07-13T22:00:36.645 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-07-13T22:00:36.649 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-07-13T22:00:36.711 INFO:teuthology.orchestra.run.smithi026.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-07-13T22:00:36.836 INFO:teuthology.orchestra.run.smithi026.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-07-13T22:00:36.836 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:00:36.837 INFO:teuthology.orchestra.run.smithi026.stdout:Total 24 MB/s | 18 MB 00:00 2023-07-13T22:00:36.877 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-13T22:00:37.139 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-07-13T22:00:37.178 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-07-13T22:00:37.203 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-07-13T22:00:37.203 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-07-13T22:00:37.324 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-13T22:00:37.520 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:37.702 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-13T22:00:37.739 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-13T22:00:37.750 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-07-13T22:00:37.754 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-07-13T22:00:37.888 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-13T22:00:37.975 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-13T22:00:38.144 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-13T22:00:38.238 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-13T22:00:38.306 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-07-13T22:00:38.336 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:38.472 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-13T22:00:38.594 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-13T22:00:38.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-13T22:00:38.597 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-13T22:00:38.656 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:38.907 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-13T22:00:39.002 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-13T22:00:39.112 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-13T22:00:39.547 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-13T22:00:39.776 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-13T22:00:39.777 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:45 ago on Thu 13 Jul 2023 09:54:54 PM UTC. 2023-07-13T22:00:39.881 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-13T22:00:40.116 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-13T22:00:40.282 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-13T22:00:40.448 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-13T22:00:40.482 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:40.757 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:40.757 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-13T22:00:40.757 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:40.757 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-13T22:00:40.757 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-13T22:00:40.758 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-13T22:00:40.758 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-13T22:00:40.758 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-13T22:00:40.758 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-13T22:00:40.758 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-13T22:00:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-13T22:00:41.712 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-13T22:00:41.894 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-13T22:00:41.931 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-13T22:00:41.931 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:05:54 ago on Thu 13 Jul 2023 09:54:47 PM UTC. 2023-07-13T22:00:42.143 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:42.527 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-13T22:00:42.527 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-13T22:00:42.527 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-13T22:00:42.527 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-13T22:00:42.528 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-13T22:00:43.932 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-13T22:00:43.932 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:23 ago on Thu 13 Jul 2023 09:58:20 PM UTC. 2023-07-13T22:00:45.065 INFO:teuthology.orchestra.run.smithi117.stdout:Installed products updated. 2023-07-13T22:00:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:45.305 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2023-07-13T22:00:45.305 INFO:teuthology.orchestra.run.smithi117.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:45.306 INFO:teuthology.orchestra.run.smithi117.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-13T22:00:45.307 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:00:45.307 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2023-07-13T22:00:45.468 DEBUG:teuthology.parallel:result is None 2023-07-13T22:00:46.538 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-07-13T22:00:46.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:46.747 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-13T22:00:46.748 INFO:teuthology.orchestra.run.smithi121.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:46.749 INFO:teuthology.orchestra.run.smithi121.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:46.749 INFO:teuthology.orchestra.run.smithi121.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:46.749 INFO:teuthology.orchestra.run.smithi121.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-13T22:00:46.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:00:46.749 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-07-13T22:00:46.905 DEBUG:teuthology.parallel:result is None 2023-07-13T22:00:51.059 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-07-13T22:00:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:51.291 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-07-13T22:00:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-13T22:00:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-13T22:00:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-13T22:00:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-13T22:00:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-13T22:00:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-13T22:00:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:00:51.294 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-07-13T22:00:51.501 DEBUG:teuthology.parallel:result is None 2023-07-13T22:00:51.502 INFO:teuthology.run_tasks:Running task install... 2023-07-13T22:00:51.530 DEBUG:teuthology.task.install:project ceph 2023-07-13T22:00:51.531 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'}} 2023-07-13T22:00:51.531 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b'} 2023-07-13T22:00:51.532 INFO:teuthology.task.install:Using flavor: default 2023-07-13T22:00:51.539 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-13T22:00:51.540 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-13T22:00:51.540 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-13T22:00:51.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:00:51.543 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-13T22:00:51.544 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:00:51.545 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'tag': None, 'wait_for_package': False} 2023-07-13T22:00:51.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:00:51.826 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-13T22:00:51.827 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-13T22:00:51.842 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-13T22:00:51.843 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-13T22:00:51.848 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/ 2023-07-13T22:00:51.849 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4757.g3fc0089c 2023-07-13T22:00:52.013 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-13T22:00:52.014 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:00:52.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-13T22:00:52.030 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-13T22:00:52.031 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:00:52.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-13T22:00:52.036 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-13T22:00:52.036 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:00:52.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-13T22:00:52.063 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-13T22:00:52.064 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-13T22:00:52.066 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-13T22:00:52.066 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-13T22:00:52.087 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-13T22:00:52.088 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3fc0089cc1082313a98853d48b892acdc1385b8b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-13T22:00:52.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-13T22:00:52.138 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-13T22:00:52.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-13T22:00:52.211 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-13T22:00:52.214 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-13T22:00:52.277 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2023-07-13T22:00:52.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2023-07-13T22:00:52.283 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2023-07-13T22:00:52.284 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2023-07-13T22:00:52.296 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-13T22:00:52.365 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2023-07-13T22:00:52.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2023-07-13T22:00:52.662 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:52.665 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:52.853 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:53.501 INFO:teuthology.orchestra.run.smithi121.stdout:44 files removed 2023-07-13T22:00:53.520 INFO:teuthology.orchestra.run.smithi117.stdout:44 files removed 2023-07-13T22:00:53.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-13T22:00:53.568 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-13T22:00:53.902 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:53.905 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:54.065 INFO:teuthology.orchestra.run.smithi026.stdout:44 files removed 2023-07-13T22:00:54.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-13T22:00:54.600 INFO:teuthology.orchestra.run.smithi026.stdout:Updating Subscription Management repositories. 2023-07-13T22:00:55.415 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 168 kB/s | 81 kB 00:00 2023-07-13T22:00:55.483 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 180 kB/s | 81 kB 00:00 2023-07-13T22:00:55.840 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-07-13T22:00:55.969 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 21 kB/s | 9.4 kB 00:00 2023-07-13T22:00:56.272 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 147 kB/s | 81 kB 00:00 2023-07-13T22:00:56.324 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 3.2 kB/s | 1.5 kB 00:00 2023-07-13T22:00:56.489 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 3.0 kB/s | 1.5 kB 00:00 2023-07-13T22:00:56.965 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-07-13T22:00:57.449 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 3.2 kB/s | 1.5 kB 00:00 2023-07-13T22:00:59.124 INFO:teuthology.orchestra.run.smithi117.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T22:00:59.311 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T22:01:00.282 INFO:teuthology.orchestra.run.smithi026.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-13T22:01:01.932 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 16 MB 00:01 2023-07-13T22:01:02.672 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 16 MB 00:03 2023-07-13T22:01:06.174 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 432 kB/s | 24 kB 00:00 2023-07-13T22:01:06.208 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 375 kB/s | 24 kB 00:00 2023-07-13T22:01:07.276 INFO:teuthology.orchestra.run.smithi117.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-07-13T22:01:07.687 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 MB/s | 53 MB 00:01 2023-07-13T22:01:15.344 INFO:teuthology.orchestra.run.smithi117.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-07-13T22:01:16.878 INFO:teuthology.orchestra.run.smithi026.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 43 MB/s | 47 MB 00:01 2023-07-13T22:01:20.012 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 772 kB/s | 16 MB 00:20 2023-07-13T22:01:23.459 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 407 kB/s | 24 kB 00:00 2023-07-13T22:01:24.395 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-07-13T22:01:26.325 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:26.325 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:26.339 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-13T22:01:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-13T22:01:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-13T22:01:26.348 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-13T22:01:26.354 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-13T22:01:26.481 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2023-07-13T22:01:26.493 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-07-13T22:01:26.493 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2023-07-13T22:01:26.493 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-07-13T22:01:26.493 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-13T22:01:26.494 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-13T22:01:26.495 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-13T22:01:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-13T22:01:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-13T22:01:26.496 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-13T22:01:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-13T22:01:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-13T22:01:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-13T22:01:26.497 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-13T22:01:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-13T22:01:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-13T22:01:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-13T22:01:26.498 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-13T22:01:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-13T22:01:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-13T22:01:26.499 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-13T22:01:26.500 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-13T22:01:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-13T22:01:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-13T22:01:26.501 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-13T22:01:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-13T22:01:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-13T22:01:26.502 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-13T22:01:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-13T22:01:26.503 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-13T22:01:26.503 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2023-07-13T22:01:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-13T22:01:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-13T22:01:26.504 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-13T22:01:26.504 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2023-07-13T22:01:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-13T22:01:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-13T22:01:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-13T22:01:26.505 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-13T22:01:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-13T22:01:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-13T22:01:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-13T22:01:26.506 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-13T22:01:26.507 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-13T22:01:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-13T22:01:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-13T22:01:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-13T22:01:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-13T22:01:26.508 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-13T22:01:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-13T22:01:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-13T22:01:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-13T22:01:26.509 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-13T22:01:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-13T22:01:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-13T22:01:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-13T22:01:26.511 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-13T22:01:26.512 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-13T22:01:26.513 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-13T22:01:26.514 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-13T22:01:26.515 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-13T22:01:26.516 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-13T22:01:26.517 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-13T22:01:26.518 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-13T22:01:26.519 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling module streams: 2023-07-13T22:01:26.520 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql 10 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout:Install 128 Packages 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 3 Packages 2023-07-13T22:01:26.521 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:01:26.522 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 205 M 2023-07-13T22:01:26.522 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2023-07-13T22:01:26.968 INFO:teuthology.orchestra.run.smithi117.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-07-13T22:01:28.002 INFO:teuthology.orchestra.run.smithi117.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 821 kB/s | 848 kB 00:01 2023-07-13T22:01:28.319 INFO:teuthology.orchestra.run.smithi117.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 522 kB/s | 165 kB 00:00 2023-07-13T22:01:29.180 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:29.182 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:29.197 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:29.198 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:29.199 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-13T22:01:29.200 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-13T22:01:29.203 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-13T22:01:29.206 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-13T22:01:29.207 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-13T22:01:29.359 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2023-07-13T22:01:29.373 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-07-13T22:01:29.374 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2023-07-13T22:01:29.374 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-07-13T22:01:29.374 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2023-07-13T22:01:29.374 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-13T22:01:29.375 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-13T22:01:29.375 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-13T22:01:29.376 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-13T22:01:29.376 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-13T22:01:29.377 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-13T22:01:29.377 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-13T22:01:29.377 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-13T22:01:29.377 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-13T22:01:29.378 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-13T22:01:29.378 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-13T22:01:29.378 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-13T22:01:29.379 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-13T22:01:29.380 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-13T22:01:29.381 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-13T22:01:29.381 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-13T22:01:29.381 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-13T22:01:29.381 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-13T22:01:29.382 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-13T22:01:29.382 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-13T22:01:29.382 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-13T22:01:29.383 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-13T22:01:29.383 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-13T22:01:29.384 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-13T22:01:29.384 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-13T22:01:29.384 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-13T22:01:29.384 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-13T22:01:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-13T22:01:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-13T22:01:29.385 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-13T22:01:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-13T22:01:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-13T22:01:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-13T22:01:29.386 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-13T22:01:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-13T22:01:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-13T22:01:29.387 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-13T22:01:29.387 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2023-07-13T22:01:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-13T22:01:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-13T22:01:29.388 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-13T22:01:29.389 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2023-07-13T22:01:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-13T22:01:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-13T22:01:29.389 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-13T22:01:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-13T22:01:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-13T22:01:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-13T22:01:29.390 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-13T22:01:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-13T22:01:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-13T22:01:29.391 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-13T22:01:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-13T22:01:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-13T22:01:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-13T22:01:29.392 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-13T22:01:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-13T22:01:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-13T22:01:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-13T22:01:29.393 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-13T22:01:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-13T22:01:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-13T22:01:29.394 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-13T22:01:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-13T22:01:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-13T22:01:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-13T22:01:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-13T22:01:29.395 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-13T22:01:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-13T22:01:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-13T22:01:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-13T22:01:29.396 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-13T22:01:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-13T22:01:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-13T22:01:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-13T22:01:29.397 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-13T22:01:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-13T22:01:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-13T22:01:29.398 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-13T22:01:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-13T22:01:29.399 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-13T22:01:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-13T22:01:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-13T22:01:29.400 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-13T22:01:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-13T22:01:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-13T22:01:29.401 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-13T22:01:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-13T22:01:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-13T22:01:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-13T22:01:29.402 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-13T22:01:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-13T22:01:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-13T22:01:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-13T22:01:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-13T22:01:29.403 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-13T22:01:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-13T22:01:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-13T22:01:29.404 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-13T22:01:29.405 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-13T22:01:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-13T22:01:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-13T22:01:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-13T22:01:29.411 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-13T22:01:29.412 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-13T22:01:29.413 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-13T22:01:29.414 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-13T22:01:29.415 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling module streams: 2023-07-13T22:01:29.416 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql 10 2023-07-13T22:01:29.416 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:01:29.416 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2023-07-13T22:01:29.416 INFO:teuthology.orchestra.run.smithi026.stdout:=========================================================================================================================== 2023-07-13T22:01:29.417 INFO:teuthology.orchestra.run.smithi026.stdout:Install 128 Packages 2023-07-13T22:01:29.417 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 3 Packages 2023-07-13T22:01:29.417 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:01:29.418 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 205 M 2023-07-13T22:01:29.418 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2023-07-13T22:01:31.210 INFO:teuthology.orchestra.run.smithi117.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-07-13T22:01:31.419 INFO:teuthology.orchestra.run.smithi117.stdout:(5/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 756 kB/s | 2.3 MB 00:03 2023-07-13T22:01:32.052 INFO:teuthology.orchestra.run.smithi026.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-07-13T22:01:32.291 INFO:teuthology.orchestra.run.smithi117.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-07-13T22:01:32.297 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-13T22:01:32.928 INFO:teuthology.orchestra.run.smithi026.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 970 kB/s | 848 kB 00:00 2023-07-13T22:01:33.069 INFO:teuthology.orchestra.run.smithi026.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 1.1 MB/s | 165 kB 00:00 2023-07-13T22:01:33.861 INFO:teuthology.orchestra.run.smithi026.stdout:(4/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 2.6 MB/s | 5.4 MB 00:02 2023-07-13T22:01:34.095 INFO:teuthology.orchestra.run.smithi026.stdout:(5/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-07-13T22:01:34.195 INFO:teuthology.orchestra.run.smithi026.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 4.7 MB/s | 1.5 MB 00:00 2023-07-13T22:01:34.852 INFO:teuthology.orchestra.run.smithi117.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 1.3 MB/s | 4.6 MB 00:03 2023-07-13T22:01:35.870 INFO:teuthology.orchestra.run.smithi026.stdout:(7/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 5.5 MB/s | 22 MB 00:04 2023-07-13T22:01:35.945 INFO:teuthology.orchestra.run.smithi026.stdout:(8/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 2.5 MB/s | 4.6 MB 00:01 2023-07-13T22:01:36.063 INFO:teuthology.orchestra.run.smithi026.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 209 kB/s | 24 kB 00:00 2023-07-13T22:01:36.396 INFO:teuthology.orchestra.run.smithi026.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 8.1 MB/s | 18 MB 00:02 2023-07-13T22:01:36.509 INFO:teuthology.orchestra.run.smithi026.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 2.2 MB/s | 258 kB 00:00 2023-07-13T22:01:36.595 INFO:teuthology.orchestra.run.smithi026.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 394 kB/s | 31 kB 00:00 2023-07-13T22:01:36.706 INFO:teuthology.orchestra.run.smithi026.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 6.5 MB/s | 771 kB 00:00 2023-07-13T22:01:36.796 INFO:teuthology.orchestra.run.smithi026.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.6 MB/s | 137 kB 00:00 2023-07-13T22:01:36.890 INFO:teuthology.orchestra.run.smithi026.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.4 MB/s | 139 kB 00:00 2023-07-13T22:01:37.016 INFO:teuthology.orchestra.run.smithi026.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 3.7 MB/s | 473 kB 00:00 2023-07-13T22:01:37.223 INFO:teuthology.orchestra.run.smithi026.stdout:(17/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 10 MB/s | 13 MB 00:01 2023-07-13T22:01:37.315 INFO:teuthology.orchestra.run.smithi026.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 489 kB/s | 45 kB 00:00 2023-07-13T22:01:37.398 INFO:teuthology.orchestra.run.smithi026.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.4 MB/s | 119 kB 00:00 2023-07-13T22:01:37.482 INFO:teuthology.orchestra.run.smithi026.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 2.3 MB/s | 197 kB 00:00 2023-07-13T22:01:37.582 INFO:teuthology.orchestra.run.smithi026.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 3.6 MB/s | 371 kB 00:00 2023-07-13T22:01:37.672 INFO:teuthology.orchestra.run.smithi026.stdout:(22/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 10 MB/s | 6.7 MB 00:00 2023-07-13T22:01:37.700 INFO:teuthology.orchestra.run.smithi026.stdout:(23/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 3.1 MB/s | 364 kB 00:00 2023-07-13T22:01:37.735 INFO:teuthology.orchestra.run.smithi026.stdout:(24/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.6 MB/s | 107 kB 00:00 2023-07-13T22:01:37.743 INFO:teuthology.orchestra.run.smithi117.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 2.0 MB/s | 22 MB 00:10 2023-07-13T22:01:37.759 INFO:teuthology.orchestra.run.smithi026.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.6 MB/s | 94 kB 00:00 2023-07-13T22:01:37.818 INFO:teuthology.orchestra.run.smithi117.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 325 kB/s | 24 kB 00:00 2023-07-13T22:01:37.842 INFO:teuthology.orchestra.run.smithi026.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 2.1 MB/s | 179 kB 00:00 2023-07-13T22:01:37.951 INFO:teuthology.orchestra.run.smithi026.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 213 kB/s | 23 kB 00:00 2023-07-13T22:01:38.260 INFO:teuthology.orchestra.run.smithi026.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 418 kB/s | 129 kB 00:00 2023-07-13T22:01:38.326 INFO:teuthology.orchestra.run.smithi026.stdout:(29/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 5.8 MB/s | 3.4 MB 00:00 2023-07-13T22:01:38.493 INFO:teuthology.orchestra.run.smithi026.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 7.3 MB/s | 1.7 MB 00:00 2023-07-13T22:01:38.587 INFO:teuthology.orchestra.run.smithi026.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 2.6 MB/s | 240 kB 00:00 2023-07-13T22:01:38.677 INFO:teuthology.orchestra.run.smithi026.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 518 kB/s | 47 kB 00:00 2023-07-13T22:01:38.777 INFO:teuthology.orchestra.run.smithi026.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 145 kB/s | 15 kB 00:00 2023-07-13T22:01:38.895 INFO:teuthology.orchestra.run.smithi026.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 2.2 MB/s | 258 kB 00:00 2023-07-13T22:01:38.994 INFO:teuthology.orchestra.run.smithi026.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 1.6 MB/s | 166 kB 00:00 2023-07-13T22:01:39.095 INFO:teuthology.orchestra.run.smithi026.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 180 kB/s | 18 kB 00:00 2023-07-13T22:01:39.237 INFO:teuthology.orchestra.run.smithi026.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 8.1 MB/s | 7.4 MB 00:00 2023-07-13T22:01:39.437 INFO:teuthology.orchestra.run.smithi026.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 294 kB/s | 59 kB 00:00 2023-07-13T22:01:39.546 INFO:teuthology.orchestra.run.smithi026.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 680 kB/s | 306 kB 00:00 2023-07-13T22:01:39.588 INFO:teuthology.orchestra.run.smithi026.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 500 kB/s | 75 kB 00:00 2023-07-13T22:01:39.663 INFO:teuthology.orchestra.run.smithi026.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 582 kB/s | 44 kB 00:00 2023-07-13T22:01:39.688 INFO:teuthology.orchestra.run.smithi026.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-07-13T22:01:39.748 INFO:teuthology.orchestra.run.smithi026.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-07-13T22:01:39.831 INFO:teuthology.orchestra.run.smithi026.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-07-13T22:01:39.856 INFO:teuthology.orchestra.run.smithi026.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 963 kB/s | 185 kB 00:00 2023-07-13T22:01:39.898 INFO:teuthology.orchestra.run.smithi026.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-07-13T22:01:39.950 INFO:teuthology.orchestra.run.smithi026.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-07-13T22:01:39.982 INFO:teuthology.orchestra.run.smithi026.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-07-13T22:01:39.999 INFO:teuthology.orchestra.run.smithi026.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-07-13T22:01:40.032 INFO:teuthology.orchestra.run.smithi026.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-07-13T22:01:40.091 INFO:teuthology.orchestra.run.smithi026.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 839 kB/s | 49 kB 00:00 2023-07-13T22:01:40.141 INFO:teuthology.orchestra.run.smithi026.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-07-13T22:01:40.158 INFO:teuthology.orchestra.run.smithi026.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 877 kB/s | 59 kB 00:00 2023-07-13T22:01:40.192 INFO:teuthology.orchestra.run.smithi026.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-07-13T22:01:40.234 INFO:teuthology.orchestra.run.smithi026.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.7 MB/s | 283 kB 00:00 2023-07-13T22:01:40.251 INFO:teuthology.orchestra.run.smithi026.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-07-13T22:01:40.301 INFO:teuthology.orchestra.run.smithi026.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-07-13T22:01:40.328 INFO:teuthology.orchestra.run.smithi026.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-07-13T22:01:40.360 INFO:teuthology.orchestra.run.smithi026.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2023-07-13T22:01:40.381 INFO:teuthology.orchestra.run.smithi026.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 374 kB/s | 19 kB 00:00 2023-07-13T22:01:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 410 kB/s | 24 kB 00:00 2023-07-13T22:01:40.448 INFO:teuthology.orchestra.run.smithi026.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 437 kB/s | 29 kB 00:00 2023-07-13T22:01:40.487 INFO:teuthology.orchestra.run.smithi026.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 412 kB/s | 27 kB 00:00 2023-07-13T22:01:40.512 INFO:teuthology.orchestra.run.smithi026.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-07-13T22:01:40.563 INFO:teuthology.orchestra.run.smithi026.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-07-13T22:01:40.588 INFO:teuthology.orchestra.run.smithi026.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 814 kB/s | 61 kB 00:00 2023-07-13T22:01:40.621 INFO:teuthology.orchestra.run.smithi026.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-07-13T22:01:40.648 INFO:teuthology.orchestra.run.smithi026.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-07-13T22:01:40.664 INFO:teuthology.orchestra.run.smithi026.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-07-13T22:01:40.722 INFO:teuthology.orchestra.run.smithi026.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 810 kB/s | 47 kB 00:00 2023-07-13T22:01:40.789 INFO:teuthology.orchestra.run.smithi026.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-07-13T22:01:40.814 INFO:teuthology.orchestra.run.smithi026.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-07-13T22:01:40.865 INFO:teuthology.orchestra.run.smithi026.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 567 kB/s | 43 kB 00:00 2023-07-13T22:01:40.890 INFO:teuthology.orchestra.run.smithi026.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2023-07-13T22:01:40.915 INFO:teuthology.orchestra.run.smithi026.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 963 kB/s | 48 kB 00:00 2023-07-13T22:01:40.941 INFO:teuthology.orchestra.run.smithi026.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-13T22:01:40.966 INFO:teuthology.orchestra.run.smithi026.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 850 kB/s | 43 kB 00:00 2023-07-13T22:01:40.991 INFO:teuthology.orchestra.run.smithi026.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-13T22:01:41.025 INFO:teuthology.orchestra.run.smithi026.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-13T22:01:41.067 INFO:teuthology.orchestra.run.smithi026.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-07-13T22:01:41.092 INFO:teuthology.orchestra.run.smithi026.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 280 kB/s | 19 kB 00:00 2023-07-13T22:01:41.117 INFO:teuthology.orchestra.run.smithi026.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-13T22:01:41.169 INFO:teuthology.orchestra.run.smithi026.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2023-07-13T22:01:41.193 INFO:teuthology.orchestra.run.smithi026.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 238 kB/s | 18 kB 00:00 2023-07-13T22:01:41.244 INFO:teuthology.orchestra.run.smithi117.stdout:(10/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 2.0 MB/s | 18 MB 00:08 2023-07-13T22:01:41.252 INFO:teuthology.orchestra.run.smithi026.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-07-13T22:01:41.285 INFO:teuthology.orchestra.run.smithi026.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-13T22:01:41.327 INFO:teuthology.orchestra.run.smithi026.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-13T22:01:41.372 INFO:teuthology.orchestra.run.smithi026.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 2.0 MB/s | 83 kB 00:00 2023-07-13T22:01:41.393 INFO:teuthology.orchestra.run.smithi117.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 1.7 MB/s | 258 kB 00:00 2023-07-13T22:01:41.422 INFO:teuthology.orchestra.run.smithi026.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 14 MB/s | 2.3 MB 00:00 2023-07-13T22:01:41.437 INFO:teuthology.orchestra.run.smithi026.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 893 kB/s | 60 kB 00:00 2023-07-13T22:01:41.461 INFO:teuthology.orchestra.run.smithi117.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 465 kB/s | 31 kB 00:00 2023-07-13T22:01:41.461 INFO:teuthology.orchestra.run.smithi026.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-13T22:01:41.487 INFO:teuthology.orchestra.run.smithi026.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-13T22:01:41.507 INFO:teuthology.orchestra.run.smithi026.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 880 kB/s | 39 kB 00:00 2023-07-13T22:01:41.548 INFO:teuthology.orchestra.run.smithi026.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 10 MB/s | 644 kB 00:00 2023-07-13T22:01:41.573 INFO:teuthology.orchestra.run.smithi026.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-13T22:01:41.595 INFO:teuthology.orchestra.run.smithi117.stdout:(13/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 2.0 MB/s | 13 MB 00:06 2023-07-13T22:01:41.599 INFO:teuthology.orchestra.run.smithi026.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-13T22:01:41.641 INFO:teuthology.orchestra.run.smithi026.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-13T22:01:41.661 INFO:teuthology.orchestra.run.smithi117.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 2.0 MB/s | 137 kB 00:00 2023-07-13T22:01:41.719 INFO:teuthology.orchestra.run.smithi117.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 2.3 MB/s | 139 kB 00:00 2023-07-13T22:01:41.722 INFO:teuthology.orchestra.run.smithi026.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 672 kB/s | 54 kB 00:00 2023-07-13T22:01:41.774 INFO:teuthology.orchestra.run.smithi026.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.8 MB/s | 259 kB 00:00 2023-07-13T22:01:41.795 INFO:teuthology.orchestra.run.smithi117.stdout:(16/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 2.3 MB/s | 771 kB 00:00 2023-07-13T22:01:41.825 INFO:teuthology.orchestra.run.smithi026.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-13T22:01:41.869 INFO:teuthology.orchestra.run.smithi026.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 936 kB/s | 39 kB 00:00 2023-07-13T22:01:41.936 INFO:teuthology.orchestra.run.smithi026.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-13T22:01:41.936 INFO:teuthology.orchestra.run.smithi117.stdout:(17/131): libradosstriper1-18.0.0-4757.g3fc0089 2.1 MB/s | 473 kB 00:00 2023-07-13T22:01:41.986 INFO:teuthology.orchestra.run.smithi026.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-13T22:01:42.003 INFO:teuthology.orchestra.run.smithi117.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 672 kB/s | 45 kB 00:00 2023-07-13T22:01:42.026 INFO:teuthology.orchestra.run.smithi026.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-13T22:01:42.068 INFO:teuthology.orchestra.run.smithi026.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-13T22:01:42.070 INFO:teuthology.orchestra.run.smithi117.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.7 MB/s | 119 kB 00:00 2023-07-13T22:01:42.129 INFO:teuthology.orchestra.run.smithi026.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-13T22:01:42.187 INFO:teuthology.orchestra.run.smithi117.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 1.6 MB/s | 197 kB 00:00 2023-07-13T22:01:42.302 INFO:teuthology.orchestra.run.smithi026.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 19 MB/s | 14 MB 00:00 2023-07-13T22:01:42.344 INFO:teuthology.orchestra.run.smithi026.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 644 kB/s | 27 kB 00:00 2023-07-13T22:01:42.371 INFO:teuthology.orchestra.run.smithi117.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.0 MB/s | 371 kB 00:00 2023-07-13T22:01:42.394 INFO:teuthology.orchestra.run.smithi026.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-13T22:01:42.436 INFO:teuthology.orchestra.run.smithi026.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-07-13T22:01:42.519 INFO:teuthology.orchestra.run.smithi026.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-07-13T22:01:42.554 INFO:teuthology.orchestra.run.smithi117.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 1.9 MB/s | 364 kB 00:00 2023-07-13T22:01:42.578 INFO:teuthology.orchestra.run.smithi026.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-07-13T22:01:42.630 INFO:teuthology.orchestra.run.smithi117.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.4 MB/s | 107 kB 00:00 2023-07-13T22:01:42.695 INFO:teuthology.orchestra.run.smithi026.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 14 MB/s | 3.7 MB 00:00 2023-07-13T22:01:42.723 INFO:teuthology.orchestra.run.smithi117.stdout:(24/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-13T22:01:42.729 INFO:teuthology.orchestra.run.smithi026.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2023-07-13T22:01:42.771 INFO:teuthology.orchestra.run.smithi026.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 851 kB/s | 64 kB 00:00 2023-07-13T22:01:42.796 INFO:teuthology.orchestra.run.smithi026.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.8 MB/s | 126 kB 00:00 2023-07-13T22:01:42.822 INFO:teuthology.orchestra.run.smithi026.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-07-13T22:01:42.847 INFO:teuthology.orchestra.run.smithi026.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-13T22:01:42.897 INFO:teuthology.orchestra.run.smithi026.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-07-13T22:01:42.955 INFO:teuthology.orchestra.run.smithi026.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-07-13T22:01:43.056 INFO:teuthology.orchestra.run.smithi026.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 23 MB/s | 4.9 MB 00:00 2023-07-13T22:01:43.141 INFO:teuthology.orchestra.run.smithi026.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 2.3 MB/s | 202 kB 00:00 2023-07-13T22:01:43.223 INFO:teuthology.orchestra.run.smithi026.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.2 MB/s | 354 kB 00:00 2023-07-13T22:01:43.312 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:43.313 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-13T22:01:43.315 INFO:teuthology.orchestra.run.smithi026.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-07-13T22:01:43.327 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:43.328 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-13T22:01:43.329 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-13T22:01:43.330 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-13T22:01:43.332 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-13T22:01:43.335 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-13T22:01:43.336 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-13T22:01:43.382 INFO:teuthology.orchestra.run.smithi026.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-07-13T22:01:43.467 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-07-13T22:01:43.479 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================================================== 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================================================== 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-13T22:01:43.480 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.4 k 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 5.4 M 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 848 k 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 165 k 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 1.5 M 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 129 k 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 1.7 M 2023-07-13T22:01:43.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 7.4 M 2023-07-13T22:01:43.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 47 k 2023-07-13T22:01:43.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 13 M 2023-07-13T22:01:43.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 48 M 2023-07-13T22:01:43.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 258 k 2023-07-13T22:01:43.482 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 166 k 2023-07-13T22:01:43.483 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 258 k 2023-07-13T22:01:43.483 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 18 k 2023-07-13T22:01:43.484 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-13T22:01:43.484 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-13T22:01:43.484 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-13T22:01:43.485 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 31 k 2023-07-13T22:01:43.485 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 771 k 2023-07-13T22:01:43.485 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 139 k 2023-07-13T22:01:43.485 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 197 k 2023-07-13T22:01:43.486 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 371 k 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 364 k 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 107 k 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 94 k 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.4 M 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 179 k 2023-07-13T22:01:43.487 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.7 M 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 3.6 M 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-13T22:01:43.488 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 22 M 2023-07-13T22:01:43.489 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 23 k 2023-07-13T22:01:43.489 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 2.3 M 2023-07-13T22:01:43.489 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 240 k 2023-07-13T22:01:43.489 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 4.6 M 2023-07-13T22:01:43.489 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 18 M 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4757.g3fc0089c.el8 ceph-noarch 15 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 24 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-13T22:01:43.490 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 137 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-13T22:01:43.491 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-13T22:01:43.492 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 473 k 2023-07-13T22:01:43.493 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-13T22:01:43.494 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 6.7 M 2023-07-13T22:01:43.494 INFO:teuthology.orchestra.run.smithi121.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-13T22:01:43.494 INFO:teuthology.orchestra.run.smithi121.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-13T22:01:43.494 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-13T22:01:43.494 INFO:teuthology.orchestra.run.smithi121.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-13T22:01:43.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 45 k 2023-07-13T22:01:43.496 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:18.0.0-4757.g3fc0089c.el8 ceph 119 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-13T22:01:43.497 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-13T22:01:43.498 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-13T22:01:43.499 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-13T22:01:43.500 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-13T22:01:43.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-13T22:01:43.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-13T22:01:43.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-13T22:01:43.501 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-13T22:01:43.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-13T22:01:43.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-13T22:01:43.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-13T22:01:43.502 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-13T22:01:43.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-13T22:01:43.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-13T22:01:43.503 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-13T22:01:43.504 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-13T22:01:43.504 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-13T22:01:43.504 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-13T22:01:43.505 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-13T22:01:43.505 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-13T22:01:43.505 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-13T22:01:43.506 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-13T22:01:43.506 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-13T22:01:43.506 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-13T22:01:43.507 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-13T22:01:43.507 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-13T22:01:43.508 INFO:teuthology.orchestra.run.smithi121.stdout:Enabling module streams: 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql 10 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================================================== 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout:Install 128 Packages 2023-07-13T22:01:43.509 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 3 Packages 2023-07-13T22:01:43.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:01:43.510 INFO:teuthology.orchestra.run.smithi026.stdout:(126/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-07-13T22:01:43.511 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 205 M 2023-07-13T22:01:43.511 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-07-13T22:01:43.543 INFO:teuthology.orchestra.run.smithi026.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-07-13T22:01:43.797 INFO:teuthology.orchestra.run.smithi117.stdout:(25/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 3.2 MB/s | 3.4 MB 00:01 2023-07-13T22:01:43.869 INFO:teuthology.orchestra.run.smithi121.stdout:(1/131): ceph-18.0.0-4757.g3fc0089c.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2023-07-13T22:01:43.947 INFO:teuthology.orchestra.run.smithi117.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-13T22:01:44.056 INFO:teuthology.orchestra.run.smithi117.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 213 kB/s | 23 kB 00:00 2023-07-13T22:01:44.173 INFO:teuthology.orchestra.run.smithi117.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.1 MB/s | 129 kB 00:00 2023-07-13T22:01:44.216 INFO:teuthology.orchestra.run.smithi026.stdout:(128/131): librados2-18.0.0-4757.g3fc0089c.el8. 5.1 MB/s | 3.7 MB 00:00 2023-07-13T22:01:44.277 INFO:teuthology.orchestra.run.smithi026.stdout:(129/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 4.9 MB/s | 3.6 MB 00:00 2023-07-13T22:01:44.299 INFO:teuthology.orchestra.run.smithi117.stdout:(29/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 2.7 MB/s | 6.7 MB 00:02 2023-07-13T22:01:44.325 INFO:teuthology.orchestra.run.smithi026.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 7.2 MB/s | 798 kB 00:00 2023-07-13T22:01:44.656 INFO:teuthology.orchestra.run.smithi121.stdout:(2/131): ceph-fuse-18.0.0-4757.g3fc0089c.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-13T22:01:44.690 INFO:teuthology.orchestra.run.smithi117.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 3.3 MB/s | 1.7 MB 00:00 2023-07-13T22:01:44.769 INFO:teuthology.orchestra.run.smithi121.stdout:(3/131): ceph-immutable-object-cache-18.0.0-475 1.4 MB/s | 165 kB 00:00 2023-07-13T22:01:44.815 INFO:teuthology.orchestra.run.smithi117.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 1.9 MB/s | 240 kB 00:00 2023-07-13T22:01:44.883 INFO:teuthology.orchestra.run.smithi117.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 711 kB/s | 47 kB 00:00 2023-07-13T22:01:44.942 INFO:teuthology.orchestra.run.smithi117.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 249 kB/s | 15 kB 00:00 2023-07-13T22:01:45.058 INFO:teuthology.orchestra.run.smithi117.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 2.2 MB/s | 258 kB 00:00 2023-07-13T22:01:45.142 INFO:teuthology.orchestra.run.smithi117.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 2.0 MB/s | 166 kB 00:00 2023-07-13T22:01:45.209 INFO:teuthology.orchestra.run.smithi117.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 269 kB/s | 18 kB 00:00 2023-07-13T22:01:45.359 INFO:teuthology.orchestra.run.smithi117.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.0 MB/s | 306 kB 00:00 2023-07-13T22:01:45.393 INFO:teuthology.orchestra.run.smithi117.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-07-13T22:01:45.426 INFO:teuthology.orchestra.run.smithi117.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2023-07-13T22:01:45.502 INFO:teuthology.orchestra.run.smithi117.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2023-07-13T22:01:45.593 INFO:teuthology.orchestra.run.smithi117.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 477 kB/s | 44 kB 00:00 2023-07-13T22:01:45.594 INFO:teuthology.orchestra.run.smithi121.stdout:(4/131): ceph-mds-18.0.0-4757.g3fc0089c.el8.x86 2.8 MB/s | 2.3 MB 00:00 2023-07-13T22:01:45.635 INFO:teuthology.orchestra.run.smithi117.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 4.4 MB/s | 185 kB 00:00 2023-07-13T22:01:45.677 INFO:teuthology.orchestra.run.smithi117.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 793 kB/s | 33 kB 00:00 2023-07-13T22:01:45.719 INFO:teuthology.orchestra.run.smithi117.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2023-07-13T22:01:45.761 INFO:teuthology.orchestra.run.smithi117.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-07-13T22:01:45.828 INFO:teuthology.orchestra.run.smithi117.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-07-13T22:01:45.870 INFO:teuthology.orchestra.run.smithi117.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-07-13T22:01:45.911 INFO:teuthology.orchestra.run.smithi121.stdout:(5/131): ceph-base-18.0.0-4757.g3fc0089c.el8.x8 2.4 MB/s | 5.4 MB 00:02 2023-07-13T22:01:45.913 INFO:teuthology.orchestra.run.smithi117.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-07-13T22:01:45.954 INFO:teuthology.orchestra.run.smithi117.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 423 kB/s | 18 kB 00:00 2023-07-13T22:01:46.061 INFO:teuthology.orchestra.run.smithi121.stdout:(6/131): ceph-mgr-18.0.0-4757.g3fc0089c.el8.x86 3.3 MB/s | 1.5 MB 00:00 2023-07-13T22:01:46.071 INFO:teuthology.orchestra.run.smithi117.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 10 MB/s | 1.2 MB 00:00 2023-07-13T22:01:46.113 INFO:teuthology.orchestra.run.smithi117.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-07-13T22:01:46.155 INFO:teuthology.orchestra.run.smithi117.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-07-13T22:01:46.197 INFO:teuthology.orchestra.run.smithi117.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-07-13T22:01:46.239 INFO:teuthology.orchestra.run.smithi117.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 6.6 MB/s | 283 kB 00:00 2023-07-13T22:01:46.281 INFO:teuthology.orchestra.run.smithi117.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2023-07-13T22:01:46.324 INFO:teuthology.orchestra.run.smithi117.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 809 kB/s | 34 kB 00:00 2023-07-13T22:01:46.365 INFO:teuthology.orchestra.run.smithi117.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-07-13T22:01:46.407 INFO:teuthology.orchestra.run.smithi117.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 1.4 MB/s | 61 kB 00:00 2023-07-13T22:01:46.449 INFO:teuthology.orchestra.run.smithi117.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 451 kB/s | 19 kB 00:00 2023-07-13T22:01:46.491 INFO:teuthology.orchestra.run.smithi117.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 574 kB/s | 24 kB 00:00 2023-07-13T22:01:46.533 INFO:teuthology.orchestra.run.smithi117.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 702 kB/s | 29 kB 00:00 2023-07-13T22:01:46.583 INFO:teuthology.orchestra.run.smithi117.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 551 kB/s | 27 kB 00:00 2023-07-13T22:01:46.625 INFO:teuthology.orchestra.run.smithi117.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 6.0 MB/s | 255 kB 00:00 2023-07-13T22:01:46.667 INFO:teuthology.orchestra.run.smithi117.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 5.9 MB/s | 251 kB 00:00 2023-07-13T22:01:46.709 INFO:teuthology.orchestra.run.smithi117.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-07-13T22:01:46.751 INFO:teuthology.orchestra.run.smithi117.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-07-13T22:01:46.802 INFO:teuthology.orchestra.run.smithi117.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-07-13T22:01:46.911 INFO:teuthology.orchestra.run.smithi121.stdout:(7/131): ceph-mon-18.0.0-4757.g3fc0089c.el8.x86 4.6 MB/s | 4.6 MB 00:00 2023-07-13T22:01:46.955 INFO:teuthology.orchestra.run.smithi117.stdout:(68/131): ceph-mgr-diskprediction-local-18.0.0- 2.8 MB/s | 7.4 MB 00:02 2023-07-13T22:01:46.985 INFO:teuthology.orchestra.run.smithi117.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.3 MB/s | 70 kB 00:00 2023-07-13T22:01:47.036 INFO:teuthology.orchestra.run.smithi117.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-07-13T22:01:47.102 INFO:teuthology.orchestra.run.smithi117.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 400 kB/s | 47 kB 00:00 2023-07-13T22:01:47.128 INFO:teuthology.orchestra.run.smithi117.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-07-13T22:01:47.153 INFO:teuthology.orchestra.run.smithi117.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-07-13T22:01:47.203 INFO:teuthology.orchestra.run.smithi117.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2023-07-13T22:01:47.229 INFO:teuthology.orchestra.run.smithi117.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 639 kB/s | 48 kB 00:00 2023-07-13T22:01:47.271 INFO:teuthology.orchestra.run.smithi117.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-13T22:01:47.329 INFO:teuthology.orchestra.run.smithi117.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 692 kB/s | 87 kB 00:00 2023-07-13T22:01:47.388 INFO:teuthology.orchestra.run.smithi117.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 932 kB/s | 109 kB 00:00 2023-07-13T22:01:47.413 INFO:teuthology.orchestra.run.smithi117.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-07-13T22:01:47.455 INFO:teuthology.orchestra.run.smithi117.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-07-13T22:01:47.473 INFO:teuthology.orchestra.run.smithi117.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 318 kB/s | 19 kB 00:00 2023-07-13T22:01:47.548 INFO:teuthology.orchestra.run.smithi117.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-07-13T22:01:47.581 INFO:teuthology.orchestra.run.smithi117.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-07-13T22:01:47.648 INFO:teuthology.orchestra.run.smithi117.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 179 kB/s | 18 kB 00:00 2023-07-13T22:01:47.707 INFO:teuthology.orchestra.run.smithi117.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-07-13T22:01:47.749 INFO:teuthology.orchestra.run.smithi117.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-07-13T22:01:47.882 INFO:teuthology.orchestra.run.smithi117.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 13 MB/s | 2.3 MB 00:00 2023-07-13T22:01:47.899 INFO:teuthology.orchestra.run.smithi117.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 481 kB/s | 72 kB 00:00 2023-07-13T22:01:47.950 INFO:teuthology.orchestra.run.smithi117.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-07-13T22:01:47.975 INFO:teuthology.orchestra.run.smithi117.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 798 kB/s | 60 kB 00:00 2023-07-13T22:01:48.000 INFO:teuthology.orchestra.run.smithi117.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-13T22:01:48.034 INFO:teuthology.orchestra.run.smithi117.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 984 kB/s | 58 kB 00:00 2023-07-13T22:01:48.067 INFO:teuthology.orchestra.run.smithi117.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 577 kB/s | 39 kB 00:00 2023-07-13T22:01:48.134 INFO:teuthology.orchestra.run.smithi117.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.3 MB/s | 644 kB 00:00 2023-07-13T22:01:48.160 INFO:teuthology.orchestra.run.smithi117.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.8 MB/s | 171 kB 00:00 2023-07-13T22:01:48.185 INFO:teuthology.orchestra.run.smithi117.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-13T22:01:48.235 INFO:teuthology.orchestra.run.smithi117.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-07-13T22:01:48.285 INFO:teuthology.orchestra.run.smithi117.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-13T22:01:48.327 INFO:teuthology.orchestra.run.smithi117.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-13T22:01:48.519 INFO:teuthology.orchestra.run.smithi117.stdout:(100/131): python3-scipy-1.0.0-21.module+el8.5. 38 MB/s | 14 MB 00:00 2023-07-13T22:01:48.544 INFO:teuthology.orchestra.run.smithi117.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2023-07-13T22:01:48.570 INFO:teuthology.orchestra.run.smithi117.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 777 kB/s | 39 kB 00:00 2023-07-13T22:01:48.612 INFO:teuthology.orchestra.run.smithi117.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-13T22:01:48.646 INFO:teuthology.orchestra.run.smithi117.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-07-13T22:01:48.671 INFO:teuthology.orchestra.run.smithi117.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-13T22:01:48.696 INFO:teuthology.orchestra.run.smithi117.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-13T22:01:48.730 INFO:teuthology.orchestra.run.smithi117.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-13T22:01:48.772 INFO:teuthology.orchestra.run.smithi117.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 638 kB/s | 27 kB 00:00 2023-07-13T22:01:48.814 INFO:teuthology.orchestra.run.smithi117.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-13T22:01:48.889 INFO:teuthology.orchestra.run.smithi117.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-07-13T22:01:48.914 INFO:teuthology.orchestra.run.smithi117.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 438 kB/s | 44 kB 00:00 2023-07-13T22:01:48.965 INFO:teuthology.orchestra.run.smithi117.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-07-13T22:01:49.028 INFO:teuthology.orchestra.run.smithi121.stdout:(8/131): ceph-common-18.0.0-4757.g3fc0089c.el8. 4.1 MB/s | 22 MB 00:05 2023-07-13T22:01:49.032 INFO:teuthology.orchestra.run.smithi117.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 26 MB/s | 3.7 MB 00:00 2023-07-13T22:01:49.057 INFO:teuthology.orchestra.run.smithi117.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-07-13T22:01:49.082 INFO:teuthology.orchestra.run.smithi117.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-13T22:01:49.109 INFO:teuthology.orchestra.run.smithi117.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-13T22:01:49.133 INFO:teuthology.orchestra.run.smithi117.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 399 kB/s | 20 kB 00:00 2023-07-13T22:01:49.136 INFO:teuthology.orchestra.run.smithi121.stdout:(9/131): ceph-selinux-18.0.0-4757.g3fc0089c.el8 225 kB/s | 24 kB 00:00 2023-07-13T22:01:49.158 INFO:teuthology.orchestra.run.smithi117.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-13T22:01:49.192 INFO:teuthology.orchestra.run.smithi117.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-13T22:01:49.234 INFO:teuthology.orchestra.run.smithi117.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-13T22:01:49.342 INFO:teuthology.orchestra.run.smithi117.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 26 MB/s | 4.9 MB 00:00 2023-07-13T22:01:49.384 INFO:teuthology.orchestra.run.smithi117.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-13T22:01:49.459 INFO:teuthology.orchestra.run.smithi117.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-07-13T22:01:49.485 INFO:teuthology.orchestra.run.smithi117.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.5 MB/s | 354 kB 00:00 2023-07-13T22:01:49.552 INFO:teuthology.orchestra.run.smithi117.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-07-13T22:01:49.587 INFO:teuthology.orchestra.run.smithi121.stdout:(10/131): ceph-radosgw-18.0.0-4757.g3fc0089c.el 5.0 MB/s | 13 MB 00:02 2023-07-13T22:01:49.627 INFO:teuthology.orchestra.run.smithi117.stdout:(126/131): postgresql-server-10.21-2.module+el8 30 MB/s | 5.1 MB 00:00 2023-07-13T22:01:49.677 INFO:teuthology.orchestra.run.smithi117.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-13T22:01:49.695 INFO:teuthology.orchestra.run.smithi121.stdout:(11/131): cephfs-mirror-18.0.0-4757.g3fc0089c.e 2.3 MB/s | 258 kB 00:00 2023-07-13T22:01:49.796 INFO:teuthology.orchestra.run.smithi121.stdout:(12/131): libcephfs-devel-18.0.0-4757.g3fc0089c 310 kB/s | 31 kB 00:00 2023-07-13T22:01:50.229 INFO:teuthology.orchestra.run.smithi121.stdout:(13/131): libcephfs2-18.0.0-4757.g3fc0089c.el8. 1.7 MB/s | 771 kB 00:00 2023-07-13T22:01:50.355 INFO:teuthology.orchestra.run.smithi121.stdout:(14/131): libcephsqlite-18.0.0-4757.g3fc0089c.e 1.1 MB/s | 137 kB 00:00 2023-07-13T22:01:50.458 INFO:teuthology.orchestra.run.smithi026.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 3.3 MB/s | 48 MB 00:14 2023-07-13T22:01:50.462 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:01:50.462 INFO:teuthology.orchestra.run.smithi026.stdout:Total 9.7 MB/s | 205 MB 00:21 2023-07-13T22:01:50.480 INFO:teuthology.orchestra.run.smithi121.stdout:(15/131): librados-devel-18.0.0-4757.g3fc0089c. 1.1 MB/s | 139 kB 00:00 2023-07-13T22:01:50.697 INFO:teuthology.orchestra.run.smithi121.stdout:(16/131): libradosstriper1-18.0.0-4757.g3fc0089 2.1 MB/s | 473 kB 00:00 2023-07-13T22:01:51.694 INFO:teuthology.orchestra.run.smithi117.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 1.8 MB/s | 3.6 MB 00:02 2023-07-13T22:01:51.752 INFO:teuthology.orchestra.run.smithi117.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-13T22:01:51.802 INFO:teuthology.orchestra.run.smithi117.stdout:(130/131): librados2-18.0.0-4757.g3fc0089c.el8. 1.7 MB/s | 3.7 MB 00:02 2023-07-13T22:01:52.080 INFO:teuthology.orchestra.run.smithi121.stdout:(17/131): ceph-osd-18.0.0-4757.g3fc0089c.el8.x8 2.9 MB/s | 18 MB 00:06 2023-07-13T22:01:52.150 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2023-07-13T22:01:52.165 INFO:teuthology.orchestra.run.smithi121.stdout:(18/131): python3-ceph-argparse-18.0.0-4757.g3f 539 kB/s | 45 kB 00:00 2023-07-13T22:01:52.239 INFO:teuthology.orchestra.run.smithi121.stdout:(19/131): python3-ceph-common-18.0.0-4757.g3fc0 1.5 MB/s | 119 kB 00:00 2023-07-13T22:01:52.296 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2023-07-13T22:01:52.296 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2023-07-13T22:01:52.397 INFO:teuthology.orchestra.run.smithi121.stdout:(20/131): python3-cephfs-18.0.0-4757.g3fc0089c. 1.2 MB/s | 197 kB 00:00 2023-07-13T22:01:52.573 INFO:teuthology.orchestra.run.smithi121.stdout:(21/131): python3-rados-18.0.0-4757.g3fc0089c.e 2.1 MB/s | 371 kB 00:00 2023-07-13T22:01:52.748 INFO:teuthology.orchestra.run.smithi121.stdout:(22/131): python3-rbd-18.0.0-4757.g3fc0089c.el8 2.0 MB/s | 364 kB 00:00 2023-07-13T22:01:52.840 INFO:teuthology.orchestra.run.smithi121.stdout:(23/131): python3-rgw-18.0.0-4757.g3fc0089c.el8 1.1 MB/s | 107 kB 00:00 2023-07-13T22:01:52.932 INFO:teuthology.orchestra.run.smithi121.stdout:(24/131): librgw2-18.0.0-4757.g3fc0089c.el8.x86 3.0 MB/s | 6.7 MB 00:02 2023-07-13T22:01:52.957 INFO:teuthology.orchestra.run.smithi121.stdout:(25/131): rbd-fuse-18.0.0-4757.g3fc0089c.el8.x8 809 kB/s | 94 kB 00:00 2023-07-13T22:01:53.083 INFO:teuthology.orchestra.run.smithi121.stdout:(26/131): rbd-nbd-18.0.0-4757.g3fc0089c.el8.x86 1.4 MB/s | 179 kB 00:00 2023-07-13T22:01:53.150 INFO:teuthology.orchestra.run.smithi121.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4757.g 347 kB/s | 23 kB 00:00 2023-07-13T22:01:53.217 INFO:teuthology.orchestra.run.smithi121.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4757.g3fc0089 1.9 MB/s | 129 kB 00:00 2023-07-13T22:01:53.767 INFO:teuthology.orchestra.run.smithi121.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4757.g3fc00 3.1 MB/s | 1.7 MB 00:00 2023-07-13T22:01:53.925 INFO:teuthology.orchestra.run.smithi121.stdout:(30/131): rbd-mirror-18.0.0-4757.g3fc0089c.el8. 3.5 MB/s | 3.4 MB 00:00 2023-07-13T22:01:53.969 INFO:teuthology.orchestra.run.smithi117.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 3.0 MB/s | 48 MB 00:16 2023-07-13T22:01:53.975 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:01:53.975 INFO:teuthology.orchestra.run.smithi117.stdout:Total 7.5 MB/s | 205 MB 00:27 2023-07-13T22:01:54.026 INFO:teuthology.orchestra.run.smithi121.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4757.g3f 2.4 MB/s | 240 kB 00:00 2023-07-13T22:01:54.092 INFO:teuthology.orchestra.run.smithi121.stdout:(32/131): ceph-mgr-rook-18.0.0-4757.g3fc0089c.e 712 kB/s | 47 kB 00:00 2023-07-13T22:01:54.160 INFO:teuthology.orchestra.run.smithi121.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4757.g3 218 kB/s | 15 kB 00:00 2023-07-13T22:01:54.193 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2023-07-13T22:01:54.198 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2023-07-13T22:01:54.276 INFO:teuthology.orchestra.run.smithi121.stdout:(34/131): ceph-volume-18.0.0-4757.g3fc0089c.el8 2.2 MB/s | 258 kB 00:00 2023-07-13T22:01:54.352 INFO:teuthology.orchestra.run.smithi121.stdout:(35/131): cephadm-18.0.0-4757.g3fc0089c.el8.noa 2.2 MB/s | 166 kB 00:00 2023-07-13T22:01:54.427 INFO:teuthology.orchestra.run.smithi121.stdout:(36/131): cephfs-top-18.0.0-4757.g3fc0089c.el8. 239 kB/s | 18 kB 00:00 2023-07-13T22:01:54.810 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2023-07-13T22:01:54.930 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2023-07-13T22:01:54.930 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2023-07-13T22:01:54.980 INFO:teuthology.orchestra.run.smithi121.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 554 kB/s | 306 kB 00:00 2023-07-13T22:01:55.086 INFO:teuthology.orchestra.run.smithi121.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 555 kB/s | 59 kB 00:00 2023-07-13T22:01:55.178 INFO:teuthology.orchestra.run.smithi121.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 821 kB/s | 75 kB 00:00 2023-07-13T22:01:55.331 INFO:teuthology.orchestra.run.smithi121.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-07-13T22:01:55.396 INFO:teuthology.orchestra.run.smithi121.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 655 kB/s | 44 kB 00:00 2023-07-13T22:01:55.463 INFO:teuthology.orchestra.run.smithi121.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-07-13T22:01:55.521 INFO:teuthology.orchestra.run.smithi121.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 565 kB/s | 33 kB 00:00 2023-07-13T22:01:55.580 INFO:teuthology.orchestra.run.smithi121.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-07-13T22:01:55.655 INFO:teuthology.orchestra.run.smithi121.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.3 MB/s | 173 kB 00:00 2023-07-13T22:01:55.747 INFO:teuthology.orchestra.run.smithi121.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-07-13T22:01:55.822 INFO:teuthology.orchestra.run.smithi121.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-07-13T22:01:55.890 INFO:teuthology.orchestra.run.smithi121.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-07-13T22:01:55.948 INFO:teuthology.orchestra.run.smithi121.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 302 kB/s | 18 kB 00:00 2023-07-13T22:01:56.041 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2023-07-13T22:01:56.045 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2023-07-13T22:01:56.090 INFO:teuthology.orchestra.run.smithi121.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2023-07-13T22:01:56.140 INFO:teuthology.orchestra.run.smithi121.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 977 kB/s | 49 kB 00:00 2023-07-13T22:01:56.191 INFO:teuthology.orchestra.run.smithi121.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-07-13T22:01:56.274 INFO:teuthology.orchestra.run.smithi121.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 724 kB/s | 60 kB 00:00 2023-07-13T22:01:56.359 INFO:teuthology.orchestra.run.smithi121.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-07-13T22:01:56.408 INFO:teuthology.orchestra.run.smithi121.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 312 kB/s | 16 kB 00:00 2023-07-13T22:01:56.458 INFO:teuthology.orchestra.run.smithi121.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 675 kB/s | 34 kB 00:00 2023-07-13T22:01:56.525 INFO:teuthology.orchestra.run.smithi121.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-07-13T22:01:56.593 INFO:teuthology.orchestra.run.smithi121.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 923 kB/s | 61 kB 00:00 2023-07-13T22:01:56.651 INFO:teuthology.orchestra.run.smithi121.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 321 kB/s | 19 kB 00:00 2023-07-13T22:01:56.718 INFO:teuthology.orchestra.run.smithi121.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 360 kB/s | 24 kB 00:00 2023-07-13T22:01:56.768 INFO:teuthology.orchestra.run.smithi121.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 587 kB/s | 29 kB 00:00 2023-07-13T22:01:56.844 INFO:teuthology.orchestra.run.smithi121.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 367 kB/s | 27 kB 00:00 2023-07-13T22:01:56.919 INFO:teuthology.orchestra.run.smithi121.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-07-13T22:01:56.994 INFO:teuthology.orchestra.run.smithi121.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-07-13T22:01:57.079 INFO:teuthology.orchestra.run.smithi121.stdout:(65/131): ceph-mgr-diskprediction-local-18.0.0- 2.2 MB/s | 7.4 MB 00:03 2023-07-13T22:01:57.112 INFO:teuthology.orchestra.run.smithi121.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 523 kB/s | 61 kB 00:00 2023-07-13T22:01:57.179 INFO:teuthology.orchestra.run.smithi121.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 344 kB/s | 23 kB 00:00 2023-07-13T22:01:57.341 INFO:teuthology.orchestra.run.smithi121.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-07-13T22:01:57.371 INFO:teuthology.orchestra.run.smithi121.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 292 kB/s | 85 kB 00:00 2023-07-13T22:01:57.396 INFO:teuthology.orchestra.run.smithi121.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-07-13T22:01:57.455 INFO:teuthology.orchestra.run.smithi121.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-07-13T22:01:57.497 INFO:teuthology.orchestra.run.smithi121.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-13T22:01:57.541 INFO:teuthology.orchestra.run.smithi121.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-13T22:01:57.589 INFO:teuthology.orchestra.run.smithi121.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-13T22:01:57.631 INFO:teuthology.orchestra.run.smithi121.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 936 kB/s | 39 kB 00:00 2023-07-13T22:01:57.681 INFO:teuthology.orchestra.run.smithi121.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 14 MB/s | 688 kB 00:00 2023-07-13T22:01:57.731 INFO:teuthology.orchestra.run.smithi121.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-07-13T22:01:57.874 INFO:teuthology.orchestra.run.smithi121.stdout:(78/131): python3-scipy-1.0.0-21.module+el8.5.0 27 MB/s | 14 MB 00:00 2023-07-13T22:01:57.891 INFO:teuthology.orchestra.run.smithi121.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 990 kB/s | 157 kB 00:00 2023-07-13T22:01:57.917 INFO:teuthology.orchestra.run.smithi121.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-13T22:01:57.949 INFO:teuthology.orchestra.run.smithi121.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-13T22:01:57.991 INFO:teuthology.orchestra.run.smithi121.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 638 kB/s | 27 kB 00:00 2023-07-13T22:01:58.041 INFO:teuthology.orchestra.run.smithi121.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-07-13T22:01:58.142 INFO:teuthology.orchestra.run.smithi121.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-07-13T22:01:58.167 INFO:teuthology.orchestra.run.smithi121.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 351 kB/s | 44 kB 00:00 2023-07-13T22:01:58.209 INFO:teuthology.orchestra.run.smithi121.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-07-13T22:01:58.251 INFO:teuthology.orchestra.run.smithi121.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-13T22:01:58.310 INFO:teuthology.orchestra.run.smithi121.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 22 MB/s | 3.7 MB 00:00 2023-07-13T22:01:58.335 INFO:teuthology.orchestra.run.smithi121.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-07-13T22:01:58.352 INFO:teuthology.orchestra.run.smithi121.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 3.0 MB/s | 126 kB 00:00 2023-07-13T22:01:58.369 INFO:teuthology.orchestra.run.smithi121.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 596 kB/s | 20 kB 00:00 2023-07-13T22:01:58.394 INFO:teuthology.orchestra.run.smithi121.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-07-13T22:01:58.428 INFO:teuthology.orchestra.run.smithi121.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-13T22:01:58.470 INFO:teuthology.orchestra.run.smithi121.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-13T22:01:58.562 INFO:teuthology.orchestra.run.smithi121.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-07-13T22:01:58.604 INFO:teuthology.orchestra.run.smithi121.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-13T22:01:58.671 INFO:teuthology.orchestra.run.smithi121.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-07-13T22:01:58.696 INFO:teuthology.orchestra.run.smithi121.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-07-13T22:01:58.746 INFO:teuthology.orchestra.run.smithi121.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 16 MB/s | 808 kB 00:00 2023-07-13T22:01:58.838 INFO:teuthology.orchestra.run.smithi121.stdout:(100/131): postgresql-server-10.21-2.module+el8 30 MB/s | 5.1 MB 00:00 2023-07-13T22:01:58.905 INFO:teuthology.orchestra.run.smithi121.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-07-13T22:01:58.932 INFO:teuthology.orchestra.run.smithi121.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 509 kB/s | 47 kB 00:00 2023-07-13T22:01:58.956 INFO:teuthology.orchestra.run.smithi121.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-13T22:01:58.981 INFO:teuthology.orchestra.run.smithi121.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-13T22:01:59.006 INFO:teuthology.orchestra.run.smithi121.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-13T22:01:59.032 INFO:teuthology.orchestra.run.smithi121.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-07-13T22:01:59.057 INFO:teuthology.orchestra.run.smithi121.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-13T22:01:59.085 INFO:teuthology.orchestra.run.smithi121.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 853 kB/s | 43 kB 00:00 2023-07-13T22:01:59.108 INFO:teuthology.orchestra.run.smithi121.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-13T22:01:59.133 INFO:teuthology.orchestra.run.smithi121.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-13T22:01:59.166 INFO:teuthology.orchestra.run.smithi121.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-07-13T22:01:59.192 INFO:teuthology.orchestra.run.smithi121.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-07-13T22:01:59.217 INFO:teuthology.orchestra.run.smithi121.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-13T22:01:59.251 INFO:teuthology.orchestra.run.smithi121.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-07-13T22:01:59.276 INFO:teuthology.orchestra.run.smithi121.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-07-13T22:01:59.309 INFO:teuthology.orchestra.run.smithi121.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-13T22:01:59.314 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2023-07-13T22:01:59.335 INFO:teuthology.orchestra.run.smithi121.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-13T22:01:59.369 INFO:teuthology.orchestra.run.smithi121.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-07-13T22:01:59.411 INFO:teuthology.orchestra.run.smithi121.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 23 MB/s | 2.3 MB 00:00 2023-07-13T22:01:59.436 INFO:teuthology.orchestra.run.smithi121.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-07-13T22:01:59.463 INFO:teuthology.orchestra.run.smithi121.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-13T22:01:59.487 INFO:teuthology.orchestra.run.smithi121.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-13T22:01:59.512 INFO:teuthology.orchestra.run.smithi121.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-13T22:01:59.531 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-13T22:01:59.537 INFO:teuthology.orchestra.run.smithi121.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-07-13T22:01:59.579 INFO:teuthology.orchestra.run.smithi121.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-07-13T22:01:59.604 INFO:teuthology.orchestra.run.smithi121.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-07-13T22:01:59.655 INFO:teuthology.orchestra.run.smithi121.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.8 MB/s | 138 kB 00:00 2023-07-13T22:01:59.735 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-13T22:01:59.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-13T22:02:00.059 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-13T22:02:00.265 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-13T22:02:00.782 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-13T22:02:00.813 INFO:teuthology.orchestra.run.smithi121.stdout:(128/131): librbd1-18.0.0-4757.g3fc0089c.el8.x8 3.1 MB/s | 3.6 MB 00:01 2023-07-13T22:02:00.820 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2023-07-13T22:02:01.005 INFO:teuthology.orchestra.run.smithi121.stdout:(129/131): librados2-18.0.0-4757.g3fc0089c.el8. 2.6 MB/s | 3.7 MB 00:01 2023-07-13T22:02:01.031 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-13T22:02:01.040 INFO:teuthology.orchestra.run.smithi121.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 3.5 MB/s | 798 kB 00:00 2023-07-13T22:02:01.050 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-13T22:02:01.225 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-13T22:02:01.387 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-13T22:02:01.498 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-13T22:02:01.518 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-13T22:02:01.692 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:(131/131): ceph-test-18.0.0-4757.g3fc0089c.el8. 3.8 MB/s | 48 MB 00:12 2023-07-13T22:02:01.701 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:02:01.701 INFO:teuthology.orchestra.run.smithi121.stdout:Total 11 MB/s | 205 MB 00:18 2023-07-13T22:02:01.719 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-13T22:02:01.886 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:02.073 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:02.113 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-13T22:02:02.197 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:02.335 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-13T22:02:02.395 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:02:02.535 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-07-13T22:02:02.574 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-13T22:02:02.654 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-07-13T22:02:02.655 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-07-13T22:02:02.727 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 5/134 2023-07-13T22:02:02.858 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-13T22:02:02.894 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:03.017 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:03.054 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-13T22:02:03.227 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:03.323 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:03.473 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:02:03.506 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-13T22:02:03.643 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-13T22:02:03.761 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-13T22:02:03.767 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-07-13T22:02:03.770 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-07-13T22:02:03.865 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-13T22:02:03.914 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:04.043 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:04.071 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-13T22:02:04.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:04.394 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:04.505 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-13T22:02:04.655 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-13T22:02:04.729 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-13T22:02:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-13T22:02:04.859 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:04.961 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:05.023 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-13T22:02:05.093 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:05.197 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-13T22:02:05.237 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:05.366 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-13T22:02:05.488 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-13T22:02:05.530 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-13T22:02:05.657 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-13T22:02:05.834 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-13T22:02:05.976 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-13T22:02:06.113 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-13T22:02:06.267 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-13T22:02:06.291 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-13T22:02:06.455 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-13T22:02:06.869 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-13T22:02:07.051 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-13T22:02:07.212 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-13T22:02:07.380 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-13T22:02:07.644 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-13T22:02:07.850 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-13T22:02:08.306 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-13T22:02:08.546 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-13T22:02:08.547 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-13T22:02:08.714 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-07-13T22:02:08.760 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-13T22:02:08.782 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-13T22:02:08.915 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-13T22:02:09.007 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-13T22:02:09.077 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-13T22:02:09.224 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-13T22:02:09.251 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-13T22:02:09.299 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-13T22:02:09.388 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-13T22:02:09.431 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-13T22:02:09.484 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-13T22:02:09.553 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-13T22:02:09.561 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-13T22:02:09.598 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-13T22:02:09.779 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-13T22:02:09.809 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-13T22:02:09.860 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-13T22:02:10.008 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-13T22:02:10.038 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-13T22:02:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-13T22:02:10.474 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:10.854 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:02:11.025 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:11.154 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:02:11.524 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:02:11.613 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:02:11.770 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-13T22:02:11.793 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-13T22:02:12.067 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-13T22:02:12.236 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-13T22:02:12.262 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-13T22:02:12.664 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-13T22:02:12.776 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-13T22:02:12.902 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-13T22:02:13.032 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:13.097 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-13T22:02:13.123 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 14/134 2023-07-13T22:02:13.275 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:13.348 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-13T22:02:13.361 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 15/134 2023-07-13T22:02:13.424 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-13T22:02:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-13T22:02:13.588 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-13T22:02:13.666 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-13T22:02:13.760 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-13T22:02:13.785 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-13T22:02:13.840 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-13T22:02:13.951 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-13T22:02:14.033 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-13T22:02:14.105 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-13T22:02:14.239 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-13T22:02:14.343 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-13T22:02:14.374 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-13T22:02:14.437 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-13T22:02:14.511 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-13T22:02:14.569 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-13T22:02:14.624 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-13T22:02:14.732 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-13T22:02:14.803 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-13T22:02:14.806 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-13T22:02:14.980 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-13T22:02:15.020 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-13T22:02:15.203 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-13T22:02:15.213 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-13T22:02:15.218 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-13T22:02:15.380 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-13T22:02:15.382 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-13T22:02:15.390 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-13T22:02:15.464 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-13T22:02:15.542 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-13T22:02:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-13T22:02:15.738 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-13T22:02:15.833 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-13T22:02:15.852 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-13T22:02:16.005 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-13T22:02:16.096 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-13T22:02:16.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-13T22:02:16.175 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-13T22:02:16.337 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-13T22:02:16.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-13T22:02:16.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-13T22:02:16.485 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-13T22:02:16.504 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-13T22:02:16.565 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-13T22:02:16.594 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-13T22:02:16.672 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-13T22:02:16.797 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-13T22:02:16.853 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-13T22:02:16.867 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-13T22:02:17.027 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-13T22:02:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-13T22:02:17.222 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-13T22:02:17.237 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-13T22:02:17.385 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-13T22:02:17.448 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-13T22:02:17.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-13T22:02:17.624 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-13T22:02:17.668 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-13T22:02:17.777 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-13T22:02:17.778 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-13T22:02:17.821 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 26/134 2023-07-13T22:02:17.951 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 27/134 2023-07-13T22:02:17.986 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-13T22:02:17.997 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-13T22:02:18.228 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-13T22:02:18.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-13T22:02:18.532 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-13T22:02:18.711 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-13T22:02:18.748 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-13T22:02:18.871 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 28/134 2023-07-13T22:02:18.940 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-13T22:02:19.070 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-13T22:02:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-13T22:02:19.193 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-13T22:02:19.312 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-13T22:02:19.385 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-13T22:02:19.441 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-13T22:02:19.567 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-13T22:02:19.606 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-13T22:02:19.708 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-13T22:02:19.832 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-13T22:02:19.871 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-13T22:02:19.897 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-13T22:02:20.120 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-13T22:02:20.224 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-13T22:02:20.361 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-13T22:02:20.463 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-13T22:02:20.590 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-13T22:02:20.683 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-13T22:02:20.764 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-13T22:02:20.899 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-13T22:02:20.958 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-13T22:02:21.103 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-13T22:02:21.215 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-13T22:02:21.284 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-13T22:02:21.437 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-13T22:02:21.626 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-13T22:02:21.663 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-13T22:02:21.691 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-13T22:02:21.774 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-13T22:02:21.971 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-13T22:02:21.980 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-13T22:02:22.016 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-13T22:02:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-13T22:02:22.208 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-13T22:02:22.246 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-13T22:02:22.340 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-13T22:02:22.399 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-13T22:02:22.498 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-13T22:02:22.516 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-13T22:02:22.578 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 36/134 2023-07-13T22:02:22.686 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-13T22:02:22.688 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-13T22:02:22.768 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 37/134 2023-07-13T22:02:22.870 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-13T22:02:22.906 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-13T22:02:22.937 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-13T22:02:23.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-13T22:02:23.133 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-13T22:02:23.168 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-13T22:02:23.291 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-13T22:02:23.328 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-13T22:02:23.428 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-13T22:02:23.474 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-13T22:02:23.567 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-13T22:02:23.591 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-13T22:02:23.719 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-13T22:02:23.750 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-13T22:02:23.779 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-13T22:02:23.910 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-13T22:02:23.970 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-13T22:02:23.988 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-13T22:02:24.163 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-13T22:02:24.221 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-13T22:02:24.276 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-13T22:02:24.463 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-13T22:02:24.591 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-13T22:02:24.648 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-13T22:02:24.789 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-13T22:02:24.852 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-13T22:02:25.030 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-13T22:02:25.100 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-13T22:02:25.231 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-13T22:02:25.252 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-13T22:02:25.326 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-13T22:02:25.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-13T22:02:25.510 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-13T22:02:25.571 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-13T22:02:25.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-13T22:02:25.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-13T22:02:25.804 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-13T22:02:26.000 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-13T22:02:26.021 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-13T22:02:26.055 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-13T22:02:26.216 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-13T22:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-13T22:02:26.374 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-13T22:02:26.376 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-13T22:02:26.567 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-13T22:02:26.571 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-13T22:02:26.582 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-13T22:02:26.892 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-13T22:02:27.124 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-13T22:02:27.354 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-13T22:02:27.375 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-13T22:02:27.504 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:27.578 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:27.602 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-13T22:02:27.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-13T22:02:27.769 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-13T22:02:27.905 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-13T22:02:27.994 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-13T22:02:28.112 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-13T22:02:28.170 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-13T22:02:28.307 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-13T22:02:28.419 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-13T22:02:28.559 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-13T22:02:28.615 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-13T22:02:28.754 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-13T22:02:28.815 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-13T22:02:28.943 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-13T22:02:29.041 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-13T22:02:29.213 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-13T22:02:29.335 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-13T22:02:29.395 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-13T22:02:29.551 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-13T22:02:29.573 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-13T22:02:29.854 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-13T22:02:30.068 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-13T22:02:30.224 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-13T22:02:30.413 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-13T22:02:30.475 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-13T22:02:30.619 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-13T22:02:30.684 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:30.785 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:30.814 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-13T22:02:30.960 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-13T22:02:30.989 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-13T22:02:31.181 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-13T22:02:31.341 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-13T22:02:31.368 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:31.554 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-13T22:02:31.764 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-13T22:02:31.983 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-13T22:02:32.195 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:32.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-13T22:02:32.434 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-13T22:02:32.608 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-13T22:02:32.799 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:32.891 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-13T22:02:33.049 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:33.163 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:33.286 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:02:33.973 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-13T22:02:34.264 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-13T22:02:34.503 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-13T22:02:34.810 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-13T22:02:35.111 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-13T22:02:35.333 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-13T22:02:36.097 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:36.108 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-13T22:02:36.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:36.334 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:36.350 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 92/134 2023-07-13T22:02:36.535 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 93/134 2023-07-13T22:02:37.080 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:37.470 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:37.613 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:37.792 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:02:40.225 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:40.477 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:41.065 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 94/134 2023-07-13T22:02:41.374 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:41.446 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 95/134 2023-07-13T22:02:41.576 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:02:51.983 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:02:51.984 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2023-07-13T22:02:51.984 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2023-07-13T22:02:51.984 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2023-07-13T22:02:51.984 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /run 2023-07-13T22:02:51.984 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2023-07-13T22:02:51.985 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2023-07-13T22:02:51.985 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2023-07-13T22:02:51.985 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2023-07-13T22:02:51.985 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2023-07-13T22:02:51.985 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:02:52.332 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:02:53.894 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:02:54.052 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:02:54.271 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:02:54.490 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:02:54.955 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:02:55.098 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:02:55.115 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:02:56.813 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:02:57.012 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:02:57.158 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-13T22:02:57.377 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-13T22:02:57.572 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:02:57.796 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:02:58.092 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:02:59.091 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:02:59.111 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:02:59.112 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2023-07-13T22:02:59.112 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2023-07-13T22:02:59.112 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-07-13T22:02:59.112 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /run 2023-07-13T22:02:59.112 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2023-07-13T22:02:59.113 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2023-07-13T22:02:59.113 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2023-07-13T22:02:59.113 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2023-07-13T22:02:59.113 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /dev 2023-07-13T22:02:59.113 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:02:59.247 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:02:59.339 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:02:59.522 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:02:59.714 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-13T22:02:59.917 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:02:59.951 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:03:00.146 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:00.157 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 96/134 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2023-07-13T22:03:00.160 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2023-07-13T22:03:00.161 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2023-07-13T22:03:00.161 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2023-07-13T22:03:00.161 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2023-07-13T22:03:00.161 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-07-13T22:03:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:03:00.547 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:03:01.366 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:03:01.502 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-13T22:03:01.532 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:03:01.701 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:01.816 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:03:01.879 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:01.924 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:02.021 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:03:02.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:02.107 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 97/134 2023-07-13T22:03:02.270 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:03:02.453 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:02.487 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 98/134 2023-07-13T22:03:02.583 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:03:02.613 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:02.663 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:03:02.796 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:03:02.840 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:03:03.116 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 99/134 2023-07-13T22:03:03.246 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:03:03.263 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 100/134 2023-07-13T22:03:04.915 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:03:05.146 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:03:05.268 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-13T22:03:05.453 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-13T22:03:05.628 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:03:05.648 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:03:05.826 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 101/134 2023-07-13T22:03:05.842 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:03:05.936 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 102/134 2023-07-13T22:03:06.159 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 103/134 2023-07-13T22:03:06.160 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:03:06.336 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:03:06.610 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 104/134 2023-07-13T22:03:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:03:07.118 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:03:07.264 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:03:07.345 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:07.352 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:03:07.712 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-13T22:03:07.910 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:03:07.946 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:03:08.026 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 105/134 2023-07-13T22:03:08.111 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:08.122 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:08.204 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:03:08.214 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-13T22:03:08.337 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 106/134 2023-07-13T22:03:08.509 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-13T22:03:08.689 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-13T22:03:08.758 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 107/134 2023-07-13T22:03:08.917 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-13T22:03:09.019 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:03:09.061 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-13T22:03:09.086 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 108/134 2023-07-13T22:03:09.255 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-13T22:03:09.286 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 109/134 2023-07-13T22:03:09.442 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-13T22:03:09.515 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-13T22:03:09.641 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:09.674 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:09.690 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-13T22:03:09.837 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-13T22:03:09.871 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:09.911 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:10.094 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:10.438 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:10.637 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:10.858 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 110/134 2023-07-13T22:03:11.060 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:11.135 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 111/134 2023-07-13T22:03:11.327 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:11.410 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 112/134 2023-07-13T22:03:11.586 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:11.957 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:12.020 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 113/134 2023-07-13T22:03:12.210 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:12.229 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:12.336 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:12.377 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-13T22:03:12.676 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-13T22:03:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-13T22:03:13.300 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-13T22:03:13.476 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-13T22:03:13.556 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-13T22:03:13.768 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-13T22:03:13.781 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-13T22:03:13.960 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-13T22:03:13.968 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-13T22:03:14.104 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-13T22:03:14.104 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:14.313 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:14.313 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:14.548 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:14.733 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:15.195 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:15.230 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:15.230 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-13T22:03:15.230 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:03:15.231 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-13T22:03:15.232 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-13T22:03:15.233 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-13T22:03:15.234 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-13T22:03:15.242 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-13T22:03:15.242 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-13T22:03:15.242 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-13T22:03:15.242 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-13T22:03:15.243 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-13T22:03:15.243 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-13T22:03:15.243 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-13T22:03:15.243 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-13T22:03:15.243 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-13T22:03:15.244 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-13T22:03:15.245 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-13T22:03:15.246 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-13T22:03:15.247 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-13T22:03:15.248 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-13T22:03:15.249 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-13T22:03:15.250 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-13T22:03:15.250 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-13T22:03:15.250 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-13T22:03:15.250 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-13T22:03:15.250 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-13T22:03:15.251 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-13T22:03:15.252 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-13T22:03:15.253 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-13T22:03:15.254 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-13T22:03:15.255 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-13T22:03:15.256 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:15.257 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-13T22:03:15.494 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-13T22:03:15.651 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-13T22:03:15.845 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-13T22:03:16.734 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-13T22:03:17.040 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-13T22:03:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 119/134 2023-07-13T22:03:17.411 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-13T22:03:17.588 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-13T22:03:17.691 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:17.692 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:02:03 ago on Thu 13 Jul 2023 10:01:14 PM UTC. 2023-07-13T22:03:17.711 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 114/134 2023-07-13T22:03:17.735 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-13T22:03:19.060 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 115/134 2023-07-13T22:03:19.452 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-13T22:03:19.641 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 117/134 2023-07-13T22:03:19.901 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:19.929 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-13T22:03:20.092 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-13T22:03:20.142 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:20.160 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:20.337 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-13T22:03:20.357 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-13T22:03:20.614 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-13T22:03:20.649 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-13T22:03:20.848 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 121/134 2023-07-13T22:03:20.961 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-13T22:03:21.118 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-13T22:03:21.276 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-13T22:03:21.400 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-13T22:03:21.408 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-13T22:03:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-13T22:03:21.757 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-13T22:03:21.838 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-13T22:03:22.066 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-13T22:03:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-13T22:03:22.079 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:22.277 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:22.278 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:22.495 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:22.689 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:23.248 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:23.248 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-13T22:03:23.249 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-13T22:03:23.249 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-13T22:03:23.249 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-13T22:03:23.250 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-13T22:03:23.250 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:03:23.250 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:03:23.250 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:03:23.251 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-13T22:03:23.252 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-13T22:03:23.252 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-13T22:03:23.252 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-13T22:03:23.253 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-13T22:03:23.253 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-13T22:03:23.253 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-13T22:03:23.254 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-13T22:03:23.254 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-13T22:03:23.254 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-13T22:03:23.254 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-13T22:03:23.255 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-13T22:03:23.256 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-13T22:03:23.257 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-13T22:03:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-13T22:03:23.259 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-13T22:03:23.260 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-13T22:03:23.260 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-13T22:03:23.260 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-13T22:03:23.260 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-13T22:03:23.261 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-13T22:03:23.261 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-13T22:03:23.262 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-13T22:03:23.263 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-13T22:03:23.264 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-13T22:03:23.264 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-13T22:03:23.264 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-13T22:03:23.264 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-13T22:03:23.265 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-13T22:03:23.265 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-13T22:03:23.265 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-13T22:03:23.265 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-13T22:03:23.266 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-13T22:03:23.266 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-13T22:03:23.266 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-13T22:03:23.267 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-13T22:03:23.267 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-13T22:03:23.267 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-13T22:03:23.267 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-13T22:03:23.268 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-13T22:03:23.268 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-13T22:03:23.268 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-13T22:03:23.269 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-13T22:03:23.269 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-13T22:03:23.269 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-13T22:03:23.269 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-13T22:03:23.270 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-13T22:03:23.270 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-13T22:03:23.270 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-13T22:03:23.270 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-13T22:03:23.271 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-13T22:03:23.271 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-13T22:03:23.271 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-13T22:03:23.272 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-13T22:03:23.272 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-13T22:03:23.272 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-13T22:03:23.272 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-13T22:03:23.273 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-13T22:03:23.273 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-13T22:03:23.273 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-13T22:03:23.274 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-13T22:03:23.274 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-13T22:03:23.274 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-13T22:03:23.274 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-13T22:03:23.274 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-13T22:03:23.275 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-13T22:03:23.275 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-13T22:03:23.275 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-13T22:03:23.275 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-13T22:03:23.276 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-13T22:03:23.276 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-13T22:03:23.276 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-13T22:03:23.276 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-13T22:03:23.277 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-13T22:03:23.277 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-13T22:03:23.277 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-13T22:03:23.277 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-13T22:03:23.277 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-13T22:03:23.278 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-13T22:03:23.278 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:23.278 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-13T22:03:24.881 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:25.408 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:25.447 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-13T22:03:25.601 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 124/134 2023-07-13T22:03:25.717 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:25.718 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:53 ago on Thu 13 Jul 2023 10:01:32 PM UTC. 2023-07-13T22:03:25.835 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 125/134 2023-07-13T22:03:26.163 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 126/134 2023-07-13T22:03:26.550 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-13T22:03:26.766 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-13T22:03:26.882 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-13T22:03:27.125 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-13T22:03:27.154 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-13T22:03:27.358 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-13T22:03:27.381 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-13T22:03:27.545 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-13T22:03:27.545 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:27.781 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:27.782 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:28.073 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-13T22:03:28.266 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:28.528 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:28.529 INFO:teuthology.orchestra.run.smithi026.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-13T22:03:28.529 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:03:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 1/134 2023-07-13T22:03:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 3/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 4/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4757.g3fc00 5/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 6/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 7/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 8/134 2023-07-13T22:03:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 9/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 10/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 11/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 12/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 13/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_ 14/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 15/134 2023-07-13T22:03:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 16/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_6 17/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86 18/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 19/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el 20/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8. 21/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_6 22/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 23/134 2023-07-13T22:03:29.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 24/134 2023-07-13T22:03:29.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 25/134 2023-07-13T22:03:29.165 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 26/134 2023-07-13T22:03:29.165 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 27/134 2023-07-13T22:03:29.165 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 28/134 2023-07-13T22:03:29.166 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c. 29/134 2023-07-13T22:03:29.166 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noa 30/134 2023-07-13T22:03:29.166 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.n 31/134 2023-07-13T22:03:29.166 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc 32/134 2023-07-13T22:03:29.167 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el 33/134 2023-07-13T22:03:29.167 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 34/134 2023-07-13T22:03:29.167 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.e 35/134 2023-07-13T22:03:29.167 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 36/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 37/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 38/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-13T22:03:29.168 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-13T22:03:29.169 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-13T22:03:29.170 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-13T22:03:29.171 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-13T22:03:29.172 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-13T22:03:29.173 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-13T22:03:29.174 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-13T22:03:29.175 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-13T22:03:29.176 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-13T22:03:29.177 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-13T22:03:29.178 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-13T22:03:29.179 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 129/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 131/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-13T22:03:29.180 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-13T22:03:29.731 INFO:teuthology.orchestra.run.smithi117.stdout:Installed products updated. 2023-07-13T22:03:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2023-07-13T22:03:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.1-7.el8.noarch 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.0.4-10.el8.x86_64 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.1-9.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-3.el8.x86_64 2023-07-13T22:03:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-13T22:03:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-13T22:03:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: libpq-13.5-1.el8.x86_64 2023-07-13T22:03:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-13T22:03:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-13T22:03:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-13T22:03:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-13T22:03:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-13T22:03:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-13T22:03:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-13T22:03:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-2.6-1.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-13T22:03:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-13T22:03:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-13T22:03:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-13T22:03:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-13T22:03:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-13T22:03:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-13T22:03:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-13T22:03:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-13T22:03:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-07-13T22:03:29.962 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2023-07-13T22:03:30.157 DEBUG:teuthology.parallel:result is None 2023-07-13T22:03:32.000 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-13T22:03:32.000 INFO:teuthology.orchestra.run.smithi026.stdout:Last metadata expiration check: 0:02:16 ago on Thu 13 Jul 2023 10:01:16 PM UTC. 2023-07-13T22:03:32.642 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-07-13T22:03:32.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:03:32.857 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.1-7.el8.noarch 2023-07-13T22:03:32.858 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.0.4-10.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.859 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.860 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.861 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.1-9.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: jq-1.6-3.el8.x86_64 2023-07-13T22:03:32.862 INFO:teuthology.orchestra.run.smithi121.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-13T22:03:32.863 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: libpq-13.5-1.el8.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.864 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-13T22:03:32.865 INFO:teuthology.orchestra.run.smithi121.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-13T22:03:32.866 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-13T22:03:32.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-13T22:03:32.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-13T22:03:32.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-13T22:03:32.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-13T22:03:32.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-13T22:03:32.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-13T22:03:32.870 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-13T22:03:32.870 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-13T22:03:32.870 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-13T22:03:32.870 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-13T22:03:32.871 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-13T22:03:32.871 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-13T22:03:32.871 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-13T22:03:32.871 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-13T22:03:32.872 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-13T22:03:32.872 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-13T22:03:32.872 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-13T22:03:32.873 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-13T22:03:32.873 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-13T22:03:32.873 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-13T22:03:32.874 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.875 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-13T22:03:32.876 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-13T22:03:32.877 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-13T22:03:32.878 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-13T22:03:32.878 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-13T22:03:32.878 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-13T22:03:32.878 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.878 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-13T22:03:32.879 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-13T22:03:32.880 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-13T22:03:32.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:03:32.881 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-07-13T22:03:33.038 DEBUG:teuthology.parallel:result is None 2023-07-13T22:03:43.017 INFO:teuthology.orchestra.run.smithi026.stdout:Installed products updated. 2023-07-13T22:03:43.269 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:03:43.269 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2023-07-13T22:03:43.269 INFO:teuthology.orchestra.run.smithi026.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.1-7.el8.noarch 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.0.4-10.el8.x86_64 2023-07-13T22:03:43.270 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.271 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.272 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:18.0.0-4757.g3fc0089c.el8.noarch 2023-07-13T22:03:43.273 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.1-9.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-3.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-13T22:03:43.274 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libpq-13.5-1.el8.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-13T22:03:43.275 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-13T22:03:43.276 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-13T22:03:43.277 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-13T22:03:43.278 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.279 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-13T22:03:43.280 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-13T22:03:43.281 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-2.6-1.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-13T22:03:43.282 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-13T22:03:43.283 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-13T22:03:43.284 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-13T22:03:43.285 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.0.0-4757.g3fc0089c.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-13T22:03:43.286 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-13T22:03:43.287 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-13T22:03:43.287 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:03:43.287 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2023-07-13T22:03:44.009 DEBUG:teuthology.parallel:result is None 2023-07-13T22:03:44.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:44.356 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-13T22:03:44.411 INFO:teuthology.orchestra.run.smithi026.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:44.412 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:44.413 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-13T22:03:44.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:44.759 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-13T22:03:44.787 INFO:teuthology.orchestra.run.smithi117.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:44.787 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:44.788 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-13T22:03:44.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:45.126 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-13T22:03:45.153 INFO:teuthology.orchestra.run.smithi121.stdout:18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:45.154 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4757.g3fc0089c.el8 2023-07-13T22:03:45.154 INFO:teuthology.task.install:The correct ceph version 18.0.0-4757.g3fc0089c is installed. 2023-07-13T22:03:45.154 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-13T22:03:45.155 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:03:45.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-13T22:03:45.252 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:03:45.253 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-13T22:03:45.300 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:03:45.301 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-13T22:03:45.349 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-13T22:03:45.349 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:03:45.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2023-07-13T22:03:45.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-13T22:03:45.530 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:03:45.531 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2023-07-13T22:03:45.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-13T22:03:45.632 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:03:45.632 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-07-13T22:03:45.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-13T22:03:45.734 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-13T22:03:45.735 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:03:45.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-13T22:03:45.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-13T22:03:45.914 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:03:45.915 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-13T22:03:45.947 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-13T22:03:46.019 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:03:46.020 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-13T22:03:46.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-13T22:03:46.121 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-13T22:03:46.122 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:03:46.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2023-07-13T22:03:46.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-13T22:03:46.298 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:03:46.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2023-07-13T22:03:46.330 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-13T22:03:46.401 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:03:46.402 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2023-07-13T22:03:46.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-13T22:03:46.505 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-13T22:03:46.597 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-13T22:03:46.598 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:46.610 INFO:tasks.cephadm:Cluster fsid is 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:03:46.611 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-13T22:03:46.611 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.26', 'mon.b': '172.21.15.117', 'mon.c': '172.21.15.121'} 2023-07-13T22:03:46.612 INFO:tasks.cephadm:First mon is mon.a on smithi026 2023-07-13T22:03:46.612 INFO:tasks.cephadm:First mgr is x 2023-07-13T22:03:46.612 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-13T22:03:46.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo hostname $(hostname -s) 2023-07-13T22:03:46.685 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2023-07-13T22:03:46.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname $(hostname -s) 2023-07-13T22:03:46.791 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-13T22:03:46.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:47.100 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3fc0089cc1082313a98853d48b892acdc1385b8b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71399/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4757-g3fc0089c', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4757.g3fc0089c'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-13 19:47:46.444052', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-13T22:03:47.367 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13, sha1 3fc0089cc1082313a98853d48b892acdc1385b8b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:03:47.368 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm 2023-07-13T22:03:47.368 DEBUG:teuthology.orchestra.run.smithi026:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:48.576 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 13 22:03 /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:48.576 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:49.723 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 13 22:03 /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:49.723 DEBUG:teuthology.orchestra.run.smithi121:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13/3fc0089cc1082313a98853d48b892acdc1385b8b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:51.082 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 13 22:03 /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:51.082 DEBUG:teuthology.orchestra.run.smithi026:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:51.134 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:51.156 DEBUG:teuthology.orchestra.run.smithi121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-13T22:03:51.203 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b on all hosts... 2023-07-13T22:03:51.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-13T22:03:51.209 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-13T22:03:51.213 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b pull 2023-07-13T22:03:52.522 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-13T22:03:53.201 INFO:teuthology.orchestra.run.smithi121.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-13T22:03:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-13T22:04:21.706 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2023-07-13T22:04:21.706 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:04:21.706 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-13T22:04:21.707 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2023-07-13T22:04:21.707 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-13T22:04:21.707 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2023-07-13T22:04:21.707 INFO:teuthology.orchestra.run.smithi117.stdout:} 2023-07-13T22:04:36.253 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-13T22:04:36.254 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:04:36.255 INFO:teuthology.orchestra.run.smithi121.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-13T22:04:36.255 INFO:teuthology.orchestra.run.smithi121.stdout: "repo_digests": [ 2023-07-13T22:04:36.256 INFO:teuthology.orchestra.run.smithi121.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-13T22:04:36.256 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2023-07-13T22:04:36.256 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-13T22:04:43.772 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:04:43.772 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev)", 2023-07-13T22:04:43.773 INFO:teuthology.orchestra.run.smithi026.stdout: "image_id": "afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295", 2023-07-13T22:04:43.773 INFO:teuthology.orchestra.run.smithi026.stdout: "repo_digests": [ 2023-07-13T22:04:43.773 INFO:teuthology.orchestra.run.smithi026.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c4bf3497f7cb9cae2d0db734c6fa7ef173cd8dc715907bde16e4a68082d82e1d" 2023-07-13T22:04:43.773 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:04:43.774 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:04:43.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph 2023-07-13T22:04:43.878 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2023-07-13T22:04:43.919 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph 2023-07-13T22:04:43.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /etc/ceph 2023-07-13T22:04:44.030 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2023-07-13T22:04:44.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /etc/ceph 2023-07-13T22:04:44.096 INFO:tasks.cephadm:Writing seed config... 2023-07-13T22:04:44.098 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-13T22:04:44.098 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-13T22:04:44.098 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-13T22:04:44.099 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-13T22:04:44.099 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-13T22:04:44.099 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-13T22:04:44.099 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-13T22:04:44.100 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-13T22:04:44.100 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-13T22:04:44.101 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-13T22:04:44.101 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-13T22:04:44.105 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-13T22:04:44.105 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-13T22:04:44.106 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-13T22:04:44.106 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-13T22:04:44.107 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-13T22:04:44.107 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-13T22:04:44.107 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-13T22:04:44.108 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-13T22:04:44.109 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-13T22:04:44.109 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-13T22:04:44.109 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-13T22:04:44.109 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-13T22:04:44.110 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-13T22:04:44.110 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-13T22:04:44.110 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-13T22:04:44.110 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-13T22:04:44.110 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2023-07-13T22:04:44.111 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2023-07-13T22:04:44.111 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2023-07-13T22:04:44.111 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-13T22:04:44.111 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-13T22:04:44.111 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-07-13T22:04:44.112 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-07-13T22:04:44.112 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-13T22:04:44.112 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-13T22:04:44.112 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-07-13T22:04:44.112 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-13T22:04:44.113 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-13T22:04:44.113 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-13T22:04:44.113 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-13T22:04:44.113 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-13T22:04:44.114 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-13T22:04:44.114 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-13T22:04:44.115 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-13T22:04:44.120 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:04:44.120 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-13T22:04:44.163 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 23b26990-21c9-11ee-9b32-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-13T22:04:44.163 DEBUG:teuthology.orchestra.run.smithi026:mon.a> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service 2023-07-13T22:04:44.208 DEBUG:teuthology.orchestra.run.smithi026:mgr.x> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x.service 2023-07-13T22:04:44.255 INFO:tasks.cephadm:Bootstrapping... 2023-07-13T22:04:44.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b -v bootstrap --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.26 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:04:44.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:04:44.324 INFO:journalctl@ceph.mgr.x.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:04:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2023-07-13T22:04:44.484 INFO:teuthology.orchestra.run.smithi026.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b', '-v', 'bootstrap', '--fsid', '23b26990-21c9-11ee-9b32-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.26', '--skip-admin-label'] 2023-07-13T22:04:44.658 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-07-13T22:04:44.658 INFO:teuthology.orchestra.run.smithi026.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-13T22:04:44.659 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying podman|docker is present... 2023-07-13T22:04:44.830 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-07-13T22:04:44.830 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying lvm2 is present... 2023-07-13T22:04:44.830 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying time synchronization is in place... 2023-07-13T22:04:44.844 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-13T22:04:44.844 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-13T22:04:44.857 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-13T22:04:44.858 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:04:44.871 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2023-07-13T22:04:44.884 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2023-07-13T22:04:44.885 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2023-07-13T22:04:44.885 INFO:teuthology.orchestra.run.smithi026.stdout:Repeating the final host check... 2023-07-13T22:04:45.056 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout 4.1.1 2023-07-13T22:04:45.056 INFO:teuthology.orchestra.run.smithi026.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-13T22:04:45.056 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl is present 2023-07-13T22:04:45.056 INFO:teuthology.orchestra.run.smithi026.stdout:lvcreate is present 2023-07-13T22:04:45.069 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-13T22:04:45.070 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-13T22:04:45.082 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-13T22:04:45.083 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:04:45.096 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout enabled 2023-07-13T22:04:45.109 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout active 2023-07-13T22:04:45.109 INFO:teuthology.orchestra.run.smithi026.stdout:Unit chronyd.service is enabled and running 2023-07-13T22:04:45.109 INFO:teuthology.orchestra.run.smithi026.stdout:Host looks OK 2023-07-13T22:04:45.109 INFO:teuthology.orchestra.run.smithi026.stdout:Cluster fsid: 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:45.110 INFO:teuthology.orchestra.run.smithi026.stdout:Acquiring lock 140542162119144 on /run/cephadm/23b26990-21c9-11ee-9b32-001a4aab830c.lock 2023-07-13T22:04:45.110 INFO:teuthology.orchestra.run.smithi026.stdout:Lock 140542162119144 acquired on /run/cephadm/23b26990-21c9-11ee-9b32-001a4aab830c.lock 2023-07-13T22:04:45.110 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 3300 ... 2023-07-13T22:04:45.111 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying IP 172.21.15.26 port 6789 ... 2023-07-13T22:04:45.111 INFO:teuthology.orchestra.run.smithi026.stdout:Base mon IP(s) is [172.21.15.26:3300, 172.21.15.26:6789], mon addrv is [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2023-07-13T22:04:45.118 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-07-13T22:04:45.118 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.26 metric 100 2023-07-13T22:04:45.125 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-13T22:04:45.125 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-07-13T22:04:45.125 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1768sec hoplimit 64 pref medium 2023-07-13T22:04:45.131 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-13T22:04:45.132 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-13T22:04:45.132 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-13T22:04:45.132 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-07-13T22:04:45.132 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-07-13T22:04:45.132 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-13T22:04:45.133 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2023-07-13T22:04:45.133 INFO:teuthology.orchestra.run.smithi026.stdout:Mon IP `172.21.15.26` is in CIDR network `172.21.0.0/20` 2023-07-13T22:04:45.133 INFO:teuthology.orchestra.run.smithi026.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-13T22:04:45.134 INFO:teuthology.orchestra.run.smithi026.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-13T22:04:45.135 INFO:teuthology.orchestra.run.smithi026.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-13T22:04:45.609 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stdout afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-13T22:04:45.609 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b... 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Getting image source signatures 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:d8eac6e0ff3c58bd5607bffacb1c2214d20cdd3ff8cfaf23cfeb0cc60b2d0d6b 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Copying config sha256:afd12e232ec04c7e77932076425916985d97c3486288fff2ba931630e84a9295 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-13T22:04:45.610 INFO:teuthology.orchestra.run.smithi026.stdout:/bin/podman: stderr Storing signatures 2023-07-13T22:04:46.585 INFO:teuthology.orchestra.run.smithi026.stdout:ceph: stdout ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-13T22:04:46.585 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph version: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev) 2023-07-13T22:04:46.585 INFO:teuthology.orchestra.run.smithi026.stdout:Extracting ceph user uid/gid from container image... 2023-07-13T22:04:47.461 INFO:teuthology.orchestra.run.smithi026.stdout:stat: stdout 167 167 2023-07-13T22:04:47.462 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial keys... 2023-07-13T22:04:48.239 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQD/dLBkjbv0NBAARtM3HJUji1Zg4P7odBT5RQ== 2023-07-13T22:04:49.013 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQAAdbBkpHUPJRAAUWL1jnZU48iMSzITawJ74g== 2023-07-13T22:04:49.895 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph-authtool: stdout AQABdbBkibyOGhAAVR06vjufhw1DFNASKZjD4g== 2023-07-13T22:04:49.896 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial monmap... 2023-07-13T22:04:50.771 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool for a [v2:172.21.15.26:3300,v1:172.21.15.26:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = pacific 2023-07-13T22:04:50.772 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: set fsid to 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:50.773 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-13T22:04:50.773 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:04:50.773 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mon... 2023-07-13T22:04:51.646 INFO:teuthology.orchestra.run.smithi026.stdout:create mon.a on 2023-07-13T22:04:51.897 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-13T22:04:52.105 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-13T22:04:52.358 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target → /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target. 2023-07-13T22:04:52.358 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target → /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target. 2023-07-13T22:04:52.746 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a 2023-07-13T22:04:52.746 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service: Unit ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service not loaded. 2023-07-13T22:04:52.948 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target.wants/ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c@.service. 2023-07-13T22:04:53.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:52 smithi026 systemd[1]: Starting Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:04:53.734 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:53 smithi026 podman[95104]: 2023-07-13T22:04:53.734 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:53 smithi026 bash[95104]: d47ab3ba05b97b4b5613f03e2463d0790f7bafc560d3fe42fc2c10d55da954ae 2023-07-13T22:04:53.767 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-13T22:04:53.767 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-07-13T22:04:53.784 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-13T22:04:53.784 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:04:53.784 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-07-13T22:04:53.785 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2023-07-13T22:04:53.785 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon to start... 2023-07-13T22:04:53.785 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mon... 2023-07-13T22:04:54.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:53 smithi026 ceph-mon[95144]: mkfs 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:54.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:53 smithi026 ceph-mon[95144]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-13T22:04:54.074 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:53 smithi026 systemd[1]: Started Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:04:54.871 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout cluster: 2023-07-13T22:04:54.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout id: 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:54.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-13T22:04:54.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:54.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout services: 2023-07-13T22:04:54.872 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.692443s) 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout data: 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-13T22:04:54.873 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout pgs: 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.337+0000 7efeab1e2700 1 Processor -- start 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.337+0000 7efeab1e2700 1 -- start start 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.338+0000 7efeab1e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.338+0000 7efeab1e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40796f0 con 0x7efea407acb0 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.338+0000 7efea8f7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.338+0000 7efea8f7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55932/0 (socket says 172.21.15.26:55932) 2023-07-13T22:04:54.874 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.338+0000 7efea8f7e700 1 -- 172.21.15.26:0/2538496300 learned_addr learned my addr 172.21.15.26:0/2538496300 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.339+0000 7efea8f7e700 1 -- 172.21.15.26:0/2538496300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4079830 con 0x7efea407acb0 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.339+0000 7efea8f7e700 1 --2- 172.21.15.26:0/2538496300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe980098a0 tx=0x7efe98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a345410da50f4b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.339+0000 7efea37fe700 1 -- 172.21.15.26:0/2538496300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98004160 con 0x7efea407acb0 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.339+0000 7efea37fe700 1 -- 172.21.15.26:0/2538496300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe980042c0 con 0x7efea407acb0 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.339+0000 7efea37fe700 1 -- 172.21.15.26:0/2538496300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe980045a0 con 0x7efea407acb0 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 -- 172.21.15.26:0/2538496300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 msgr2=0x7efea40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:54.875 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 --2- 172.21.15.26:0/2538496300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efe980098a0 tx=0x7efe98009bb0 comp rx=0 tx=0).stop 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 -- 172.21.15.26:0/2538496300 shutdown_connections 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 --2- 172.21.15.26:0/2538496300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea40791b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 -- 172.21.15.26:0/2538496300 >> 172.21.15.26:0/2538496300 conn(0x7efea41009f0 msgr2=0x7efea4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 -- 172.21.15.26:0/2538496300 shutdown_connections 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.340+0000 7efeab1e2700 1 -- 172.21.15.26:0/2538496300 wait complete. 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.341+0000 7efeab1e2700 1 Processor -- start 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.341+0000 7efeab1e2700 1 -- start start 2023-07-13T22:04:54.876 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.342+0000 7efeab1e2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.342+0000 7efeab1e2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea419b050 con 0x7efea407acb0 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.342+0000 7efea8f7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.342+0000 7efea8f7e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55936/0 (socket says 172.21.15.26:55936) 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.342+0000 7efea8f7e700 1 -- 172.21.15.26:0/3452202693 learned_addr learned my addr 172.21.15.26:0/3452202693 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efea8f7e700 1 -- 172.21.15.26:0/3452202693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe98009550 con 0x7efea407acb0 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efea8f7e700 1 --2- 172.21.15.26:0/3452202693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efe98004000 tx=0x7efe98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:54.877 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98011630 con 0x7efea407acb0 2023-07-13T22:04:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98011790 con 0x7efea407acb0 2023-07-13T22:04:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe98011a30 con 0x7efea407acb0 2023-07-13T22:04:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea419b250 con 0x7efea407acb0 2023-07-13T22:04:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.343+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea419b670 con 0x7efea407acb0 2023-07-13T22:04:54.878 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.344+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efe98022070 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.344+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efe9801c450 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.344+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea41948d0 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.347+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7efe9801b450 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.424+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7efea402c850 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.424+0000 7efea1ffb700 1 -- 172.21.15.26:0/3452202693 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7efe9801b5f0 con 0x7efea407acb0 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 msgr2=0x7efea419ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 --2- 172.21.15.26:0/3452202693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efe98004000 tx=0x7efe98004750 comp rx=0 tx=0).stop 2023-07-13T22:04:54.879 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 shutdown_connections 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 --2- 172.21.15.26:0/3452202693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea407acb0 0x7efea419ab10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 >> 172.21.15.26:0/3452202693 conn(0x7efea41009f0 msgr2=0x7efea4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 shutdown_connections 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:54.425+0000 7efeab1e2700 1 -- 172.21.15.26:0/3452202693 wait complete. 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:mon is available 2023-07-13T22:04:54.880 INFO:teuthology.orchestra.run.smithi026.stdout:Assimilating anything we can from ceph.conf... 2023-07-13T22:04:55.122 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-13T22:04:55.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: monmap e1: 1 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0]} removed_ranks: {} 2023-07-13T22:04:55.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: fsmap 2023-07-13T22:04:55.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: osdmap e1: 0 total, 0 up, 0 in 2023-07-13T22:04:55.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: mgrmap e1: no daemons active 2023-07-13T22:04:55.123 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:54 smithi026 ceph-mon[95144]: from='client.? 172.21.15.26:0/3452202693' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-13T22:04:55.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:55 smithi026 ceph-mon[95144]: from='client.? 172.21.15.26:0/2027574897' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-13T22:04:55.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:55 smithi026 ceph-mon[95144]: from='client.? 172.21.15.26:0/2027574897' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-13T22:04:55.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:55.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2023-07-13T22:04:55.956 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.26:3300,v1:172.21.15.26:6789] 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:55.957 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-13T22:04:55.958 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe714eab700 1 Processor -- start 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe714eab700 1 -- start start 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe714eab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe714eab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe710105cd0 con 0x7fe710105380 2023-07-13T22:04:55.959 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe70e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.423+0000 7fe70e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55952/0 (socket says 172.21.15.26:55952) 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70e59c700 1 -- 172.21.15.26:0/153792676 learned_addr learned my addr 172.21.15.26:0/153792676 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70e59c700 1 -- 172.21.15.26:0/153792676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710105e10 con 0x7fe710105380 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70e59c700 1 --2- 172.21.15.26:0/153792676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7000098a0 tx=0x7fe700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01356a62af0af5f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70d59a700 1 -- 172.21.15.26:0/153792676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe700004160 con 0x7fe710105380 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70d59a700 1 -- 172.21.15.26:0/153792676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7000042c0 con 0x7fe710105380 2023-07-13T22:04:55.960 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.424+0000 7fe70d59a700 1 -- 172.21.15.26:0/153792676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7000045a0 con 0x7fe710105380 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 -- 172.21.15.26:0/153792676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 msgr2=0x7fe710105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 --2- 172.21.15.26:0/153792676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7000098a0 tx=0x7fe700009bb0 comp rx=0 tx=0).stop 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 -- 172.21.15.26:0/153792676 shutdown_connections 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 --2- 172.21.15.26:0/153792676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe710105790 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 -- 172.21.15.26:0/153792676 >> 172.21.15.26:0/153792676 conn(0x7fe7101009f0 msgr2=0x7fe710102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 -- 172.21.15.26:0/153792676 shutdown_connections 2023-07-13T22:04:55.961 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.425+0000 7fe714eab700 1 -- 172.21.15.26:0/153792676 wait complete. 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.426+0000 7fe714eab700 1 Processor -- start 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.426+0000 7fe714eab700 1 -- start start 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe714eab700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe714eab700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71007b060 con 0x7fe710105380 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe70e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe70e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55958/0 (socket says 172.21.15.26:55958) 2023-07-13T22:04:55.962 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe70e59c700 1 -- 172.21.15.26:0/2027574897 learned_addr learned my addr 172.21.15.26:0/2027574897 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe70e59c700 1 -- 172.21.15.26:0/2027574897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe700009550 con 0x7fe710105380 2023-07-13T22:04:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.427+0000 7fe70e59c700 1 --2- 172.21.15.26:0/2027574897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe7101060b0 tx=0x7fe700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe700011630 con 0x7fe710105380 2023-07-13T22:04:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700011790 con 0x7fe710105380 2023-07-13T22:04:55.963 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe700011a30 con 0x7fe710105380 2023-07-13T22:04:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe710079170 con 0x7fe710105380 2023-07-13T22:04:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe710079590 con 0x7fe710105380 2023-07-13T22:04:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe700022070 con 0x7fe710105380 2023-07-13T22:04:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.428+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe70001c450 con 0x7fe710105380 2023-07-13T22:04:55.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.429+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710061b40 con 0x7fe710105380 2023-07-13T22:04:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.432+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fe70001b450 con 0x7fe710105380 2023-07-13T22:04:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.505+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe7101920f0 con 0x7fe710105380 2023-07-13T22:04:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.509+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe70001b5f0 con 0x7fe710105380 2023-07-13T22:04:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.509+0000 7fe6ff7fe700 1 -- 172.21.15.26:0/2027574897 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fe700024630 con 0x7fe710105380 2023-07-13T22:04:55.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 msgr2=0x7fe71007ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:55.966 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 --2- 172.21.15.26:0/2027574897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe7101060b0 tx=0x7fe700004750 comp rx=0 tx=0).stop 2023-07-13T22:04:55.966 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 shutdown_connections 2023-07-13T22:04:55.966 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 --2- 172.21.15.26:0/2027574897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe710105380 0x7fe71007ab20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:55.966 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 >> 172.21.15.26:0/2027574897 conn(0x7fe7101009f0 msgr2=0x7fe71018cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:55.967 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 shutdown_connections 2023-07-13T22:04:55.967 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:55.510+0000 7fe714eab700 1 -- 172.21.15.26:0/2027574897 wait complete. 2023-07-13T22:04:55.967 INFO:teuthology.orchestra.run.smithi026.stdout:Generating new minimal ceph.conf... 2023-07-13T22:04:56.929 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.477+0000 7f358dd6d700 1 Processor -- start 2023-07-13T22:04:56.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.477+0000 7f358dd6d700 1 -- start start 2023-07-13T22:04:56.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.477+0000 7f358dd6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:56.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.478+0000 7f358dd6d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880796f0 con 0x7f358807acb0 2023-07-13T22:04:56.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.478+0000 7f35877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:56.930 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.478+0000 7f35877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55964/0 (socket says 172.21.15.26:55964) 2023-07-13T22:04:56.931 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.478+0000 7f35877fe700 1 -- 172.21.15.26:0/1173061415 learned_addr learned my addr 172.21.15.26:0/1173061415 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.478+0000 7f35877fe700 1 -- 172.21.15.26:0/1173061415 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588079830 con 0x7f358807acb0 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f35877fe700 1 --2- 172.21.15.26:0/1173061415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f35780098a0 tx=0x7f3578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cad299ce67bd8c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f35867fc700 1 -- 172.21.15.26:0/1173061415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3578004160 con 0x7f358807acb0 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f35867fc700 1 -- 172.21.15.26:0/1173061415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f35780042c0 con 0x7f358807acb0 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f35867fc700 1 -- 172.21.15.26:0/1173061415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3578011a00 con 0x7f358807acb0 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f358dd6d700 1 -- 172.21.15.26:0/1173061415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 msgr2=0x7f35880791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:56.932 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f358dd6d700 1 --2- 172.21.15.26:0/1173061415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f35780098a0 tx=0x7f3578009bb0 comp rx=0 tx=0).stop 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f358dd6d700 1 -- 172.21.15.26:0/1173061415 shutdown_connections 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f358dd6d700 1 --2- 172.21.15.26:0/1173061415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f35880791b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.479+0000 7f358dd6d700 1 -- 172.21.15.26:0/1173061415 >> 172.21.15.26:0/1173061415 conn(0x7f35881009f0 msgr2=0x7f3588102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.480+0000 7f358dd6d700 1 -- 172.21.15.26:0/1173061415 shutdown_connections 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.480+0000 7f358dd6d700 1 -- 172.21.15.26:0/1173061415 wait complete. 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f358dd6d700 1 Processor -- start 2023-07-13T22:04:56.933 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f358dd6d700 1 -- start start 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f358dd6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f358dd6d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358819b880 con 0x7f358807acb0 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f35877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f35877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55968/0 (socket says 172.21.15.26:55968) 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.481+0000 7f35877fe700 1 -- 172.21.15.26:0/1888890891 learned_addr learned my addr 172.21.15.26:0/1888890891 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f35877fe700 1 -- 172.21.15.26:0/1888890891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578009550 con 0x7f358807acb0 2023-07-13T22:04:56.934 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f35877fe700 1 --2- 172.21.15.26:0/1888890891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3578006af0 tx=0x7f3578011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3578003a40 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3578003ba0 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f357801a700 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358819ba80 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.482+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358819bea0 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.483+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f357801ac80 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.483+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3578019700 con 0x7f358807acb0 2023-07-13T22:04:56.935 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.483+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588061b40 con 0x7f358807acb0 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.486+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f3578024ad0 con 0x7f358807acb0 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.560+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f358819e8e0 con 0x7f358807acb0 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.561+0000 7f3584ff9700 1 -- 172.21.15.26:0/1888890891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f3578024c70 con 0x7f358807acb0 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 msgr2=0x7f358819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 --2- 172.21.15.26:0/1888890891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3578006af0 tx=0x7f3578011ad0 comp rx=0 tx=0).stop 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 shutdown_connections 2023-07-13T22:04:56.936 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 --2- 172.21.15.26:0/1888890891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f358807acb0 0x7f358819b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:04:56.937 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 >> 172.21.15.26:0/1888890891 conn(0x7f35881009f0 msgr2=0x7f35881020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:04:56.937 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 shutdown_connections 2023-07-13T22:04:56.937 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:56.562+0000 7f358dd6d700 1 -- 172.21.15.26:0/1888890891 wait complete. 2023-07-13T22:04:56.937 INFO:teuthology.orchestra.run.smithi026.stdout:Restarting the monitor... 2023-07-13T22:04:56.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:56 smithi026 ceph-mon[95144]: from='client.? 172.21.15.26:0/1888890891' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:04:57.384 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:56 smithi026 systemd[1]: Stopping Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:04:57.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:57 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a[95120]: 2023-07-13T22:04:57.385+0000 7f5870a9b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:04:57.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:57 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a[95120]: 2023-07-13T22:04:57.385+0000 7f5870a9b700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-13T22:04:58.120 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:57 smithi026 bash[95483]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a 2023-07-13T22:04:58.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 systemd[1]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service: Succeeded. 2023-07-13T22:04:58.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 systemd[1]: Stopped Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:04:58.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 systemd[1]: Starting Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:04:58.856 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 podman[95607]: 2023-07-13T22:04:58.856 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 bash[95607]: be719f8b8815814af0f2bf550b740dbace51c8c53b283fe76385bb259ec569fb 2023-07-13T22:04:58.857 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: set uid:gid to 167:167 (ceph:ceph) 2023-07-13T22:04:58.906 INFO:teuthology.orchestra.run.smithi026.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-13T22:04:58.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-13T22:04:58.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: pidfile_write: ignore empty --pid-file 2023-07-13T22:04:58.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: load: jerasure load: lrc 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: RocksDB version: 7.9.2 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Git sha 0 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: DB SUMMARY 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: DB Session ID: 8ZCQUPA6JK15VDQRXHZJ 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: CURRENT file: CURRENT 2023-07-13T22:04:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: IDENTITY file: IDENTITY 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 99079 ; 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.error_if_exists: 0 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.create_if_missing: 0 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.paranoid_checks: 1 2023-07-13T22:04:58.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.env: 0x560facd65720 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.fs: PosixFileSystem 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.info_log: 0x560fae67d280 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_file_opening_threads: 16 2023-07-13T22:04:58.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.statistics: (nil) 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.use_fsync: 0 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_log_file_size: 0 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.keep_log_file_num: 1000 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.recycle_log_file_num: 0 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_fallocate: 1 2023-07-13T22:04:58.934 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_mmap_reads: 0 2023-07-13T22:04:58.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_mmap_writes: 0 2023-07-13T22:04:58.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.use_direct_reads: 0 2023-07-13T22:04:58.935 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-13T22:04:58.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.create_missing_column_families: 0 2023-07-13T22:04:58.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.db_log_dir: 2023-07-13T22:04:58.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.wal_dir: 2023-07-13T22:04:58.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-13T22:04:58.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-13T22:04:58.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-13T22:04:58.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-13T22:04:58.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.advise_random_on_open: 1 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.db_write_buffer_size: 0 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.write_buffer_manager: 0x560fae59e3c0 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-13T22:04:58.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.rate_limiter: (nil) 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.wal_recovery_mode: 2 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enable_thread_tracking: 0 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enable_pipelined_write: 0 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.unordered_write: 0 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-13T22:04:58.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.row_cache: None 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.wal_filter: None 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_ingest_behind: 0 2023-07-13T22:04:58.941 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.two_write_queues: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.manual_wal_flush: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.wal_compression: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.atomic_flush: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-13T22:04:58.942 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.log_readahead_size: 0 2023-07-13T22:04:58.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-13T22:04:58.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.best_efforts_recovery: 0 2023-07-13T22:04:58.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-13T22:04:58.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-13T22:04:58.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.allow_data_in_errors: 0 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.db_host_id: __hostname__ 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_background_jobs: 2 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_background_compactions: -1 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_subcompactions: 1 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-13T22:04:58.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_total_wal_size: 0 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_open_files: -1 2023-07-13T22:04:58.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bytes_per_sync: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_readahead_size: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_background_flushes: -1 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Compression algorithms supported: 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kZSTD supported: 0 2023-07-13T22:04:58.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kXpressCompression supported: 0 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kLZ4HCCompression supported: 1 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kZlibCompression supported: 1 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kSnappyCompression supported: 1 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kLZ4Compression supported: 1 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: kBZip2Compression supported: 0 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-13T22:04:58.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.merge_operator: 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_filter: None 2023-07-13T22:04:58.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_filter_factory: None 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.sst_partitioner_factory: None 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560fae67d380) 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks: 1 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-13T22:04:58.949 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: pin_top_level_index_and_filter: 1 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_type: 0 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_index_type: 0 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_shortening: 1 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: checksum: 4 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: no_block_cache: 0 2023-07-13T22:04:58.950 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache: 0x560fae63d1f0 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_name: BinnedLRUCache 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_options: 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: capacity : 536870912 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_shard_bits : 4 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: strict_capacity_limit : 0 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: high_pri_pool_ratio: 0.000 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_cache_compressed: (nil) 2023-07-13T22:04:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout: persistent_cache: (nil) 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size: 4096 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_size_deviation: 10 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_restart_interval: 16 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: index_block_restart_interval: 1 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: metadata_block_size: 4096 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: partition_filters: 0 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: use_delta_encoding: 1 2023-07-13T22:04:58.952 INFO:journalctl@ceph.mon.a.smithi026.stdout: filter_policy: bloomfilter 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: whole_key_filtering: 1 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: verify_compression: 0 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: read_amp_bytes_per_bit: 0 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: format_version: 5 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_index_compression: 1 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: block_align: 0 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: max_auto_readahead_size: 262144 2023-07-13T22:04:58.953 INFO:journalctl@ceph.mon.a.smithi026.stdout: prepopulate_block_cache: 0 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout: initial_auto_readahead_size: 8192 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout: num_file_reads_for_auto_readahead: 2 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.write_buffer_size: 33554432 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_write_buffer_number: 2 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression: NoCompression 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression: Disabled 2023-07-13T22:04:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.prefix_extractor: nullptr 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.num_levels: 7 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-13T22:04:58.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-13T22:04:58.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.level: 32767 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.strategy: 0 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.enabled: false 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:04:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.target_file_size_base: 67108864 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-13T22:04:58.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-13T22:04:58.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.arena_block_size: 1048576 2023-07-13T22:04:58.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-13T22:04:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-13T22:04:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.disable_auto_compactions: 0 2023-07-13T22:04:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-13T22:04:58.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-13T22:04:58.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-13T22:04:58.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-13T22:04:58.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-13T22:04:58.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-13T22:04:58.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-13T22:04:58.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-13T22:04:58.963 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-13T22:04:58.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-13T22:04:58.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.table_properties_collectors: 2023-07-13T22:04:58.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.inplace_update_support: 0 2023-07-13T22:04:58.964 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-13T22:04:58.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-13T22:04:58.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-13T22:04:58.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-13T22:04:58.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.bloom_locality: 0 2023-07-13T22:04:58.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.max_successive_merges: 0 2023-07-13T22:04:58.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-13T22:04:58.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.paranoid_file_checks: 0 2023-07-13T22:04:58.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.force_consistency_checks: 1 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.report_bg_io_stats: 0 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.ttl: 2592000 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enable_blob_files: false 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.min_blob_size: 0 2023-07-13T22:04:58.967 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_file_size: 268435456 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-13T22:04:58.968 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.blob_file_starting_level: 0 2023-07-13T22:04:58.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-13T22:04:58.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-13T22:04:58.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-13T22:04:58.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8a3bb05d-f513-43f8-882d-b333b3de396a 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285898867438, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285898869270, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94298, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92296, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13747, "raw_average_key_size": 49, "raw_value_size": 84793, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689285898, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8a3bb05d-f513-43f8-882d-b333b3de396a", "db_session_id": "8ZCQUPA6JK15VDQRXHZJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285898869428, "job": 1, "event": "recovery_finished"} 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-13T22:04:58.970 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560fae6d6000 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: DB pointer 0x560fae6c2000 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** DB Stats ** 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.972 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: L0 2/0 93.92 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 71.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: Sum 2/0 93.92 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 71.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 71.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** Compaction Stats [default] ** 2023-07-13T22:04:58.973 INFO:journalctl@ceph.mon.a.smithi026.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:04:58.974 INFO:journalctl@ceph.mon.a.smithi026.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-13T22:04:58.974 INFO:journalctl@ceph.mon.a.smithi026.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 71.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-13T22:04:58.974 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.974 INFO:journalctl@ceph.mon.a.smithi026.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-13T22:04:58.974 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-13T22:04:58.975 INFO:journalctl@ceph.mon.a.smithi026.stdout: Cumulative compaction: 0.00 GB write, 14.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: Interval compaction: 0.00 GB write, 14.02 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache BinnedLRUCache@0x560fae63d1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.976 INFO:journalctl@ceph.mon.a.smithi026.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: starting mon.a rank 0 at public addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] at bind addrs [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???) e1 preinit fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).mds e1 new map 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).mds e1 print_map 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout: e1 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-13T22:04:58.977 INFO:journalctl@ceph.mon.a.smithi026.stdout: legacy client fscid: -1 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout: No filesystems configured 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-13T22:04:58.978 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expand map: {default=false} 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta from 'false' to 'false' 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expanded map: {default=false} 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expand map: {default=info} 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta from 'info' to 'info' 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expanded map: {default=info} 2023-07-13T22:04:58.979 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expand map: {default=daemon} 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expanded map: {default=daemon} 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expand map: {default=debug} 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta from 'debug' to 'debug' 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: expand_channel_meta expanded map: {default=debug} 2023-07-13T22:04:58.980 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 systemd[1]: Started Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:04:58.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-13T22:04:58.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: monmap e1: 1 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0]} removed_ranks: {} 2023-07-13T22:04:58.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: fsmap 2023-07-13T22:04:58.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: osdmap e1: 0 total, 0 up, 0 in 2023-07-13T22:04:58.981 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:04:58 smithi026 ceph-mon[95647]: mgrmap e1: no daemons active 2023-07-13T22:04:59.997 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.482+0000 7f36452f5700 1 Processor -- start 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.483+0000 7f36452f5700 1 -- start start 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.483+0000 7f36452f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.483+0000 7f36452f5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36400796f0 con 0x7f364007acb0 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.484+0000 7f363effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.484+0000 7f363effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55972/0 (socket says 172.21.15.26:55972) 2023-07-13T22:04:59.998 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.484+0000 7f363effd700 1 -- 172.21.15.26:0/3310367763 learned_addr learned my addr 172.21.15.26:0/3310367763 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:04:59.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.484+0000 7f363effd700 1 -- 172.21.15.26:0/3310367763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3640079830 con 0x7f364007acb0 2023-07-13T22:04:59.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.484+0000 7f363effd700 1 --2- 172.21.15.26:0/3310367763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f36300098a0 tx=0x7f3630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e710cf7e666724 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:04:59.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f363dffb700 1 -- 172.21.15.26:0/3310367763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3630004160 con 0x7f364007acb0 2023-07-13T22:04:59.999 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f363dffb700 1 -- 172.21.15.26:0/3310367763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f36300042c0 con 0x7f364007acb0 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f363dffb700 1 -- 172.21.15.26:0/3310367763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3630011a00 con 0x7f364007acb0 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 -- 172.21.15.26:0/3310367763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 msgr2=0x7f36400791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 --2- 172.21.15.26:0/3310367763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f36300098a0 tx=0x7f3630009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 -- 172.21.15.26:0/3310367763 shutdown_connections 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 --2- 172.21.15.26:0/3310367763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f36400791b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:00.000 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 -- 172.21.15.26:0/3310367763 >> 172.21.15.26:0/3310367763 conn(0x7f36401009f0 msgr2=0x7f3640102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 -- 172.21.15.26:0/3310367763 shutdown_connections 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.485+0000 7f36452f5700 1 -- 172.21.15.26:0/3310367763 wait complete. 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.486+0000 7f36452f5700 1 Processor -- start 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f36452f5700 1 -- start start 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f36452f5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f36452f5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364019b880 con 0x7f364007acb0 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f363effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:00.001 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f363effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55986/0 (socket says 172.21.15.26:55986) 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.487+0000 7f363effd700 1 -- 172.21.15.26:0/1218035299 learned_addr learned my addr 172.21.15.26:0/1218035299 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f363effd700 1 -- 172.21.15.26:0/1218035299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3630009550 con 0x7f364007acb0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f363effd700 1 --2- 172.21.15.26:0/1218035299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fb0 tx=0x7f3630011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3630003930 con 0x7f364007acb0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3630003a90 con 0x7f364007acb0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36300195e0 con 0x7f364007acb0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364019ba80 con 0x7f364007acb0 2023-07-13T22:05:00.002 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.488+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364019bea0 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.489+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3630019b50 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.489+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f36300243f0 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.489+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3640061b40 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.492+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f363002de30 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.565+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f364019e8e0 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f3627fff700 1 -- 172.21.15.26:0/1218035299 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3630003c00 con 0x7f364007acb0 2023-07-13T22:05:00.003 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 msgr2=0x7f364019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f36452f5700 1 --2- 172.21.15.26:0/1218035299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fb0 tx=0x7f3630011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 shutdown_connections 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f36452f5700 1 --2- 172.21.15.26:0/1218035299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f364007acb0 0x7f364019b340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.568+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 >> 172.21.15.26:0/1218035299 conn(0x7f36401009f0 msgr2=0x7f36401020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.569+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 shutdown_connections 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:04:59.569+0000 7f36452f5700 1 -- 172.21.15.26:0/1218035299 wait complete. 2023-07-13T22:05:00.004 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-13T22:05:00.031 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:05:00.031 INFO:teuthology.orchestra.run.smithi026.stdout:Creating mgr... 2023-07-13T22:05:00.032 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 9283 ... 2023-07-13T22:05:00.032 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying port 8765 ... 2023-07-13T22:05:00.393 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x 2023-07-13T22:05:00.393 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Failed to reset failed state of unit ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x.service: Unit ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x.service not loaded. 2023-07-13T22:05:00.612 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c.target.wants/ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-23b26990-21c9-11ee-9b32-001a4aab830c@.service. 2023-07-13T22:05:00.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:00 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1218035299' entity='client.admin' 2023-07-13T22:05:00.710 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:00 smithi026 systemd[1]: Starting Ceph mgr.x for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:05:01.459 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 podman[95881]: 2023-07-13T22:05:01.526 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-13T22:05:01.527 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-07-13T22:05:01.543 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-13T22:05:01.543 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:05:01.543 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-07-13T22:05:01.544 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to enable service . firewalld.service is not available 2023-07-13T22:05:01.561 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-13T22:05:01.562 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-07-13T22:05:01.578 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-13T22:05:01.579 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:05:01.579 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-07-13T22:05:01.579 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-13T22:05:01.579 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr to start... 2023-07-13T22:05:01.579 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr... 2023-07-13T22:05:01.772 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 bash[95881]: ff2264bfdf99794d38110e215df7c39db9990801c8351c60a534a2924872945d 2023-07-13T22:05:01.772 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 systemd[1]: Started Ceph mgr.x for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:05:01.772 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:01.665+0000 7fe89b9e9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:05:01.772 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:01.774+0000 7fe89b9e9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:02.211 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:01 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:01.893+0000 7fe89b9e9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:05:02.211 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:02 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:02.017+0000 7fe89b9e9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:05:02.661 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:02 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:02.245+0000 7fe89b9e9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:05:02.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:02 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:02.662+0000 7fe89b9e9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:05:02.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:02 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:02.783+0000 7fe89b9e9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:05:03.459 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:02 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:02.950+0000 7fe89b9e9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:05:03.459 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:03 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:03.079+0000 7fe89b9e9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:05:03.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:03 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:03.648+0000 7fe89b9e9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:05:04.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:04 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:04.476+0000 7fe89b9e9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:05:04.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:04 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:04.690+0000 7fe89b9e9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:05:05.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:05 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:05.688+0000 7fe89b9e9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:05.960 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:05 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:05.812+0000 7fe89b9e9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:05:06.411 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:06.041+0000 7fe89b9e9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:06.696 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:06.414+0000 7fe89b9e9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:05:06.960 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:06.699+0000 7fe89b9e9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:05:06.961 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:06.909+0000 7fe89b9e9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:05:07.262 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2057958318' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-13T22:05:07.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:07 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:07.265+0000 7fe89b9e9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:05:08.650 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:08.225+0000 7fe89b9e9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:05:08.651 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:08.403+0000 7fe89b9e9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:05:08.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:08.653+0000 7fe89b9e9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:05:08.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:08.806+0000 7fe89b9e9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:09.460 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:09.096+0000 7fe89b9e9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:05:10.344 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:10.219+0000 7fe89b9e9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:05:10.344 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: Activating manager daemon x 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: mgrmap e2: x(active, starting, since 0.00431527s) 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: Manager daemon x is now available 2023-07-13T22:05:10.345 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-13T22:05:10.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-13T22:05:10.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' 2023-07-13T22:05:10.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' 2023-07-13T22:05:10.346 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:10 smithi026 ceph-mon[95647]: from='mgr.14100 172.21.15.26:0/85466077' entity='mgr.x' 2023-07-13T22:05:10.382 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:05:10.382 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:10.382 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:05:10.382 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-13T22:05:10.383 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:10.384 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-13T22:05:10.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-13T22:05:10.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-13T22:05:10.385 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:10.386 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-13T22:05:10.391 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.392 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-13T22:05:10.393 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-13T22:05:10.394 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-13T22:05:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-13T22:05:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-13T22:05:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-13T22:05:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-13T22:05:10.395 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2023-07-13T22:04:53.734956+0000", 2023-07-13T22:05:10.396 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.888+0000 7f30e94ce700 1 Processor -- start 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.888+0000 7f30e94ce700 1 -- start start 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e94ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e94ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4073c20 con 0x7f30e4105e90 2023-07-13T22:05:10.397 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:10.398 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35942/0 (socket says 172.21.15.26:35942) 2023-07-13T22:05:10.398 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e2ffd700 1 -- 172.21.15.26:0/1519165444 learned_addr learned my addr 172.21.15.26:0/1519165444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:10.398 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.889+0000 7f30e2ffd700 1 -- 172.21.15.26:0/1519165444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4073d60 con 0x7f30e4105e90 2023-07-13T22:05:10.398 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.890+0000 7f30e2ffd700 1 --2- 172.21.15.26:0/1519165444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098a0 tx=0x7f30d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37345e8bc1bea26 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:10.398 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.890+0000 7f30e1ffb700 1 -- 172.21.15.26:0/1519165444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d4004160 con 0x7f30e4105e90 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.890+0000 7f30e1ffb700 1 -- 172.21.15.26:0/1519165444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d40042c0 con 0x7f30e4105e90 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.890+0000 7f30e1ffb700 1 -- 172.21.15.26:0/1519165444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d4011a00 con 0x7f30e4105e90 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 -- 172.21.15.26:0/1519165444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 msgr2=0x7f30e4108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 --2- 172.21.15.26:0/1519165444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f30d40098a0 tx=0x7f30d4009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 -- 172.21.15.26:0/1519165444 shutdown_connections 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 --2- 172.21.15.26:0/1519165444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e4108270 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 -- 172.21.15.26:0/1519165444 >> 172.21.15.26:0/1519165444 conn(0x7f30e40ff8d0 msgr2=0x7f30e4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:10.399 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 -- 172.21.15.26:0/1519165444 shutdown_connections 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.891+0000 7f30e94ce700 1 -- 172.21.15.26:0/1519165444 wait complete. 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e94ce700 1 Processor -- start 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e94ce700 1 -- start start 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e94ce700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e94ce700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4199620 con 0x7f30e4105e90 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:10.400 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.892+0000 7f30e2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35944/0 (socket says 172.21.15.26:35944) 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30e2ffd700 1 -- 172.21.15.26:0/2057958318 learned_addr learned my addr 172.21.15.26:0/2057958318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30e2ffd700 1 -- 172.21.15.26:0/2057958318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d4009550 con 0x7f30e4105e90 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30e2ffd700 1 --2- 172.21.15.26:0/2057958318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006af0 tx=0x7f30d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d4003a40 con 0x7f30e4105e90 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30d4003ba0 con 0x7f30e4105e90 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30d401a700 con 0x7f30e4105e90 2023-07-13T22:05:10.401 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.893+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e4199820 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.894+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e4199c40 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.894+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f30d401ac30 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.894+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f30d40193c0 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.895+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4061b40 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.898+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f30d4024770 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.978+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f30e419c680 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.979+0000 7f30cbfff700 1 -- 172.21.15.26:0/2057958318 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f30d4024910 con 0x7f30e4105e90 2023-07-13T22:05:10.402 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 msgr2=0x7f30e41990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 --2- 172.21.15.26:0/2057958318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f30d4006af0 tx=0x7f30d4011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 shutdown_connections 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 --2- 172.21.15.26:0/2057958318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4105e90 0x7f30e41990e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 >> 172.21.15.26:0/2057958318 conn(0x7f30e40ff8d0 msgr2=0x7f30e4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 shutdown_connections 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:06.980+0000 7f30e94ce700 1 -- 172.21.15.26:0/2057958318 wait complete. 2023-07-13T22:05:10.403 INFO:teuthology.orchestra.run.smithi026.stdout:mgr not available, waiting (1/15)... 2023-07-13T22:05:12.670 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:12 smithi026 ceph-mon[95647]: mgrmap e3: x(active, since 1.00985s) 2023-07-13T22:05:13.660 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:13 smithi026 ceph-mon[95647]: mgrmap e4: x(active, since 2s) 2023-07-13T22:05:13.673 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:05:13.675 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "health": { 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-13T22:05:13.676 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 0 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "a" 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:13.677 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:13.678 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-13T22:05:13.679 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-13T22:05:13.684 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-13T22:05:13.685 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-13T22:05:13.686 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-13T22:05:13.687 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "restful" 2023-07-13T22:05:13.688 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ], 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "modified": "2023-07-13T22:04:53.734956+0000", 2023-07-13T22:05:13.689 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout }, 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.940+0000 7fb89e76f700 1 Processor -- start 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.940+0000 7fb89e76f700 1 -- start start 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.941+0000 7fb89e76f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:13.690 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.941+0000 7fb89e76f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981084a0 con 0x7fb898073b60 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.941+0000 7fb897fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.941+0000 7fb897fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40524/0 (socket says 172.21.15.26:40524) 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.941+0000 7fb897fff700 1 -- 172.21.15.26:0/254753273 learned_addr learned my addr 172.21.15.26:0/254753273 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.942+0000 7fb897fff700 1 -- 172.21.15.26:0/254753273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8981085e0 con 0x7fb898073b60 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.942+0000 7fb897fff700 1 --2- 172.21.15.26:0/254753273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8880098a0 tx=0x7fb888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1652c2b0cdc8f2d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.942+0000 7fb896ffd700 1 -- 172.21.15.26:0/254753273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb888004160 con 0x7fb898073b60 2023-07-13T22:05:13.691 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.942+0000 7fb896ffd700 1 -- 172.21.15.26:0/254753273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8880042c0 con 0x7fb898073b60 2023-07-13T22:05:13.692 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.942+0000 7fb896ffd700 1 -- 172.21.15.26:0/254753273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb888011a00 con 0x7fb898073b60 2023-07-13T22:05:13.692 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 -- 172.21.15.26:0/254753273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 msgr2=0x7fb898107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:13.692 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 --2- 172.21.15.26:0/254753273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb8880098a0 tx=0x7fb888009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:13.693 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 -- 172.21.15.26:0/254753273 shutdown_connections 2023-07-13T22:05:13.693 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 --2- 172.21.15.26:0/254753273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb898107f60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:13.693 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 -- 172.21.15.26:0/254753273 >> 172.21.15.26:0/254753273 conn(0x7fb8980ff8d0 msgr2=0x7fb898101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:13.693 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 -- 172.21.15.26:0/254753273 shutdown_connections 2023-07-13T22:05:13.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.943+0000 7fb89e76f700 1 -- 172.21.15.26:0/254753273 wait complete. 2023-07-13T22:05:13.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.944+0000 7fb89e76f700 1 Processor -- start 2023-07-13T22:05:13.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.944+0000 7fb89e76f700 1 -- start start 2023-07-13T22:05:13.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.944+0000 7fb89e76f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:13.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.944+0000 7fb89e76f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89819b8b0 con 0x7fb898073b60 2023-07-13T22:05:13.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb897fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:13.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb897fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40540/0 (socket says 172.21.15.26:40540) 2023-07-13T22:05:13.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb897fff700 1 -- 172.21.15.26:0/2892509618 learned_addr learned my addr 172.21.15.26:0/2892509618 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:13.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb897fff700 1 -- 172.21.15.26:0/2892509618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888009550 con 0x7fb898073b60 2023-07-13T22:05:13.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb897fff700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb888009fb0 tx=0x7fb888011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:13.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8880039c0 con 0x7fb898073b60 2023-07-13T22:05:13.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.945+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb888003b20 con 0x7fb898073b60 2023-07-13T22:05:13.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.946+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89819bab0 con 0x7fb898073b60 2023-07-13T22:05:13.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.946+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb88801a780 con 0x7fb898073b60 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.946+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89819bed0 con 0x7fb898073b60 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.947+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fb888024430 con 0x7fb898073b60 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.947+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb898061b40 con 0x7fb898073b60 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.947+0000 7fb8957fa700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 0x7fb88003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.947+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb88804c3c0 con 0x7fb898073b60 2023-07-13T22:05:13.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.947+0000 7fb8977fe700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 0x7fb88003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:13.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.948+0000 7fb8977fe700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 0x7fb88003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:13.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:12.953+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb888024af0 con 0x7fb898073b60 2023-07-13T22:05:13.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.239+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb89819efd0 con 0x7fb898073b60 2023-07-13T22:05:13.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.240+0000 7fb8957fa700 1 -- 172.21.15.26:0/2892509618 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb888018350 con 0x7fb898073b60 2023-07-13T22:05:13.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 msgr2=0x7fb88003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 0x7fb88003ab70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 msgr2=0x7fb89819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb888009fb0 tx=0x7fb888011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 shutdown_connections 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7fb8800386c0 0x7fb88003ab70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:13.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 --2- 172.21.15.26:0/2892509618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb898073b60 0x7fb89819b370 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:13.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.242+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 >> 172.21.15.26:0/2892509618 conn(0x7fb8980ff8d0 msgr2=0x7fb898100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:13.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.243+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 shutdown_connections 2023-07-13T22:05:13.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:13.243+0000 7fb89e76f700 1 -- 172.21.15.26:0/2892509618 wait complete. 2023-07-13T22:05:13.701 INFO:teuthology.orchestra.run.smithi026.stdout:mgr is available 2023-07-13T22:05:14.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:14 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2892509618' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-13T22:05:15.065 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:05:15.065 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [global] 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout fsid = 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:05:15.066 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout [osd] 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-13T22:05:15.067 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.268+0000 7f1e785af700 1 Processor -- start 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.268+0000 7f1e785af700 1 -- start start 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.268+0000 7f1e785af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.268+0000 7f1e785af700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700796f0 con 0x7f1e7007acb0 2023-07-13T22:05:15.068 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.269+0000 7f1e7634b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.269+0000 7f1e7634b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40544/0 (socket says 172.21.15.26:40544) 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.269+0000 7f1e7634b700 1 -- 172.21.15.26:0/4095881133 learned_addr learned my addr 172.21.15.26:0/4095881133 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.269+0000 7f1e7634b700 1 -- 172.21.15.26:0/4095881133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70079830 con 0x7f1e7007acb0 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.269+0000 7f1e7634b700 1 --2- 172.21.15.26:0/4095881133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0098a0 tx=0x7f1e6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00be181d7d82be2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e75349700 1 -- 172.21.15.26:0/4095881133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6c004160 con 0x7f1e7007acb0 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e75349700 1 -- 172.21.15.26:0/4095881133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1e6c0042c0 con 0x7f1e7007acb0 2023-07-13T22:05:15.069 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e75349700 1 -- 172.21.15.26:0/4095881133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6c011a00 con 0x7f1e7007acb0 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 -- 172.21.15.26:0/4095881133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 msgr2=0x7f1e700791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 --2- 172.21.15.26:0/4095881133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0098a0 tx=0x7f1e6c009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 -- 172.21.15.26:0/4095881133 shutdown_connections 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 --2- 172.21.15.26:0/4095881133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e700791b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 -- 172.21.15.26:0/4095881133 >> 172.21.15.26:0/4095881133 conn(0x7f1e701009f0 msgr2=0x7f1e70102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.270+0000 7f1e785af700 1 -- 172.21.15.26:0/4095881133 shutdown_connections 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.271+0000 7f1e785af700 1 -- 172.21.15.26:0/4095881133 wait complete. 2023-07-13T22:05:15.070 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.271+0000 7f1e785af700 1 Processor -- start 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e785af700 1 -- start start 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e785af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e785af700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7019b880 con 0x7f1e7007acb0 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e7634b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e7634b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40556/0 (socket says 172.21.15.26:40556) 2023-07-13T22:05:15.071 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.272+0000 7f1e7634b700 1 -- 172.21.15.26:0/1897914882 learned_addr learned my addr 172.21.15.26:0/1897914882 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e7634b700 1 -- 172.21.15.26:0/1897914882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c009550 con 0x7f1e7007acb0 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e7634b700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006af0 tx=0x7f1e6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6c003a40 con 0x7f1e7007acb0 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1e6c003ba0 con 0x7f1e7007acb0 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6c01a700 con 0x7f1e7007acb0 2023-07-13T22:05:15.072 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7019ba80 con 0x7f1e7007acb0 2023-07-13T22:05:15.073 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.273+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7019bea0 con 0x7f1e7007acb0 2023-07-13T22:05:15.073 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.274+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70061b40 con 0x7f1e7007acb0 2023-07-13T22:05:15.073 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.275+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f1e6c024ad0 con 0x7f1e7007acb0 2023-07-13T22:05:15.073 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.275+0000 7f1e677fe700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 0x7f1e5c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.275+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1e6c04c6b0 con 0x7f1e7007acb0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.275+0000 7f1e75b4a700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 0x7f1e5c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.276+0000 7f1e75b4a700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 0x7f1e5c03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.280+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1e6c018bd0 con 0x7f1e7007acb0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.483+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f1e7019ed20 con 0x7f1e7007acb0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.484+0000 7f1e677fe700 1 -- 172.21.15.26:0/1897914882 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f1e6c0187c0 con 0x7f1e7007acb0 2023-07-13T22:05:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 msgr2=0x7f1e5c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 0x7f1e5c03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 msgr2=0x7f1e7019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006af0 tx=0x7f1e6c011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 shutdown_connections 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f1e5c0386c0 0x7f1e5c03ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 --2- 172.21.15.26:0/1897914882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e7007acb0 0x7f1e7019b340 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:15.075 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.486+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 >> 172.21.15.26:0/1897914882 conn(0x7f1e701009f0 msgr2=0x7f1e701020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:15.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.487+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 shutdown_connections 2023-07-13T22:05:15.076 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:14.487+0000 7f1e785af700 1 -- 172.21.15.26:0/1897914882 wait complete. 2023-07-13T22:05:15.076 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling cephadm module... 2023-07-13T22:05:15.375 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:15 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1897914882' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-13T22:05:16.511 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2423367511' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-13T22:05:16.512 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: ignoring --setuser ceph since I am not root 2023-07-13T22:05:16.512 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: ignoring --setgroup ceph since I am not root 2023-07-13T22:05:16.513 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:16.514+0000 7f2da6a19200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:05:16.650 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.610+0000 7ff3d0cdf700 1 Processor -- start 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.610+0000 7ff3d0cdf700 1 -- start start 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.610+0000 7ff3d0cdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.610+0000 7ff3d0cdf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc079710 con 0x7ff3cc07acd0 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40558/0 (socket says 172.21.15.26:40558) 2023-07-13T22:05:16.651 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3cb7fe700 1 -- 172.21.15.26:0/1652662574 learned_addr learned my addr 172.21.15.26:0/1652662574 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3cb7fe700 1 -- 172.21.15.26:0/1652662574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc079850 con 0x7ff3cc07acd0 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3cb7fe700 1 --2- 172.21.15.26:0/1652662574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0098a0 tx=0x7ff3bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de49168da7c8cbb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.611+0000 7ff3ca7fc700 1 -- 172.21.15.26:0/1652662574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc004160 con 0x7ff3cc07acd0 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3ca7fc700 1 -- 172.21.15.26:0/1652662574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3bc0042c0 con 0x7ff3cc07acd0 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3ca7fc700 1 -- 172.21.15.26:0/1652662574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc011a00 con 0x7ff3cc07acd0 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/1652662574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 msgr2=0x7ff3cc0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:16.652 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/1652662574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0098a0 tx=0x7ff3bc009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/1652662574 shutdown_connections 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/1652662574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc0791d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/1652662574 >> 172.21.15.26:0/1652662574 conn(0x7ff3cc100810 msgr2=0x7ff3cc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/1652662574 shutdown_connections 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.612+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/1652662574 wait complete. 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.613+0000 7ff3d0cdf700 1 Processor -- start 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.613+0000 7ff3d0cdf700 1 -- start start 2023-07-13T22:05:16.653 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3d0cdf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3d0cdf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc19b780 con 0x7ff3cc07acd0 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3cb7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40568/0 (socket says 172.21.15.26:40568) 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3cb7fe700 1 -- 172.21.15.26:0/2423367511 learned_addr learned my addr 172.21.15.26:0/2423367511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.614+0000 7ff3cb7fe700 1 -- 172.21.15.26:0/2423367511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc009550 con 0x7ff3cc07acd0 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3cb7fe700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fb0 tx=0x7ff3bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc0039c0 con 0x7ff3cc07acd0 2023-07-13T22:05:16.654 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff3bc003b20 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc19b980 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3bc01a5e0 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.615+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc19bda0 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.616+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7ff3bc024430 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.617+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc1951a0 con 0x7ff3cc07acd0 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.617+0000 7ff3c8ff9700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 0x7ff3b403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:16.655 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.617+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3bc04c410 con 0x7ff3cc07acd0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.617+0000 7ff3caffd700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 0x7ff3b403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.618+0000 7ff3caffd700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 0x7ff3b403ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006fd0 tx=0x7ff3c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.622+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff3bc003c90 con 0x7ff3cc07acd0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:15.890+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ff3cc061b40 con 0x7ff3cc07acd0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.237+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff3bc02d910 con 0x7ff3cc07acd0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.239+0000 7ff3c8ff9700 1 -- 172.21.15.26:0/2423367511 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7ff3bc02c350 con 0x7ff3cc07acd0 2023-07-13T22:05:16.656 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 msgr2=0x7ff3b403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:16.657 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 0x7ff3b403ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0006fd0 tx=0x7ff3c0006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:16.658 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 msgr2=0x7ff3cc19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:16.658 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009fb0 tx=0x7ff3bc011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 shutdown_connections 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7ff3b40386c0 0x7ff3b403ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 --2- 172.21.15.26:0/2423367511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3cc07acd0 0x7ff3cc19b240 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 >> 172.21.15.26:0/2423367511 conn(0x7ff3cc100810 msgr2=0x7ff3cc102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 shutdown_connections 2023-07-13T22:05:16.659 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:16.242+0000 7ff3d0cdf700 1 -- 172.21.15.26:0/2423367511 wait complete. 2023-07-13T22:05:16.871 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:16.621+0000 7f2da6a19200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:16.871 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:16.748+0000 7f2da6a19200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:05:17.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:16.873+0000 7f2da6a19200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:05:17.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:17.113+0000 7f2da6a19200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:05:17.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2423367511' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-13T22:05:17.507 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-mon[95647]: mgrmap e5: x(active, since 6s) 2023-07-13T22:05:17.795 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:17.520+0000 7f2da6a19200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:05:17.796 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:17.643+0000 7f2da6a19200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:05:18.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:18.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-13T22:05:18.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-13T22:05:18.035 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-13T22:05:18.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-13T22:05:18.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:18.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.208+0000 7f265e223700 1 Processor -- start 2023-07-13T22:05:18.036 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.208+0000 7f265e223700 1 -- start start 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.208+0000 7f265e223700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.208+0000 7f265e223700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580796f0 con 0x7f265807acb0 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.209+0000 7f26577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.209+0000 7f26577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40608/0 (socket says 172.21.15.26:40608) 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.209+0000 7f26577fe700 1 -- 172.21.15.26:0/3737710462 learned_addr learned my addr 172.21.15.26:0/3737710462 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:18.037 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.209+0000 7f26577fe700 1 -- 172.21.15.26:0/3737710462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658079830 con 0x7f265807acb0 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.209+0000 7f26577fe700 1 --2- 172.21.15.26:0/3737710462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26480098a0 tx=0x7f2648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad088ec565af3cc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.210+0000 7f26567fc700 1 -- 172.21.15.26:0/3737710462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648004160 con 0x7f265807acb0 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.210+0000 7f26567fc700 1 -- 172.21.15.26:0/3737710462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26480042c0 con 0x7f265807acb0 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.210+0000 7f26567fc700 1 -- 172.21.15.26:0/3737710462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648011a00 con 0x7f265807acb0 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.210+0000 7f265e223700 1 -- 172.21.15.26:0/3737710462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 msgr2=0x7f26580791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 --2- 172.21.15.26:0/3737710462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f26480098a0 tx=0x7f2648009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.038 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 -- 172.21.15.26:0/3737710462 shutdown_connections 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 --2- 172.21.15.26:0/3737710462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f26580791b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 -- 172.21.15.26:0/3737710462 >> 172.21.15.26:0/3737710462 conn(0x7f26581009f0 msgr2=0x7f2658102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 -- 172.21.15.26:0/3737710462 shutdown_connections 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.211+0000 7f265e223700 1 -- 172.21.15.26:0/3737710462 wait complete. 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.212+0000 7f265e223700 1 Processor -- start 2023-07-13T22:05:18.039 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.212+0000 7f265e223700 1 -- start start 2023-07-13T22:05:18.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.212+0000 7f265e223700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:18.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.212+0000 7f265e223700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265819b810 con 0x7f265807acb0 2023-07-13T22:05:18.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.213+0000 7f26577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:18.040 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.213+0000 7f26577fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40620/0 (socket says 172.21.15.26:40620) 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.213+0000 7f26577fe700 1 -- 172.21.15.26:0/37540516 learned_addr learned my addr 172.21.15.26:0/37540516 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.213+0000 7f26577fe700 1 -- 172.21.15.26:0/37540516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2648009550 con 0x7f265807acb0 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.213+0000 7f26577fe700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2648006af0 tx=0x7f2648011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.214+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2648003a40 con 0x7f265807acb0 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.214+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2648003ba0 con 0x7f265807acb0 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.214+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265819ba10 con 0x7f265807acb0 2023-07-13T22:05:18.041 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.214+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f264801a700 con 0x7f265807acb0 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.214+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265819be30 con 0x7f265807acb0 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.215+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f264801a980 con 0x7f265807acb0 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.215+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2658061b40 con 0x7f265807acb0 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.215+0000 7f2654ff9700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 0x7f264003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.215+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f264804c720 con 0x7f265807acb0 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.215+0000 7f2656ffd700 1 -- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 msgr2=0x7f264003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:18.042 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.216+0000 7f2656ffd700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 0x7f264003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:05:18.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.221+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2648018b40 con 0x7f265807acb0 2023-07-13T22:05:18.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.417+0000 7f2656ffd700 1 -- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 msgr2=0x7f264003abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:18.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.417+0000 7f2656ffd700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 0x7f264003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-13T22:05:18.043 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.485+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f265819ef40 con 0x7f265807acb0 2023-07-13T22:05:18.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.485+0000 7f2654ff9700 1 -- 172.21.15.26:0/37540516 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f2648018b40 con 0x7f265807acb0 2023-07-13T22:05:18.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 msgr2=0x7f264003abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:05:18.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 0x7f264003abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 msgr2=0x7f265819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:18.044 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2648006af0 tx=0x7f2648011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 shutdown_connections 2023-07-13T22:05:18.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f2640038710 0x7f264003abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 --2- 172.21.15.26:0/37540516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f265807acb0 0x7f265819b2d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:18.045 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 >> 172.21.15.26:0/37540516 conn(0x7f26581009f0 msgr2=0x7f26581020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:18.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 shutdown_connections 2023-07-13T22:05:18.046 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:17.488+0000 7f265e223700 1 -- 172.21.15.26:0/37540516 wait complete. 2023-07-13T22:05:18.046 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2023-07-13T22:05:18.047 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 5... 2023-07-13T22:05:18.073 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:17.809+0000 7f2da6a19200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:05:18.074 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:17.936+0000 7f2da6a19200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:05:18.334 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:18 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/37540516' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-13T22:05:18.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:18 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:18.508+0000 7f2da6a19200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:05:19.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:19.273+0000 7f2da6a19200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:05:19.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:19.484+0000 7f2da6a19200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:05:20.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:20.457+0000 7f2da6a19200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:20.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:20.581+0000 7f2da6a19200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:05:21.176 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:20.809+0000 7f2da6a19200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:21.456 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:21.178+0000 7f2da6a19200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:05:21.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:21.458+0000 7f2da6a19200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:05:21.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:21.656+0000 7f2da6a19200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:05:22.459 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:22.009+0000 7f2da6a19200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:05:23.391 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:22.965+0000 7f2da6a19200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:05:23.392 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:23.143+0000 7f2da6a19200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:05:23.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:23.394+0000 7f2da6a19200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:05:23.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:23.544+0000 7f2da6a19200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:24.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:23.828+0000 7f2da6a19200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:05:25.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:24.919+0000 7f2da6a19200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:05:25.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: Active manager daemon x restarted 2023-07-13T22:05:25.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: Activating manager daemon x 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: osdmap e2: 0 total, 0 up, 0 in 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: mgrmap e6: x(active, starting, since 0.00384509s) 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-13T22:05:25.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-13T22:05:25.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-13T22:05:25.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: Manager daemon x is now available 2023-07-13T22:05:25.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:25.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:25.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:24 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: Found migration_current of "None". Setting to last migration. 2023-07-13T22:05:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-13T22:05:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-13T22:05:26.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:26.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:26.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:26.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:25 smithi026 ceph-mon[95647]: mgrmap e7: x(active, since 1.00718s) 2023-07-13T22:05:26.330 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:26.330 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-13T22:05:26.330 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-13T22:05:26.330 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:26.330 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.610+0000 7f889aaf4700 1 Processor -- start 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.610+0000 7f889aaf4700 1 -- start start 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.610+0000 7f889aaf4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.610+0000 7f889aaf4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894105cd0 con 0x7f8894105380 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.611+0000 7f8898890700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.611+0000 7f8898890700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40622/0 (socket says 172.21.15.26:40622) 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.611+0000 7f8898890700 1 -- 172.21.15.26:0/2710167325 learned_addr learned my addr 172.21.15.26:0/2710167325 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.611+0000 7f8898890700 1 -- 172.21.15.26:0/2710167325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894105e10 con 0x7f8894105380 2023-07-13T22:05:26.331 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.611+0000 7f8898890700 1 --2- 172.21.15.26:0/2710167325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f88880098a0 tx=0x7f8888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daff2686de7ba926 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.612+0000 7f88937fe700 1 -- 172.21.15.26:0/2710167325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8888004160 con 0x7f8894105380 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.612+0000 7f88937fe700 1 -- 172.21.15.26:0/2710167325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f88880042c0 con 0x7f8894105380 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.612+0000 7f88937fe700 1 -- 172.21.15.26:0/2710167325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8888011a00 con 0x7f8894105380 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.612+0000 7f889aaf4700 1 -- 172.21.15.26:0/2710167325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 msgr2=0x7f8894105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.612+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2710167325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f88880098a0 tx=0x7f8888009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.613+0000 7f889aaf4700 1 -- 172.21.15.26:0/2710167325 shutdown_connections 2023-07-13T22:05:26.332 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.613+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2710167325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f8894105790 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.613+0000 7f889aaf4700 1 -- 172.21.15.26:0/2710167325 >> 172.21.15.26:0/2710167325 conn(0x7f88941009f0 msgr2=0x7f8894102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.613+0000 7f889aaf4700 1 -- 172.21.15.26:0/2710167325 shutdown_connections 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.613+0000 7f889aaf4700 1 -- 172.21.15.26:0/2710167325 wait complete. 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f889aaf4700 1 Processor -- start 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f889aaf4700 1 -- start start 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f889aaf4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:26.333 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f889aaf4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889419b810 con 0x7f8894105380 2023-07-13T22:05:26.337 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f8898890700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:26.337 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f8898890700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40638/0 (socket says 172.21.15.26:40638) 2023-07-13T22:05:26.337 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.614+0000 7f8898890700 1 -- 172.21.15.26:0/2642697286 learned_addr learned my addr 172.21.15.26:0/2642697286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:26.337 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f8898890700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888009550 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f8898890700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8888006af0 tx=0x7f8888011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8888003a40 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889419ba10 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8888003ba0 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.615+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f888801a6e0 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.616+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889419be30 con 0x7f8894105380 2023-07-13T22:05:26.338 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8888024430 con 0x7f8894105380 2023-07-13T22:05:26.339 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8891ffb700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:26.339 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f888403b320 con 0x7f8884038760 2023-07-13T22:05:26.339 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f888804c710 con 0x7f8894105380 2023-07-13T22:05:26.339 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.339 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.617+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.818+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:18.818+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:19.219+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:19.219+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:20.020+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:20.020+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-13T22:05:26.340 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:21.621+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:21.621+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:24.825+0000 7f8893fff700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/3558776409 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:24.825+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:24.925+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f8888024a60 con 0x7f8894105380 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:24.925+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 msgr2=0x7f888403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:24.925+0000 7f8891ffb700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.341 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.926+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f8888017d90 con 0x7f8894105380 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.927+0000 7f8891ffb700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 0x7f888403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.927+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f888403b320 con 0x7f888403bfe0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.927+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 0x7f888403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.928+0000 7f8893fff700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 0x7f888403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8880003a10 tx=0x7f88800092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.929+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f888403b320 con 0x7f888403bfe0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.936+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f889419c610 con 0x7f888403bfe0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.936+0000 7f8891ffb700 1 -- 172.21.15.26:0/2642697286 <== mgr.14116 v2:172.21.15.26:6800/2707453820 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f889419c610 con 0x7f888403bfe0 2023-07-13T22:05:26.342 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 msgr2=0x7f888403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 0x7f888403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8880003a10 tx=0x7f88800092b0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 msgr2=0x7f889419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8888006af0 tx=0x7f8888011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 shutdown_connections 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f888403bfe0 0x7f888403e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8894105380 0x7f889419b2d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.343 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 --2- 172.21.15.26:0/2642697286 >> [v2:172.21.15.26:6800/3558776409,v1:172.21.15.26:6801/3558776409] conn(0x7f8884038760 0x7f888403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:26.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 >> 172.21.15.26:0/2642697286 conn(0x7f88941009f0 msgr2=0x7f8894102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:26.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 shutdown_connections 2023-07-13T22:05:26.344 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:25.937+0000 7f889aaf4700 1 -- 172.21.15.26:0/2642697286 wait complete. 2023-07-13T22:05:26.344 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 5 is available 2023-07-13T22:05:26.344 INFO:teuthology.orchestra.run.smithi026.stdout:Setting orchestrator backend to cephadm... 2023-07-13T22:05:27.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:25] ENGINE Bus STARTING 2023-07-13T22:05:27.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:25] ENGINE Serving on http://172.21.15.26:8765 2023-07-13T22:05:27.317 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:25] ENGINE Serving on https://172.21.15.26:7150 2023-07-13T22:05:27.318 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:25] ENGINE Bus STARTED 2023-07-13T22:05:27.318 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-13T22:05:27.318 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:26 smithi026 ceph-mon[95647]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-13T22:05:27.613 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.898+0000 7fcac5e16700 1 Processor -- start 2023-07-13T22:05:27.617 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.898+0000 7fcac5e16700 1 -- start start 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.899+0000 7fcac5e16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.899+0000 7fcac5e16700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0079710 con 0x7fcac007acd0 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.899+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.899+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50124/0 (socket says 172.21.15.26:50124) 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.899+0000 7fcabf7fe700 1 -- 172.21.15.26:0/3979966087 learned_addr learned my addr 172.21.15.26:0/3979966087 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.900+0000 7fcabf7fe700 1 -- 172.21.15.26:0/3979966087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0079850 con 0x7fcac007acd0 2023-07-13T22:05:27.618 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.900+0000 7fcabf7fe700 1 --2- 172.21.15.26:0/3979966087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098a0 tx=0x7fcab0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d7785a72f947d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.900+0000 7fcabe7fc700 1 -- 172.21.15.26:0/3979966087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab0004160 con 0x7fcac007acd0 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.900+0000 7fcabe7fc700 1 -- 172.21.15.26:0/3979966087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcab00042c0 con 0x7fcac007acd0 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.900+0000 7fcabe7fc700 1 -- 172.21.15.26:0/3979966087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab0011a00 con 0x7fcac007acd0 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 -- 172.21.15.26:0/3979966087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 msgr2=0x7fcac00791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 --2- 172.21.15.26:0/3979966087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098a0 tx=0x7fcab0009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 -- 172.21.15.26:0/3979966087 shutdown_connections 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 --2- 172.21.15.26:0/3979966087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac00791d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:27.619 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 -- 172.21.15.26:0/3979966087 >> 172.21.15.26:0/3979966087 conn(0x7fcac01009f0 msgr2=0x7fcac0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:27.620 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 -- 172.21.15.26:0/3979966087 shutdown_connections 2023-07-13T22:05:27.620 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.901+0000 7fcac5e16700 1 -- 172.21.15.26:0/3979966087 wait complete. 2023-07-13T22:05:27.620 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.902+0000 7fcac5e16700 1 Processor -- start 2023-07-13T22:05:27.620 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcac5e16700 1 -- start start 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcac5e16700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcac5e16700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac019b8b0 con 0x7fcac007acd0 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50126/0 (socket says 172.21.15.26:50126) 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.903+0000 7fcabf7fe700 1 -- 172.21.15.26:0/654277136 learned_addr learned my addr 172.21.15.26:0/654277136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcabf7fe700 1 -- 172.21.15.26:0/654277136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009550 con 0x7fcac007acd0 2023-07-13T22:05:27.621 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcabf7fe700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006b20 tx=0x7fcab0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab0003b70 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcab0003cd0 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac019bab0 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcab00247c0 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.904+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac019bed0 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.905+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fcab001a430 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.906+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0061b40 con 0x7fcac007acd0 2023-07-13T22:05:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.906+0000 7fcabcff9700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 0x7fcaa803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.906+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcab004c770 con 0x7fcac007acd0 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.906+0000 7fcabeffd700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 0x7fcaa803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.907+0000 7fcabeffd700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 0x7fcaa803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcab4006fd0 tx=0x7fcab4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.911+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcab001a6e0 con 0x7fcac007acd0 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:26.980+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcab001abf0 con 0x7fcac007acd0 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.138+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fcac0102d10 con 0x7fcaa8038680 2023-07-13T22:05:27.623 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.145+0000 7fcabcff9700 1 -- 172.21.15.26:0/654277136 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcac0102d10 con 0x7fcaa8038680 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 msgr2=0x7fcaa803ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 0x7fcaa803ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcab4006fd0 tx=0x7fcab4006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 msgr2=0x7fcac019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006b20 tx=0x7fcab0011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 shutdown_connections 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcaa8038680 0x7fcaa803ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:27.624 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 --2- 172.21.15.26:0/654277136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcac007acd0 0x7fcac019b370 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.147+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 >> 172.21.15.26:0/654277136 conn(0x7fcac01009f0 msgr2=0x7fcac01020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.148+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 shutdown_connections 2023-07-13T22:05:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:27.148+0000 7fcac5e16700 1 -- 172.21.15.26:0/654277136 wait complete. 2023-07-13T22:05:28.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:27 smithi026 ceph-mon[95647]: mgrmap e8: x(active, since 2s) 2023-07-13T22:05:28.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:27 smithi026 ceph-mon[95647]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:28.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:27 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:28.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:27 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:28.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-13T22:05:28.801 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.155+0000 7f30798fc700 1 Processor -- start 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.155+0000 7f30798fc700 1 -- start start 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.156+0000 7f30798fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.156+0000 7f30798fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074073c20 con 0x7f3074105e90 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.156+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.156+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50142/0 (socket says 172.21.15.26:50142) 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.156+0000 7f3072ffd700 1 -- 172.21.15.26:0/2882215118 learned_addr learned my addr 172.21.15.26:0/2882215118 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.157+0000 7f3072ffd700 1 -- 172.21.15.26:0/2882215118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074073d60 con 0x7f3074105e90 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.157+0000 7f3072ffd700 1 --2- 172.21.15.26:0/2882215118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30640098a0 tx=0x7f3064009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d7410b6094901 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.157+0000 7f3071ffb700 1 -- 172.21.15.26:0/2882215118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064004160 con 0x7f3074105e90 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.157+0000 7f3071ffb700 1 -- 172.21.15.26:0/2882215118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30640042c0 con 0x7f3074105e90 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.157+0000 7f3071ffb700 1 -- 172.21.15.26:0/2882215118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064011a00 con 0x7f3074105e90 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 -- 172.21.15.26:0/2882215118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 msgr2=0x7f3074108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 --2- 172.21.15.26:0/2882215118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30640098a0 tx=0x7f3064009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 -- 172.21.15.26:0/2882215118 shutdown_connections 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 --2- 172.21.15.26:0/2882215118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f3074108270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 -- 172.21.15.26:0/2882215118 >> 172.21.15.26:0/2882215118 conn(0x7f30740ff8d0 msgr2=0x7f3074101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 -- 172.21.15.26:0/2882215118 shutdown_connections 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.158+0000 7f30798fc700 1 -- 172.21.15.26:0/2882215118 wait complete. 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.159+0000 7f30798fc700 1 Processor -- start 2023-07-13T22:05:28.804 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.159+0000 7f30798fc700 1 -- start start 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.159+0000 7f30798fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.159+0000 7f30798fc700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074199620 con 0x7f3074105e90 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f3072ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50158/0 (socket says 172.21.15.26:50158) 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f3072ffd700 1 -- 172.21.15.26:0/3302198240 learned_addr learned my addr 172.21.15.26:0/3302198240 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f3072ffd700 1 -- 172.21.15.26:0/3302198240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009550 con 0x7f3074105e90 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f3072ffd700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3064006af0 tx=0x7f3064011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:28.805 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3064003a40 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3064003ba0 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3074199820 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.160+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306401a700 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.161+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3074199c40 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.162+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3064024ad0 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.162+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3074061b40 con 0x7f3074105e90 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.162+0000 7f30788fa700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 0x7f305c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:28.806 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.162+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f306404c6c0 con 0x7f3074105e90 2023-07-13T22:05:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.162+0000 7f30727fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 0x7f305c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.163+0000 7f30727fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 0x7f305c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f3068006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.168+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3064018b80 con 0x7f3074105e90 2023-07-13T22:05:28.807 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.375+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3074101bc0 con 0x7f305c038760 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.376+0000 7f30788fa700 1 -- 172.21.15.26:0/3302198240 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3074101bc0 con 0x7f305c038760 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 msgr2=0x7f305c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 0x7f305c03ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3068006fd0 tx=0x7f3068006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 msgr2=0x7f30741990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3064006af0 tx=0x7f3064011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 shutdown_connections 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f305c038760 0x7f305c03ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 --2- 172.21.15.26:0/3302198240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3074105e90 0x7f30741990e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.378+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 >> 172.21.15.26:0/3302198240 conn(0x7f30740ff8d0 msgr2=0x7f3074100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.379+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 shutdown_connections 2023-07-13T22:05:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:28.379+0000 7f30798fc700 1 -- 172.21.15.26:0/3302198240 wait complete. 2023-07-13T22:05:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:Generating ssh key... 2023-07-13T22:05:30.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-mon[95647]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:30.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: Generating public/private rsa key pair. 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: Your identification has been saved in /tmp/tmpgfwr_s3n/key. 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: Your public key has been saved in /tmp/tmpgfwr_s3n/key.pub. 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: The key fingerprint is: 2023-07-13T22:05:30.177 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: SHA256:HE51I5QpNkqInQ0JQSfrT4ES83G6SDo2VlYlxddEjMQ ceph-23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: The key's randomart image is: 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: +---[RSA 3072]----+ 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: |o.*=+O+oooX=o | 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: | +.O* +.+Eo= . | 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: |..=o.. o+o | 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: |o+o. ..+ . | 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: |++o . S | 2023-07-13T22:05:30.178 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: |o..o | 2023-07-13T22:05:30.179 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: | . | 2023-07-13T22:05:30.179 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: | | 2023-07-13T22:05:30.179 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: | | 2023-07-13T22:05:30.179 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: +----[SHA256]-----+ 2023-07-13T22:05:30.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd8c13700 1 Processor -- start 2023-07-13T22:05:30.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd8c13700 1 -- start start 2023-07-13T22:05:30.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd8c13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:30.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd8c13700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4073bc0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.183 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50164/0 (socket says 172.21.15.26:50164) 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.313+0000 7fcfd259c700 1 -- 172.21.15.26:0/418366688 learned_addr learned my addr 172.21.15.26:0/418366688 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.314+0000 7fcfd259c700 1 -- 172.21.15.26:0/418366688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4073d00 con 0x7fcfd4103ce0 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.314+0000 7fcfd259c700 1 --2- 172.21.15.26:0/418366688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098a0 tx=0x7fcfc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b043283550ffc1f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.314+0000 7fcfd159a700 1 -- 172.21.15.26:0/418366688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4004160 con 0x7fcfd4103ce0 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.314+0000 7fcfd159a700 1 -- 172.21.15.26:0/418366688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc40042c0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.184 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.314+0000 7fcfd159a700 1 -- 172.21.15.26:0/418366688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4011a00 con 0x7fcfd4103ce0 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 -- 172.21.15.26:0/418366688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 msgr2=0x7fcfd41040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/418366688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcfc40098a0 tx=0x7fcfc4009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 -- 172.21.15.26:0/418366688 shutdown_connections 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/418366688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd41040f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 -- 172.21.15.26:0/418366688 >> 172.21.15.26:0/418366688 conn(0x7fcfd40ff8d0 msgr2=0x7fcfd4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 -- 172.21.15.26:0/418366688 shutdown_connections 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.315+0000 7fcfd8c13700 1 -- 172.21.15.26:0/418366688 wait complete. 2023-07-13T22:05:30.185 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.316+0000 7fcfd8c13700 1 Processor -- start 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.316+0000 7fcfd8c13700 1 -- start start 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd8c13700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd8c13700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419b840 con 0x7fcfd4103ce0 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50174/0 (socket says 172.21.15.26:50174) 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd259c700 1 -- 172.21.15.26:0/3276750211 learned_addr learned my addr 172.21.15.26:0/3276750211 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd259c700 1 -- 172.21.15.26:0/3276750211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4009550 con 0x7fcfd4103ce0 2023-07-13T22:05:30.186 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.317+0000 7fcfd259c700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006af0 tx=0x7fcfc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.318+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc4003a40 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.318+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd419ba40 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.318+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcfc4003ba0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.318+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc401a700 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.318+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419be60 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.319+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcfc4024430 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.319+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd4194ad0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.187 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.319+0000 7fcfc37fe700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 0x7fcfbc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.319+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcfc404c7a0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.320+0000 7fcfd1d9b700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 0x7fcfbc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.320+0000 7fcfd1d9b700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 0x7fcfbc03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.325+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcfc40189a0 con 0x7fcfd4103ce0 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.532+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfd4060950 con 0x7fcfbc038710 2023-07-13T22:05:30.188 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.743+0000 7fcfc37fe700 1 -- 172.21.15.26:0/3276750211 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcfd4060950 con 0x7fcfbc038710 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 msgr2=0x7fcfbc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 0x7fcfbc03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 msgr2=0x7fcfd419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4006af0 tx=0x7fcfc4011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 shutdown_connections 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.745+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcfbc038710 0x7fcfbc03abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.746+0000 7fcfd8c13700 1 --2- 172.21.15.26:0/3276750211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfd4103ce0 0x7fcfd419b300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:30.189 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.746+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 >> 172.21.15.26:0/3276750211 conn(0x7fcfd40ff8d0 msgr2=0x7fcfd418f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:30.190 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.746+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 shutdown_connections 2023-07-13T22:05:30.190 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:29.746+0000 7fcfd8c13700 1 -- 172.21.15.26:0/3276750211 wait complete. 2023-07-13T22:05:31.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:30 smithi026 ceph-mon[95647]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:31.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:30 smithi026 ceph-mon[95647]: Generating ssh key... 2023-07-13T22:05:31.363 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:05:31.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.703+0000 7fc5f8053700 1 Processor -- start 2023-07-13T22:05:31.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.703+0000 7fc5f8053700 1 -- start start 2023-07-13T22:05:31.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.703+0000 7fc5f8053700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:31.364 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.703+0000 7fc5f8053700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0079730 con 0x7fc5f007acf0 2023-07-13T22:05:31.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.704+0000 7fc5f5def700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:31.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.704+0000 7fc5f5def700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50186/0 (socket says 172.21.15.26:50186) 2023-07-13T22:05:31.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.704+0000 7fc5f5def700 1 -- 172.21.15.26:0/343243108 learned_addr learned my addr 172.21.15.26:0/343243108 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:31.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.704+0000 7fc5f5def700 1 -- 172.21.15.26:0/343243108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0079870 con 0x7fc5f007acf0 2023-07-13T22:05:31.365 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.704+0000 7fc5f5def700 1 --2- 172.21.15.26:0/343243108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0098a0 tx=0x7fc5ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8089d0d8adb77779 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:31.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f4ded700 1 -- 172.21.15.26:0/343243108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5ec004160 con 0x7fc5f007acf0 2023-07-13T22:05:31.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f4ded700 1 -- 172.21.15.26:0/343243108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5ec0042c0 con 0x7fc5f007acf0 2023-07-13T22:05:31.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f4ded700 1 -- 172.21.15.26:0/343243108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5ec011a00 con 0x7fc5f007acf0 2023-07-13T22:05:31.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 -- 172.21.15.26:0/343243108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 msgr2=0x7fc5f00791f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:31.366 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 --2- 172.21.15.26:0/343243108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0098a0 tx=0x7fc5ec009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 -- 172.21.15.26:0/343243108 shutdown_connections 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 --2- 172.21.15.26:0/343243108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f00791f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 -- 172.21.15.26:0/343243108 >> 172.21.15.26:0/343243108 conn(0x7fc5f01009f0 msgr2=0x7fc5f0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 -- 172.21.15.26:0/343243108 shutdown_connections 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.705+0000 7fc5f8053700 1 -- 172.21.15.26:0/343243108 wait complete. 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.706+0000 7fc5f8053700 1 Processor -- start 2023-07-13T22:05:31.367 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f8053700 1 -- start start 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f8053700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f8053700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019b860 con 0x7fc5f007acf0 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f5def700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f5def700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50200/0 (socket says 172.21.15.26:50200) 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.707+0000 7fc5f5def700 1 -- 172.21.15.26:0/3030350013 learned_addr learned my addr 172.21.15.26:0/3030350013 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5f5def700 1 -- 172.21.15.26:0/3030350013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009550 con 0x7fc5f007acf0 2023-07-13T22:05:31.368 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5f5def700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006af0 tx=0x7fc5ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5ec003a40 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5ec003ba0 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5ec01a700 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f019ba60 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.708+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f019be80 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.709+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc5ec024430 con 0x7fc5f007acf0 2023-07-13T22:05:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.709+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0061b40 con 0x7fc5f007acf0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.710+0000 7fc5e6ffd700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 0x7fc5dc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.710+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc5ec04c750 con 0x7fc5f007acf0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.710+0000 7fc5f55ee700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 0x7fc5dc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.711+0000 7fc5f55ee700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 0x7fc5dc03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006fd0 tx=0x7fc5e0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.715+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc5ec0189a0 con 0x7fc5f007acf0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.922+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f0102ce0 con 0x7fc5dc0387b0 2023-07-13T22:05:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.923+0000 7fc5e6ffd700 1 -- 172.21.15.26:0/3030350013 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fc5f0102ce0 con 0x7fc5dc0387b0 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 msgr2=0x7fc5dc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 0x7fc5dc03ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0006fd0 tx=0x7fc5e0006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 msgr2=0x7fc5f019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006af0 tx=0x7fc5ec011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 shutdown_connections 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc5dc0387b0 0x7fc5dc03ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 --2- 172.21.15.26:0/3030350013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f007acf0 0x7fc5f019b320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:31.371 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.925+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 >> 172.21.15.26:0/3030350013 conn(0x7fc5f01009f0 msgr2=0x7fc5f01020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:31.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.926+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 shutdown_connections 2023-07-13T22:05:31.372 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:30.926+0000 7fc5f8053700 1 -- 172.21.15.26:0/3030350013 wait complete. 2023-07-13T22:05:31.372 INFO:teuthology.orchestra.run.smithi026.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-13T22:05:31.372 INFO:teuthology.orchestra.run.smithi026.stdout:Adding key to root@localhost authorized_keys... 2023-07-13T22:05:31.372 INFO:teuthology.orchestra.run.smithi026.stdout:Adding host smithi026... 2023-07-13T22:05:31.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:31 smithi026 ceph-mon[95647]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:33.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:32 smithi026 ceph-mon[95647]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:34.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:33 smithi026 ceph-mon[95647]: Deploying cephadm binary to smithi026 2023-07-13T22:05:35.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Added host 'smithi026' with addr '172.21.15.26' 2023-07-13T22:05:35.850 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.923+0000 7fcdb910c700 1 Processor -- start 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb910c700 1 -- start start 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb910c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb910c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40796f0 con 0x7fcdb407acb0 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50210/0 (socket says 172.21.15.26:50210) 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.924+0000 7fcdb2d9d700 1 -- 172.21.15.26:0/3610010593 learned_addr learned my addr 172.21.15.26:0/3610010593 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.925+0000 7fcdb2d9d700 1 -- 172.21.15.26:0/3610010593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4079830 con 0x7fcdb407acb0 2023-07-13T22:05:35.851 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.925+0000 7fcdb2d9d700 1 --2- 172.21.15.26:0/3610010593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098a0 tx=0x7fcda4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb4c379824a18e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:35.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.925+0000 7fcdb1d9b700 1 -- 172.21.15.26:0/3610010593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda4004160 con 0x7fcdb407acb0 2023-07-13T22:05:35.852 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.925+0000 7fcdb1d9b700 1 -- 172.21.15.26:0/3610010593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcda40042c0 con 0x7fcdb407acb0 2023-07-13T22:05:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.925+0000 7fcdb1d9b700 1 -- 172.21.15.26:0/3610010593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda4011a00 con 0x7fcdb407acb0 2023-07-13T22:05:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 -- 172.21.15.26:0/3610010593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 msgr2=0x7fcdb40791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3610010593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcda40098a0 tx=0x7fcda4009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 -- 172.21.15.26:0/3610010593 shutdown_connections 2023-07-13T22:05:35.855 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3610010593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb40791b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 -- 172.21.15.26:0/3610010593 >> 172.21.15.26:0/3610010593 conn(0x7fcdb41009f0 msgr2=0x7fcdb4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 -- 172.21.15.26:0/3610010593 shutdown_connections 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.926+0000 7fcdb910c700 1 -- 172.21.15.26:0/3610010593 wait complete. 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.927+0000 7fcdb910c700 1 Processor -- start 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.927+0000 7fcdb910c700 1 -- start start 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.927+0000 7fcdb910c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.927+0000 7fcdb910c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb419b880 con 0x7fcdb407acb0 2023-07-13T22:05:35.856 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcdb2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcdb2d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50226/0 (socket says 172.21.15.26:50226) 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcdb2d9d700 1 -- 172.21.15.26:0/3592253638 learned_addr learned my addr 172.21.15.26:0/3592253638 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcdb2d9d700 1 -- 172.21.15.26:0/3592253638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda4009550 con 0x7fcdb407acb0 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcdb2d9d700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006af0 tx=0x7fcda4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.928+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda4003a40 con 0x7fcdb407acb0 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.929+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb419ba80 con 0x7fcdb407acb0 2023-07-13T22:05:35.857 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.929+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcda4003ba0 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.929+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcda401a700 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.929+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb419bea0 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.930+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb4061b40 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.930+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcda4024430 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.930+0000 7fcd9bfff700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 0x7fcd9c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.930+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcda404c7a0 con 0x7fcdb407acb0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.930+0000 7fcdb259c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 0x7fcd9c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:35.858 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.931+0000 7fcdb259c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 0x7fcd9c03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006fd0 tx=0x7fcda8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:31.935+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcda40189a0 con 0x7fcdb407acb0 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:32.146+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi026", "addr": "172.21.15.26", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb4102d10 con 0x7fcd9c038710 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.370+0000 7fcd9bfff700 1 -- 172.21.15.26:0/3592253638 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fcdb4102d10 con 0x7fcd9c038710 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.373+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 msgr2=0x7fcd9c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.373+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 0x7fcd9c03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006fd0 tx=0x7fcda8006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 msgr2=0x7fcdb419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:35.859 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcda4006af0 tx=0x7fcda4011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 shutdown_connections 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd9c038710 0x7fcd9c03abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 --2- 172.21.15.26:0/3592253638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcdb407acb0 0x7fcdb419b340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 >> 172.21.15.26:0/3592253638 conn(0x7fcdb41009f0 msgr2=0x7fcdb41020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 shutdown_connections 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:35.374+0000 7fcdb910c700 1 -- 172.21.15.26:0/3592253638 wait complete. 2023-07-13T22:05:35.860 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mon service... 2023-07-13T22:05:36.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:36 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:36.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:36 smithi026 ceph-mon[95647]: Added host smithi026 2023-07-13T22:05:36.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:36 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:37.239 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.409+0000 7fcc21a83700 1 Processor -- start 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.409+0000 7fcc21a83700 1 -- start start 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.409+0000 7fcc21a83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.409+0000 7fcc21a83700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0796f0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.410+0000 7fcc20a81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.410+0000 7fcc20a81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33604/0 (socket says 172.21.15.26:33604) 2023-07-13T22:05:37.240 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.410+0000 7fcc20a81700 1 -- 172.21.15.26:0/2710832486 learned_addr learned my addr 172.21.15.26:0/2710832486 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.410+0000 7fcc20a81700 1 -- 172.21.15.26:0/2710832486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c079830 con 0x7fcc1c07acb0 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.410+0000 7fcc20a81700 1 --2- 172.21.15.26:0/2710832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e443999a932d537 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc1b7fe700 1 -- 172.21.15.26:0/2710832486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc10004160 con 0x7fcc1c07acb0 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc1b7fe700 1 -- 172.21.15.26:0/2710832486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc100042c0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc1b7fe700 1 -- 172.21.15.26:0/2710832486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc10011a00 con 0x7fcc1c07acb0 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 -- 172.21.15.26:0/2710832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 msgr2=0x7fcc1c0791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:37.241 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 --2- 172.21.15.26:0/2710832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 -- 172.21.15.26:0/2710832486 shutdown_connections 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 --2- 172.21.15.26:0/2710832486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c0791b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 -- 172.21.15.26:0/2710832486 >> 172.21.15.26:0/2710832486 conn(0x7fcc1c100810 msgr2=0x7fcc1c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 -- 172.21.15.26:0/2710832486 shutdown_connections 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.411+0000 7fcc21a83700 1 -- 172.21.15.26:0/2710832486 wait complete. 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.413+0000 7fcc21a83700 1 Processor -- start 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.413+0000 7fcc21a83700 1 -- start start 2023-07-13T22:05:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc21a83700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc21a83700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c19b6d0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc20a81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc20a81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33614/0 (socket says 172.21.15.26:33614) 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc20a81700 1 -- 172.21.15.26:0/1528054121 learned_addr learned my addr 172.21.15.26:0/1528054121 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc20a81700 1 -- 172.21.15.26:0/1528054121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009550 con 0x7fcc1c07acb0 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.414+0000 7fcc20a81700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fb0 tx=0x7fcc10011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.415+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc10003a40 con 0x7fcc1c07acb0 2023-07-13T22:05:37.243 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.415+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c19b8d0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.415+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc10003ba0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.415+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc1001a750 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.415+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c19bcf0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.416+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcc10024b60 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.416+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c195170 con 0x7fcc1c07acb0 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.416+0000 7fcc19ffb700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 0x7fcc0c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:37.244 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.416+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc1004c6c0 con 0x7fcc1c07acb0 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.417+0000 7fcc1bfff700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 0x7fcc0c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.417+0000 7fcc1bfff700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 0x7fcc0c03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc08006fd0 tx=0x7fcc08006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.422+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcc10024430 con 0x7fcc1c07acb0 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.641+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcc1c060950 con 0x7fcc0c038710 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.646+0000 7fcc19ffb700 1 -- 172.21.15.26:0/1528054121 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcc1c060950 con 0x7fcc0c038710 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.648+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 msgr2=0x7fcc0c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.648+0000 7fcc21a83700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 0x7fcc0c03abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc08006fd0 tx=0x7fcc08006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:37.245 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.648+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 msgr2=0x7fcc1c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.648+0000 7fcc21a83700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fb0 tx=0x7fcc10011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 shutdown_connections 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcc0c038710 0x7fcc0c03abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 --2- 172.21.15.26:0/1528054121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc1c07acb0 0x7fcc1c19b190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 >> 172.21.15.26:0/1528054121 conn(0x7fcc1c100810 msgr2=0x7fcc1c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:37.246 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 shutdown_connections 2023-07-13T22:05:37.247 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:36.649+0000 7fcc21a83700 1 -- 172.21.15.26:0/1528054121 wait complete. 2023-07-13T22:05:37.247 INFO:teuthology.orchestra.run.smithi026.stdout:Deploying unmanaged mgr service... 2023-07-13T22:05:37.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:37 smithi026 ceph-mon[95647]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:37.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:37 smithi026 ceph-mon[95647]: Saving service mon spec with placement count:5 2023-07-13T22:05:37.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:37 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:38.517 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.807+0000 7fcd66698700 1 Processor -- start 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.807+0000 7fcd66698700 1 -- start start 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.807+0000 7fcd66698700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.807+0000 7fcd66698700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60105cd0 con 0x7fcd60105380 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.808+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.808+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33624/0 (socket says 172.21.15.26:33624) 2023-07-13T22:05:38.518 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.808+0000 7fcd5ffff700 1 -- 172.21.15.26:0/933974314 learned_addr learned my addr 172.21.15.26:0/933974314 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.808+0000 7fcd5ffff700 1 -- 172.21.15.26:0/933974314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd60105e10 con 0x7fcd60105380 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.808+0000 7fcd5ffff700 1 --2- 172.21.15.26:0/933974314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098a0 tx=0x7fcd50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b467dc394bb9db5d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd5effd700 1 -- 172.21.15.26:0/933974314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd50004160 con 0x7fcd60105380 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd5effd700 1 -- 172.21.15.26:0/933974314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd500042c0 con 0x7fcd60105380 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd5effd700 1 -- 172.21.15.26:0/933974314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd50011a00 con 0x7fcd60105380 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 -- 172.21.15.26:0/933974314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 msgr2=0x7fcd60105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:38.519 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 --2- 172.21.15.26:0/933974314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd500098a0 tx=0x7fcd50009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 -- 172.21.15.26:0/933974314 shutdown_connections 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 --2- 172.21.15.26:0/933974314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd60105790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 -- 172.21.15.26:0/933974314 >> 172.21.15.26:0/933974314 conn(0x7fcd601009f0 msgr2=0x7fcd60102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 -- 172.21.15.26:0/933974314 shutdown_connections 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.809+0000 7fcd66698700 1 -- 172.21.15.26:0/933974314 wait complete. 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.810+0000 7fcd66698700 1 Processor -- start 2023-07-13T22:05:38.520 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd66698700 1 -- start start 2023-07-13T22:05:38.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd66698700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:38.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd66698700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6019b810 con 0x7fcd60105380 2023-07-13T22:05:38.521 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:38.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33626/0 (socket says 172.21.15.26:33626) 2023-07-13T22:05:38.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd5ffff700 1 -- 172.21.15.26:0/2448553542 learned_addr learned my addr 172.21.15.26:0/2448553542 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:38.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.811+0000 7fcd5ffff700 1 -- 172.21.15.26:0/2448553542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009550 con 0x7fcd60105380 2023-07-13T22:05:38.522 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd5ffff700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009fb0 tx=0x7fcd50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:38.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd50003930 con 0x7fcd60105380 2023-07-13T22:05:38.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcd50003a90 con 0x7fcd60105380 2023-07-13T22:05:38.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6019ba10 con 0x7fcd60105380 2023-07-13T22:05:38.523 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd50019610 con 0x7fcd60105380 2023-07-13T22:05:38.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.812+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6019be30 con 0x7fcd60105380 2023-07-13T22:05:38.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.813+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd60109810 con 0x7fcd60105380 2023-07-13T22:05:38.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.813+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fcd5001a9d0 con 0x7fcd60105380 2023-07-13T22:05:38.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.814+0000 7fcd5d7fa700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 0x7fcd4803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:38.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.814+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcd5004c8a0 con 0x7fcd60105380 2023-07-13T22:05:38.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.814+0000 7fcd5f7fe700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 0x7fcd4803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:38.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.814+0000 7fcd5f7fe700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 0x7fcd4803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:38.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:37.818+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcd50018760 con 0x7fcd60105380 2023-07-13T22:05:38.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.024+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mon.0 v2:172.21.15.26:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50018760 con 0x7fcd60105380 2023-07-13T22:05:38.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.046+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcd60102cb0 con 0x7fcd480386c0 2023-07-13T22:05:38.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.050+0000 7fcd5d7fa700 1 -- 172.21.15.26:0/2448553542 <== mgr.14116 v2:172.21.15.26:6800/2707453820 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcd60102cb0 con 0x7fcd480386c0 2023-07-13T22:05:38.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 msgr2=0x7fcd4803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:38.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 0x7fcd4803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcd54006fd0 tx=0x7fcd54006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 msgr2=0x7fcd6019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009fb0 tx=0x7fcd50011ad0 comp rx=0 tx=0).stop 2023-07-13T22:05:38.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 shutdown_connections 2023-07-13T22:05:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fcd480386c0 0x7fcd4803ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 --2- 172.21.15.26:0/2448553542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60105380 0x7fcd6019b2d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 >> 172.21.15.26:0/2448553542 conn(0x7fcd601009f0 msgr2=0x7fcd60102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 shutdown_connections 2023-07-13T22:05:38.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:38.053+0000 7fcd66698700 1 -- 172.21.15.26:0/2448553542 wait complete. 2023-07-13T22:05:39.336 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:39 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:39.337 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:39 smithi026 ceph-mon[95647]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:39.337 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:39 smithi026 ceph-mon[95647]: Saving service mgr spec with placement count:2 2023-07-13T22:05:39.337 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:39 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:39.337 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:39 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:39.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.072+0000 7f78b4cc5700 1 Processor -- start 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.072+0000 7f78b4cc5700 1 -- start start 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.072+0000 7f78b4cc5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.072+0000 7f78b4cc5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0073bc0 con 0x7f78b0105ef0 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.073+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.073+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33630/0 (socket says 172.21.15.26:33630) 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.073+0000 7f78ae59c700 1 -- 172.21.15.26:0/3592836457 learned_addr learned my addr 172.21.15.26:0/3592836457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:39.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.073+0000 7f78ae59c700 1 -- 172.21.15.26:0/3592836457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b0073d00 con 0x7f78b0105ef0 2023-07-13T22:05:39.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.074+0000 7f78ae59c700 1 --2- 172.21.15.26:0/3592836457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78a00098a0 tx=0x7f78a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248697d8ee9c987e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:39.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.074+0000 7f78ad59a700 1 -- 172.21.15.26:0/3592836457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a0004160 con 0x7f78b0105ef0 2023-07-13T22:05:39.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.074+0000 7f78ad59a700 1 -- 172.21.15.26:0/3592836457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a00042c0 con 0x7f78b0105ef0 2023-07-13T22:05:39.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.074+0000 7f78ad59a700 1 -- 172.21.15.26:0/3592836457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a00119b0 con 0x7f78b0105ef0 2023-07-13T22:05:39.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 -- 172.21.15.26:0/3592836457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 msgr2=0x7f78b01082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:39.703 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/3592836457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f78a00098a0 tx=0x7f78a0009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 -- 172.21.15.26:0/3592836457 shutdown_connections 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/3592836457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b01082d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 -- 172.21.15.26:0/3592836457 >> 172.21.15.26:0/3592836457 conn(0x7f78b00ff8d0 msgr2=0x7f78b0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 -- 172.21.15.26:0/3592836457 shutdown_connections 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.075+0000 7f78b4cc5700 1 -- 172.21.15.26:0/3592836457 wait complete. 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.076+0000 7f78b4cc5700 1 Processor -- start 2023-07-13T22:05:39.704 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78b4cc5700 1 -- start start 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78b4cc5700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78b4cc5700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0199660 con 0x7f78b0105ef0 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78ae59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33640/0 (socket says 172.21.15.26:33640) 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.077+0000 7f78ae59c700 1 -- 172.21.15.26:0/229935666 learned_addr learned my addr 172.21.15.26:0/229935666 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f78ae59c700 1 -- 172.21.15.26:0/229935666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009550 con 0x7f78b0105ef0 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f78ae59c700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f78a0000c00 tx=0x7f78a0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:39.705 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a00039f0 con 0x7f78b0105ef0 2023-07-13T22:05:39.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a0003b50 con 0x7f78b0105ef0 2023-07-13T22:05:39.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a001a780 con 0x7f78b0105ef0 2023-07-13T22:05:39.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b0199860 con 0x7f78b0105ef0 2023-07-13T22:05:39.706 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.078+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b0199c80 con 0x7f78b0105ef0 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.080+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b0061b40 con 0x7f78b0105ef0 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.080+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f78a0024440 con 0x7f78b0105ef0 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.080+0000 7f789f7fe700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 0x7f789803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.080+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f78a004c820 con 0x7f78b0105ef0 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.081+0000 7f78add9b700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 0x7f789803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:39.707 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.081+0000 7f78add9b700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 0x7f789803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a4006fd0 tx=0x7f78a4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.085+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f78a00246f0 con 0x7f78b0105ef0 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.310+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f78b019c6c0 con 0x7f78b0105ef0 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.314+0000 7f789f7fe700 1 -- 172.21.15.26:0/229935666 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f78a0018350 con 0x7f78b0105ef0 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.316+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 msgr2=0x7f789803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.316+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 0x7f789803ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f78a4006fd0 tx=0x7f78a4006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:39.708 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.316+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 msgr2=0x7f78b0199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.316+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f78a0000c00 tx=0x7f78a0011a30 comp rx=0 tx=0).stop 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 shutdown_connections 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f7898038760 0x7f789803ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 --2- 172.21.15.26:0/229935666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78b0105ef0 0x7f78b0199120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 >> 172.21.15.26:0/229935666 conn(0x7f78b00ff8d0 msgr2=0x7f78b0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 shutdown_connections 2023-07-13T22:05:39.709 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:39.317+0000 7f78b4cc5700 1 -- 172.21.15.26:0/229935666 wait complete. 2023-07-13T22:05:40.640 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:40 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/229935666' entity='client.admin' 2023-07-13T22:05:41.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.274+0000 7f68447d7700 1 Processor -- start 2023-07-13T22:05:41.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.274+0000 7f68447d7700 1 -- start start 2023-07-13T22:05:41.082 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.274+0000 7f68447d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.275+0000 7f68447d7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c079710 con 0x7f683c07acd0 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.275+0000 7f6842573700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.275+0000 7f6842573700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33650/0 (socket says 172.21.15.26:33650) 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.275+0000 7f6842573700 1 -- 172.21.15.26:0/204968598 learned_addr learned my addr 172.21.15.26:0/204968598 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.275+0000 7f6842573700 1 -- 172.21.15.26:0/204968598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c079850 con 0x7f683c07acd0 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.276+0000 7f6842573700 1 --2- 172.21.15.26:0/204968598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68380098a0 tx=0x7f6838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da460b4e700be093 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:41.083 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.276+0000 7f6841571700 1 -- 172.21.15.26:0/204968598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6838004160 con 0x7f683c07acd0 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.276+0000 7f6841571700 1 -- 172.21.15.26:0/204968598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68380042c0 con 0x7f683c07acd0 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.276+0000 7f6841571700 1 -- 172.21.15.26:0/204968598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68380119b0 con 0x7f683c07acd0 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 -- 172.21.15.26:0/204968598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 msgr2=0x7f683c0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 --2- 172.21.15.26:0/204968598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68380098a0 tx=0x7f6838009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 -- 172.21.15.26:0/204968598 shutdown_connections 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 --2- 172.21.15.26:0/204968598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c0791d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 -- 172.21.15.26:0/204968598 >> 172.21.15.26:0/204968598 conn(0x7f683c1009f0 msgr2=0x7f683c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 -- 172.21.15.26:0/204968598 shutdown_connections 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.277+0000 7f68447d7700 1 -- 172.21.15.26:0/204968598 wait complete. 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.278+0000 7f68447d7700 1 Processor -- start 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.278+0000 7f68447d7700 1 -- start start 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.278+0000 7f68447d7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.278+0000 7f68447d7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c19b8c0 con 0x7f683c07acd0 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.279+0000 7f6842573700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.279+0000 7f6842573700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33652/0 (socket says 172.21.15.26:33652) 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.279+0000 7f6842573700 1 -- 172.21.15.26:0/1522104328 learned_addr learned my addr 172.21.15.26:0/1522104328 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.279+0000 7f6842573700 1 -- 172.21.15.26:0/1522104328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6838009550 con 0x7f683c07acd0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.279+0000 7f6842573700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6838003e60 tx=0x7f6838011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.280+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6838003b20 con 0x7f683c07acd0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.280+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c19bac0 con 0x7f683c07acd0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.280+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6838003c80 con 0x7f683c07acd0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.280+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6838024760 con 0x7f683c07acd0 2023-07-13T22:05:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.280+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c19bee0 con 0x7f683c07acd0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.281+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f683802d3f0 con 0x7f683c07acd0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.281+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c061b40 con 0x7f683c07acd0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.282+0000 7f68337fe700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 0x7f682803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.282+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f683804ca10 con 0x7f683c07acd0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.282+0000 7f6841d72700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 0x7f682803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.283+0000 7f6841d72700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 0x7f682803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f682c006fd0 tx=0x7f682c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.287+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6838010970 con 0x7f683c07acd0 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.498+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f683c19e920 con 0x7f683c07acd0 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.502+0000 7f68337fe700 1 -- 172.21.15.26:0/1522104328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6838020220 con 0x7f683c07acd0 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.505+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 msgr2=0x7f682803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.505+0000 7f68447d7700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 0x7f682803ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f682c006fd0 tx=0x7f682c006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.505+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 msgr2=0x7f683c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.505+0000 7f68447d7700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6838003e60 tx=0x7f6838011a30 comp rx=0 tx=0).stop 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 shutdown_connections 2023-07-13T22:05:41.088 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f6828038760 0x7f682803ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:41.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 --2- 172.21.15.26:0/1522104328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f683c07acd0 0x7f683c19b380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:41.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 >> 172.21.15.26:0/1522104328 conn(0x7f683c1009f0 msgr2=0x7f683c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:41.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 shutdown_connections 2023-07-13T22:05:41.089 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:40.506+0000 7f68447d7700 1 -- 172.21.15.26:0/1522104328 wait complete. 2023-07-13T22:05:41.089 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling the dashboard module... 2023-07-13T22:05:41.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:41 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1522104328' entity='client.admin' 2023-07-13T22:05:41.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:41 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:42.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3719399476' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-13T22:05:42.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-mon[95647]: from='mgr.14116 172.21.15.26:0/3115418536' entity='mgr.x' 2023-07-13T22:05:42.814 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: ignoring --setuser ceph since I am not root 2023-07-13T22:05:42.814 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: ignoring --setgroup ceph since I am not root 2023-07-13T22:05:42.814 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:42.816+0000 7fb20a2a7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:05:42.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.655+0000 7f8d077c2700 1 Processor -- start 2023-07-13T22:05:42.964 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.655+0000 7f8d077c2700 1 -- start start 2023-07-13T22:05:42.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.655+0000 7f8d077c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:42.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.655+0000 7f8d077c2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d00079710 con 0x7f8d0007acd0 2023-07-13T22:05:42.965 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.656+0000 7f8d0555e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:42.969 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.656+0000 7f8d0555e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33664/0 (socket says 172.21.15.26:33664) 2023-07-13T22:05:42.969 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.656+0000 7f8d0555e700 1 -- 172.21.15.26:0/2774019807 learned_addr learned my addr 172.21.15.26:0/2774019807 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:42.969 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.656+0000 7f8d0555e700 1 -- 172.21.15.26:0/2774019807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d00079850 con 0x7f8d0007acd0 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8d0555e700 1 --2- 172.21.15.26:0/2774019807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0098a0 tx=0x7f8cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4b75f6572ff8a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8cf7fff700 1 -- 172.21.15.26:0/2774019807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cfc004160 con 0x7f8d0007acd0 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8cf7fff700 1 -- 172.21.15.26:0/2774019807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cfc0042c0 con 0x7f8d0007acd0 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8cf7fff700 1 -- 172.21.15.26:0/2774019807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cfc0119b0 con 0x7f8d0007acd0 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8d077c2700 1 -- 172.21.15.26:0/2774019807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 msgr2=0x7f8d000791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.657+0000 7f8d077c2700 1 --2- 172.21.15.26:0/2774019807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0098a0 tx=0x7f8cfc009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:42.970 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.658+0000 7f8d077c2700 1 -- 172.21.15.26:0/2774019807 shutdown_connections 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.658+0000 7f8d077c2700 1 --2- 172.21.15.26:0/2774019807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d000791d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.658+0000 7f8d077c2700 1 -- 172.21.15.26:0/2774019807 >> 172.21.15.26:0/2774019807 conn(0x7f8d001009f0 msgr2=0x7f8d00102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.658+0000 7f8d077c2700 1 -- 172.21.15.26:0/2774019807 shutdown_connections 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.658+0000 7f8d077c2700 1 -- 172.21.15.26:0/2774019807 wait complete. 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.659+0000 7f8d077c2700 1 Processor -- start 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.659+0000 7f8d077c2700 1 -- start start 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.659+0000 7f8d077c2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:42.971 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.659+0000 7f8d077c2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0019b8c0 con 0x7f8d0007acd0 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.660+0000 7f8d0555e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.660+0000 7f8d0555e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33674/0 (socket says 172.21.15.26:33674) 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.660+0000 7f8d0555e700 1 -- 172.21.15.26:0/3719399476 learned_addr learned my addr 172.21.15.26:0/3719399476 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.660+0000 7f8d0555e700 1 -- 172.21.15.26:0/3719399476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc009550 con 0x7f8d0007acd0 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.660+0000 7f8d0555e700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc003e60 tx=0x7f8cfc011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:42.972 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.661+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cfc003af0 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.661+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cfc003c50 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.661+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cfc024800 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.661+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0019bac0 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.661+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0019bee0 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.662+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8cfc01ab80 con 0x7f8d0007acd0 2023-07-13T22:05:42.973 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.662+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0010b710 con 0x7f8d0007acd0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.663+0000 7f8cf67fc700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 0x7f8cec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.663+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8cfc04c9a0 con 0x7f8d0007acd0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.663+0000 7f8d04d5d700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 0x7f8cec03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.664+0000 7f8d04d5d700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 0x7f8cec03ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.668+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8cfc01a950 con 0x7f8d0007acd0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:41.939+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8d0010b900 con 0x7f8d0007acd0 2023-07-13T22:05:42.974 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.506+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8cfc01a430 con 0x7f8d0007acd0 2023-07-13T22:05:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.508+0000 7f8cf67fc700 1 -- 172.21.15.26:0/3719399476 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8cfc02aa90 con 0x7f8d0007acd0 2023-07-13T22:05:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 msgr2=0x7f8cec03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 0x7f8cec03ab70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0006fd0 tx=0x7f8cf0006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:42.975 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 msgr2=0x7f8d0019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc003e60 tx=0x7f8cfc011980 comp rx=0 tx=0).stop 2023-07-13T22:05:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 shutdown_connections 2023-07-13T22:05:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7f8cec0386c0 0x7f8cec03ab70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 --2- 172.21.15.26:0/3719399476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d0007acd0 0x7f8d0019b380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:42.976 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 >> 172.21.15.26:0/3719399476 conn(0x7f8d001009f0 msgr2=0x7f8d001020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:42.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 shutdown_connections 2023-07-13T22:05:42.977 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:42.511+0000 7f8d077c2700 1 -- 172.21.15.26:0/3719399476 wait complete. 2023-07-13T22:05:43.064 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:42.923+0000 7fb20a2a7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:43.344 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:43.067+0000 7fb20a2a7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:05:43.344 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:43.190+0000 7fb20a2a7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:05:43.626 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:43.438+0000 7fb20a2a7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:05:43.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3719399476' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-13T22:05:43.626 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-mon[95647]: mgrmap e9: x(active, since 17s) 2023-07-13T22:05:43.931 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:43.877+0000 7fb20a2a7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:05:44.185 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:44.006+0000 7fb20a2a7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:05:44.458 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:44.187+0000 7fb20a2a7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:05:44.459 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:44.318+0000 7fb20a2a7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:05:44.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:44.524 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.628+0000 7ffbd4275700 1 Processor -- start 2023-07-13T22:05:44.525 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.628+0000 7ffbd4275700 1 -- start start 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.628+0000 7ffbd4275700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.628+0000 7ffbd4275700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0796d0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.629+0000 7ffbd2011700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.630+0000 7ffbd2011700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53672/0 (socket says 172.21.15.26:53672) 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.630+0000 7ffbd2011700 1 -- 172.21.15.26:0/581926981 learned_addr learned my addr 172.21.15.26:0/581926981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.630+0000 7ffbd2011700 1 -- 172.21.15.26:0/581926981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc079810 con 0x7ffbcc07ac90 2023-07-13T22:05:44.526 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.630+0000 7ffbd2011700 1 --2- 172.21.15.26:0/581926981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098a0 tx=0x7ffbc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d26896da588d38 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.630+0000 7ffbd100f700 1 -- 172.21.15.26:0/581926981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc8004160 con 0x7ffbcc07ac90 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.631+0000 7ffbd100f700 1 -- 172.21.15.26:0/581926981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbc80042c0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.631+0000 7ffbd100f700 1 -- 172.21.15.26:0/581926981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc80119b0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 -- 172.21.15.26:0/581926981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 msgr2=0x7ffbcc079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 --2- 172.21.15.26:0/581926981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098a0 tx=0x7ffbc8009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 -- 172.21.15.26:0/581926981 shutdown_connections 2023-07-13T22:05:44.527 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 --2- 172.21.15.26:0/581926981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc079190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 -- 172.21.15.26:0/581926981 >> 172.21.15.26:0/581926981 conn(0x7ffbcc1009f0 msgr2=0x7ffbcc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 -- 172.21.15.26:0/581926981 shutdown_connections 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.632+0000 7ffbd4275700 1 -- 172.21.15.26:0/581926981 wait complete. 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.633+0000 7ffbd4275700 1 Processor -- start 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.634+0000 7ffbd4275700 1 -- start start 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.634+0000 7ffbd4275700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.634+0000 7ffbd4275700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc19b890 con 0x7ffbcc07ac90 2023-07-13T22:05:44.528 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.634+0000 7ffbd2011700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.635+0000 7ffbd2011700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53686/0 (socket says 172.21.15.26:53686) 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.635+0000 7ffbd2011700 1 -- 172.21.15.26:0/3041770490 learned_addr learned my addr 172.21.15.26:0/3041770490 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.635+0000 7ffbd2011700 1 -- 172.21.15.26:0/3041770490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8009550 con 0x7ffbcc07ac90 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.635+0000 7ffbd2011700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.635+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc80039f0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.636+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc19ba90 con 0x7ffbcc07ac90 2023-07-13T22:05:44.529 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.636+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbc8003b50 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.636+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbc801a780 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.636+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc19beb0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.637+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc10b6f0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.637+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ffbc801a9f0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.637+0000 7ffbc37fe700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 0x7ffbb803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.637+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffbc804c7e0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.530 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.638+0000 7ffbd1810700 1 -- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 msgr2=0x7ffbb803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.638+0000 7ffbd1810700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 0x7ffbb803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.642+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffbc80237b0 con 0x7ffbcc07ac90 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.839+0000 7ffbd1810700 1 -- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 msgr2=0x7ffbb803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.839+0000 7ffbd1810700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 0x7ffbb803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.944+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ffbcc19ef70 con 0x7ffbcc07ac90 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.945+0000 7ffbc37fe700 1 -- 172.21.15.26:0/3041770490 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ffbc802c350 con 0x7ffbcc07ac90 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 msgr2=0x7ffbb803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:05:44.531 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 0x7ffbb803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 msgr2=0x7ffbcc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc8011a30 comp rx=0 tx=0).stop 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 shutdown_connections 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7ffbb80387b0 0x7ffbb803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 --2- 172.21.15.26:0/3041770490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbcc07ac90 0x7ffbcc19b350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 >> 172.21.15.26:0/3041770490 conn(0x7ffbcc1009f0 msgr2=0x7ffbcc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.947+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 shutdown_connections 2023-07-13T22:05:44.532 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:43.948+0000 7ffbd4275700 1 -- 172.21.15.26:0/3041770490 wait complete. 2023-07-13T22:05:44.533 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for the mgr to restart... 2023-07-13T22:05:44.533 INFO:teuthology.orchestra.run.smithi026.stdout:Waiting for mgr epoch 9... 2023-07-13T22:05:45.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:44 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3041770490' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-13T22:05:45.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:44.892+0000 7fb20a2a7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:05:45.958 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:45.671+0000 7fb20a2a7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:05:45.959 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:45.883+0000 7fb20a2a7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:05:47.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:46.861+0000 7fb20a2a7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:47.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:46.985+0000 7fb20a2a7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:05:47.581 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:47.214+0000 7fb20a2a7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:05:47.862 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:47.584+0000 7fb20a2a7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:05:48.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:47.865+0000 7fb20a2a7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:05:48.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:48.064+0000 7fb20a2a7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:05:48.708 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:48.418+0000 7fb20a2a7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:05:49.708 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:49.376+0000 7fb20a2a7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:05:49.709 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:49.554+0000 7fb20a2a7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:05:50.208 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:49.805+0000 7fb20a2a7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:05:50.209 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:49.958+0000 7fb20a2a7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:05:50.708 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:50.240+0000 7fb20a2a7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:05:51.708 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:05:51.336+0000 7fb20a2a7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:05:51.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: Active manager daemon x restarted 2023-07-13T22:05:51.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: Activating manager daemon x 2023-07-13T22:05:51.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: osdmap e3: 0 total, 0 up, 0 in 2023-07-13T22:05:51.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: mgrmap e10: x(active, starting, since 0.00424515s) 2023-07-13T22:05:51.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: Manager daemon x is now available 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:51.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:05:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-13T22:05:52.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-13T22:05:52.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:52 smithi026 ceph-mon[95647]: mgrmap e11: x(active, since 1.00808s) 2023-07-13T22:05:52.727 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout { 2023-07-13T22:05:52.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-13T22:05:52.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-13T22:05:52.728 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout } 2023-07-13T22:05:52.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.098+0000 7fc8538cf700 1 Processor -- start 2023-07-13T22:05:52.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.098+0000 7fc8538cf700 1 -- start start 2023-07-13T22:05:52.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.098+0000 7fc8538cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:52.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.098+0000 7fc8538cf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0796d0 con 0x7fc84c07ac90 2023-07-13T22:05:52.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.099+0000 7fc85166b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:52.730 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.099+0000 7fc85166b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53700/0 (socket says 172.21.15.26:53700) 2023-07-13T22:05:52.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.099+0000 7fc85166b700 1 -- 172.21.15.26:0/3176105728 learned_addr learned my addr 172.21.15.26:0/3176105728 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:52.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.099+0000 7fc85166b700 1 -- 172.21.15.26:0/3176105728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c079810 con 0x7fc84c07ac90 2023-07-13T22:05:52.731 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.099+0000 7fc85166b700 1 --2- 172.21.15.26:0/3176105728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8480098a0 tx=0x7fc848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5152b8e6d68901ca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:52.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc843fff700 1 -- 172.21.15.26:0/3176105728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc848004160 con 0x7fc84c07ac90 2023-07-13T22:05:52.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc843fff700 1 -- 172.21.15.26:0/3176105728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8480042c0 con 0x7fc84c07ac90 2023-07-13T22:05:52.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc843fff700 1 -- 172.21.15.26:0/3176105728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8480119b0 con 0x7fc84c07ac90 2023-07-13T22:05:52.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 -- 172.21.15.26:0/3176105728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 msgr2=0x7fc84c079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:52.732 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 --2- 172.21.15.26:0/3176105728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8480098a0 tx=0x7fc848009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.733 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 -- 172.21.15.26:0/3176105728 shutdown_connections 2023-07-13T22:05:52.733 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 --2- 172.21.15.26:0/3176105728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c079190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.733 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 -- 172.21.15.26:0/3176105728 >> 172.21.15.26:0/3176105728 conn(0x7fc84c1009f0 msgr2=0x7fc84c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:52.733 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.100+0000 7fc8538cf700 1 -- 172.21.15.26:0/3176105728 shutdown_connections 2023-07-13T22:05:52.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.101+0000 7fc8538cf700 1 -- 172.21.15.26:0/3176105728 wait complete. 2023-07-13T22:05:52.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc8538cf700 1 Processor -- start 2023-07-13T22:05:52.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc8538cf700 1 -- start start 2023-07-13T22:05:52.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc8538cf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:52.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc8538cf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c19b890 con 0x7fc84c07ac90 2023-07-13T22:05:52.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc85166b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:52.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc85166b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53706/0 (socket says 172.21.15.26:53706) 2023-07-13T22:05:52.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.102+0000 7fc85166b700 1 -- 172.21.15.26:0/953521394 learned_addr learned my addr 172.21.15.26:0/953521394 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc85166b700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848009550 con 0x7fc84c07ac90 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc85166b700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc848000c00 tx=0x7fc848011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8480039f0 con 0x7fc84c07ac90 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc848003b50 con 0x7fc84c07ac90 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c19ba90 con 0x7fc84c07ac90 2023-07-13T22:05:52.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc84801a780 con 0x7fc84c07ac90 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.103+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c19beb0 con 0x7fc84c07ac90 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.104+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc848024440 con 0x7fc84c07ac90 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.105+0000 7fc8427fc700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.105+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc83803b370 con 0x7fc8380387b0 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.105+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc84804c860 con 0x7fc84c07ac90 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.105+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.105+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.306+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.306+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.707+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:45.707+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:46.508+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:46.508+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:48.109+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:48.109+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:51.313+0000 7fc850e6a700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.26:6800/2707453820 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:51.313+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:51.341+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fc84802d910 con 0x7fc84c07ac90 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:51.341+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 msgr2=0x7fc83803ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:51.341+0000 7fc8427fc700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.343+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fc84804e0d0 con 0x7fc84c07ac90 2023-07-13T22:05:52.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.343+0000 7fc8427fc700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 0x7fc83803e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.344+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc83803b370 con 0x7fc83803bff0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.344+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 0x7fc83803e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.345+0000 7fc850e6a700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 0x7fc83803e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc83c003a10 tx=0x7fc83c009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.346+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc83803b370 con 0x7fc83803bff0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.353+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc84c19c690 con 0x7fc83803bff0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.353+0000 7fc8427fc700 1 -- 172.21.15.26:0/953521394 <== mgr.14148 v2:172.21.15.26:6800/4053809718 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc84c19c690 con 0x7fc83803bff0 2023-07-13T22:05:52.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 msgr2=0x7fc83803e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 0x7fc83803e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc83c003a10 tx=0x7fc83c009270 comp rx=0 tx=0).stop 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 msgr2=0x7fc84c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc848000c00 tx=0x7fc848011a30 comp rx=0 tx=0).stop 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 shutdown_connections 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc83803bff0 0x7fc83803e3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc84c07ac90 0x7fc84c19b350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 --2- 172.21.15.26:0/953521394 >> [v2:172.21.15.26:6800/2707453820,v1:172.21.15.26:6801/2707453820] conn(0x7fc8380387b0 0x7fc83803ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:52.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 >> 172.21.15.26:0/953521394 conn(0x7fc84c1009f0 msgr2=0x7fc84c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 shutdown_connections 2023-07-13T22:05:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:52.354+0000 7fc8538cf700 1 -- 172.21.15.26:0/953521394 wait complete. 2023-07-13T22:05:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:mgr epoch 9 is available 2023-07-13T22:05:52.742 INFO:teuthology.orchestra.run.smithi026.stdout:Generating a dashboard self-signed certificate... 2023-07-13T22:05:53.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:51] ENGINE Bus STARTING 2023-07-13T22:05:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:51] ENGINE Serving on http://172.21.15.26:8765 2023-07-13T22:05:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:51] ENGINE Serving on https://172.21.15.26:7150 2023-07-13T22:05:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: [13/Jul/2023:22:05:51] ENGINE Bus STARTED 2023-07-13T22:05:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-13T22:05:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:53 smithi026 ceph-mon[95647]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-13T22:05:54.111 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-13T22:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.266+0000 7fe04c0f6700 1 Processor -- start 2023-07-13T22:05:54.112 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.266+0000 7fe04c0f6700 1 -- start start 2023-07-13T22:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.266+0000 7fe04c0f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.267+0000 7fe04c0f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440796f0 con 0x7fe04407acb0 2023-07-13T22:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.267+0000 7fe049e92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:54.113 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.267+0000 7fe049e92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43556/0 (socket says 172.21.15.26:43556) 2023-07-13T22:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.267+0000 7fe049e92700 1 -- 172.21.15.26:0/1640684178 learned_addr learned my addr 172.21.15.26:0/1640684178 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.268+0000 7fe049e92700 1 -- 172.21.15.26:0/1640684178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044079830 con 0x7fe04407acb0 2023-07-13T22:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.268+0000 7fe049e92700 1 --2- 172.21.15.26:0/1640684178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098a0 tx=0x7fe040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fef11adb133e84 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:54.114 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.268+0000 7fe048e90700 1 -- 172.21.15.26:0/1640684178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe040004160 con 0x7fe04407acb0 2023-07-13T22:05:54.115 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.268+0000 7fe048e90700 1 -- 172.21.15.26:0/1640684178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0400042c0 con 0x7fe04407acb0 2023-07-13T22:05:54.115 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.268+0000 7fe048e90700 1 -- 172.21.15.26:0/1640684178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0400119b0 con 0x7fe04407acb0 2023-07-13T22:05:54.115 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 -- 172.21.15.26:0/1640684178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 msgr2=0x7fe0440791b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:54.116 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/1640684178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0400098a0 tx=0x7fe040009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:54.116 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 -- 172.21.15.26:0/1640684178 shutdown_connections 2023-07-13T22:05:54.116 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/1640684178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe0440791b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:54.116 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 -- 172.21.15.26:0/1640684178 >> 172.21.15.26:0/1640684178 conn(0x7fe0441009f0 msgr2=0x7fe044102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:54.117 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 -- 172.21.15.26:0/1640684178 shutdown_connections 2023-07-13T22:05:54.117 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.269+0000 7fe04c0f6700 1 -- 172.21.15.26:0/1640684178 wait complete. 2023-07-13T22:05:54.117 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.270+0000 7fe04c0f6700 1 Processor -- start 2023-07-13T22:05:54.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe04c0f6700 1 -- start start 2023-07-13T22:05:54.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe04c0f6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:54.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe04c0f6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04419b880 con 0x7fe04407acb0 2023-07-13T22:05:54.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe049e92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:54.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe049e92700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43558/0 (socket says 172.21.15.26:43558) 2023-07-13T22:05:54.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.271+0000 7fe049e92700 1 -- 172.21.15.26:0/916487196 learned_addr learned my addr 172.21.15.26:0/916487196 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:54.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe049e92700 1 -- 172.21.15.26:0/916487196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040009550 con 0x7fe04407acb0 2023-07-13T22:05:54.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe049e92700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe040003e60 tx=0x7fe040011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:54.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe040003970 con 0x7fe04407acb0 2023-07-13T22:05:54.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe040003ad0 con 0x7fe04407acb0 2023-07-13T22:05:54.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0400195e0 con 0x7fe04407acb0 2023-07-13T22:05:54.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04419ba80 con 0x7fe04407acb0 2023-07-13T22:05:54.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.272+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04419bea0 con 0x7fe04407acb0 2023-07-13T22:05:54.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.273+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fe04001a430 con 0x7fe04407acb0 2023-07-13T22:05:54.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.274+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044061b40 con 0x7fe04407acb0 2023-07-13T22:05:54.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.274+0000 7fe03affd700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 0x7fe03003aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:54.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.274+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe04004c7d0 con 0x7fe04407acb0 2023-07-13T22:05:54.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.274+0000 7fe049691700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 0x7fe03003aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:54.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.275+0000 7fe049691700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 0x7fe03003aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe034006fd0 tx=0x7fe034006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.284+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe040003c40 con 0x7fe04407acb0 2023-07-13T22:05:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.396+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fe040023390 con 0x7fe04407acb0 2023-07-13T22:05:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.513+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe044102cf0 con 0x7fe030038640 2023-07-13T22:05:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.609+0000 7fe03affd700 1 -- 172.21.15.26:0/916487196 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe044102cf0 con 0x7fe030038640 2023-07-13T22:05:54.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.611+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 msgr2=0x7fe03003aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:54.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.611+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 0x7fe03003aaf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe034006fd0 tx=0x7fe034006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:54.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 msgr2=0x7fe04419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:54.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe040003e60 tx=0x7fe040011aa0 comp rx=0 tx=0).stop 2023-07-13T22:05:54.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 shutdown_connections 2023-07-13T22:05:54.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe030038640 0x7fe03003aaf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:54.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 --2- 172.21.15.26:0/916487196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe04407acb0 0x7fe04419b340 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:54.126 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 >> 172.21.15.26:0/916487196 conn(0x7fe0441009f0 msgr2=0x7fe044102090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:54.126 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 shutdown_connections 2023-07-13T22:05:54.126 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:53.612+0000 7fe04c0f6700 1 -- 172.21.15.26:0/916487196 wait complete. 2023-07-13T22:05:54.126 INFO:teuthology.orchestra.run.smithi026.stdout:Creating initial admin user... 2023-07-13T22:05:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:54 smithi026 ceph-mon[95647]: mgrmap e12: x(active, since 2s) 2023-07-13T22:05:54.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:54 smithi026 ceph-mon[95647]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:54.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:05:54.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:05:54.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:05:55.693 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mFSeGDHpmJE1BDifrQZ0NevX0l.gTjrvGiB7uLPPiCyGE5FOlE5PO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689285955, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.643+0000 7f53af323700 1 Processor -- start 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.643+0000 7f53af323700 1 -- start start 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.643+0000 7f53af323700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.643+0000 7f53af323700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a80796d0 con 0x7f53a807ac90 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.644+0000 7f53ad0bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:55.694 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.644+0000 7f53ad0bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43570/0 (socket says 172.21.15.26:43570) 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.644+0000 7f53ad0bf700 1 -- 172.21.15.26:0/1915697140 learned_addr learned my addr 172.21.15.26:0/1915697140 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.644+0000 7f53ad0bf700 1 -- 172.21.15.26:0/1915697140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8079810 con 0x7f53a807ac90 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.644+0000 7f53ad0bf700 1 --2- 172.21.15.26:0/1915697140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dc4952e334058f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.645+0000 7f539ffff700 1 -- 172.21.15.26:0/1915697140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53a4004160 con 0x7f53a807ac90 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.645+0000 7f539ffff700 1 -- 172.21.15.26:0/1915697140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53a40042c0 con 0x7f53a807ac90 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.645+0000 7f539ffff700 1 -- 172.21.15.26:0/1915697140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53a40119b0 con 0x7f53a807ac90 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.645+0000 7f53af323700 1 -- 172.21.15.26:0/1915697140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 msgr2=0x7f53a8079190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:55.695 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.645+0000 7f53af323700 1 --2- 172.21.15.26:0/1915697140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f53a40098a0 tx=0x7f53a4009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.646+0000 7f53af323700 1 -- 172.21.15.26:0/1915697140 shutdown_connections 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.646+0000 7f53af323700 1 --2- 172.21.15.26:0/1915697140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a8079190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.646+0000 7f53af323700 1 -- 172.21.15.26:0/1915697140 >> 172.21.15.26:0/1915697140 conn(0x7f53a81009f0 msgr2=0x7f53a8102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.646+0000 7f53af323700 1 -- 172.21.15.26:0/1915697140 shutdown_connections 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.646+0000 7f53af323700 1 -- 172.21.15.26:0/1915697140 wait complete. 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.647+0000 7f53af323700 1 Processor -- start 2023-07-13T22:05:55.696 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.647+0000 7f53af323700 1 -- start start 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.648+0000 7f53af323700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.648+0000 7f53af323700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a819b8c0 con 0x7f53a807ac90 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.648+0000 7f53ad0bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.648+0000 7f53ad0bf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43572/0 (socket says 172.21.15.26:43572) 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.648+0000 7f53ad0bf700 1 -- 172.21.15.26:0/1154169935 learned_addr learned my addr 172.21.15.26:0/1154169935 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f53ad0bf700 1 -- 172.21.15.26:0/1154169935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4009550 con 0x7f53a807ac90 2023-07-13T22:05:55.697 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f53ad0bf700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000c00 tx=0x7f53a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:55.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53a40039f0 con 0x7f53a807ac90 2023-07-13T22:05:55.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53a4003b50 con 0x7f53a807ac90 2023-07-13T22:05:55.698 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f53a401a780 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a819bac0 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.649+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a819bee0 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.650+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f53a4024440 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.650+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a810b6f0 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.651+0000 7f539e7fc700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 0x7f539403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.651+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f53a404c940 con 0x7f53a807ac90 2023-07-13T22:05:55.699 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.651+0000 7f53ac8be700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 0x7f539403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.652+0000 7f53ac8be700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 0x7f539403ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.657+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f53a40189a0 con 0x7f53a807ac90 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:54.886+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f53a8061960 con 0x7f53940387b0 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.134+0000 7f539e7fc700 1 -- 172.21.15.26:0/1154169935 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f53a8061960 con 0x7f53940387b0 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.136+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 msgr2=0x7f539403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.136+0000 7f53af323700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 0x7f539403ac60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5398006fd0 tx=0x7f5398006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:55.700 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.136+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 msgr2=0x7f53a819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.136+0000 7f53af323700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f53a4000c00 tx=0x7f53a4011a30 comp rx=0 tx=0).stop 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 shutdown_connections 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f53940387b0 0x7f539403ac60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 --2- 172.21.15.26:0/1154169935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53a807ac90 0x7f53a819b380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 >> 172.21.15.26:0/1154169935 conn(0x7f53a81009f0 msgr2=0x7f53a81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 shutdown_connections 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:55.137+0000 7f53af323700 1 -- 172.21.15.26:0/1154169935 wait complete. 2023-07-13T22:05:55.701 INFO:teuthology.orchestra.run.smithi026.stdout:Fetching dashboard port number... 2023-07-13T22:05:56.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:56 smithi026 ceph-mon[95647]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:05:56.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:05:56.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:05:57.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:57 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2995247555' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-13T22:05:57.277 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stdout 8443 2023-07-13T22:05:57.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.296+0000 7f1c219da700 1 Processor -- start 2023-07-13T22:05:57.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.296+0000 7f1c219da700 1 -- start start 2023-07-13T22:05:57.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.296+0000 7f1c219da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:57.278 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.296+0000 7f1c219da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c105cc0 con 0x7f1c1c105370 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.297+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.297+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43580/0 (socket says 172.21.15.26:43580) 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.297+0000 7f1c1affd700 1 -- 172.21.15.26:0/3898934392 learned_addr learned my addr 172.21.15.26:0/3898934392 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.297+0000 7f1c1affd700 1 -- 172.21.15.26:0/3898934392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c105e00 con 0x7f1c1c105370 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.298+0000 7f1c1affd700 1 --2- 172.21.15.26:0/3898934392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0098a0 tx=0x7f1c0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b6284e154b5a9e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:57.279 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.298+0000 7f1c19ffb700 1 -- 172.21.15.26:0/3898934392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c0c004160 con 0x7f1c1c105370 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.298+0000 7f1c19ffb700 1 -- 172.21.15.26:0/3898934392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0c0042c0 con 0x7f1c1c105370 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.298+0000 7f1c19ffb700 1 -- 172.21.15.26:0/3898934392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c0c0119b0 con 0x7f1c1c105370 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 -- 172.21.15.26:0/3898934392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 msgr2=0x7f1c1c105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 --2- 172.21.15.26:0/3898934392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0098a0 tx=0x7f1c0c009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 -- 172.21.15.26:0/3898934392 shutdown_connections 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 --2- 172.21.15.26:0/3898934392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c105780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:57.280 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 -- 172.21.15.26:0/3898934392 >> 172.21.15.26:0/3898934392 conn(0x7f1c1c1009e0 msgr2=0x7f1c1c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 -- 172.21.15.26:0/3898934392 shutdown_connections 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.299+0000 7f1c219da700 1 -- 172.21.15.26:0/3898934392 wait complete. 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.300+0000 7f1c219da700 1 Processor -- start 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.300+0000 7f1c219da700 1 -- start start 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c219da700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:57.281 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c219da700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c19b8a0 con 0x7f1c1c105370 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c1affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43596/0 (socket says 172.21.15.26:43596) 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c1affd700 1 -- 172.21.15.26:0/2995247555 learned_addr learned my addr 172.21.15.26:0/2995247555 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.301+0000 7f1c1affd700 1 -- 172.21.15.26:0/2995247555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009550 con 0x7f1c1c105370 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c1affd700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006af0 tx=0x7f1c0c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c0c003a40 con 0x7f1c1c105370 2023-07-13T22:05:57.282 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0c003ba0 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c19baa0 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c0c0196c0 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.302+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c19bec0 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.303+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1c0c01aa80 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.303+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c061b40 con 0x7f1c1c105370 2023-07-13T22:05:57.283 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.304+0000 7f1c209d8700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 0x7f1c0403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.304+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c0c04cc10 con 0x7f1c1c105370 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.304+0000 7f1c1a7fc700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 0x7f1c0403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.305+0000 7f1c1a7fc700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 0x7f1c0403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006fd0 tx=0x7f1c10006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.312+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1c0c01a430 con 0x7f1c1c105370 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.538+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1c1c109810 con 0x7f1c1c105370 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.539+0000 7f1c209d8700 1 -- 172.21.15.26:0/2995247555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1c0c02c3e0 con 0x7f1c1c105370 2023-07-13T22:05:57.284 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.541+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 msgr2=0x7f1c0403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:57.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.541+0000 7f1c219da700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 0x7f1c0403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006fd0 tx=0x7f1c10006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:57.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.541+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 msgr2=0x7f1c1c19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:57.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.541+0000 7f1c219da700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c006af0 tx=0x7f1c0c011a10 comp rx=0 tx=0).stop 2023-07-13T22:05:57.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.541+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 shutdown_connections 2023-07-13T22:05:57.285 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.542+0000 7f1c219da700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1c04038760 0x7f1c0403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:57.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.542+0000 7f1c219da700 1 --2- 172.21.15.26:0/2995247555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c1c105370 0x7f1c1c19b360 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:57.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.542+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 >> 172.21.15.26:0/2995247555 conn(0x7f1c1c1009e0 msgr2=0x7f1c1c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:57.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.542+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 shutdown_connections 2023-07-13T22:05:57.286 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:56.542+0000 7f1c219da700 1 -- 172.21.15.26:0/2995247555 wait complete. 2023-07-13T22:05:57.296 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-13T22:05:57.296 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout disabled 2023-07-13T22:05:57.312 INFO:teuthology.orchestra.run.smithi026.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-13T22:05:57.313 INFO:teuthology.orchestra.run.smithi026.stdout:systemctl: stdout inactive 2023-07-13T22:05:57.313 INFO:teuthology.orchestra.run.smithi026.stdout:firewalld.service is not enabled 2023-07-13T22:05:57.313 INFO:teuthology.orchestra.run.smithi026.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-13T22:05:57.315 INFO:teuthology.orchestra.run.smithi026.stdout:Ceph Dashboard is now available at: 2023-07-13T22:05:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:05:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: URL: https://smithi026.front.sepia.ceph.com:8443/ 2023-07-13T22:05:57.315 INFO:teuthology.orchestra.run.smithi026.stdout: User: admin 2023-07-13T22:05:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: Password: 61nkhvitlc 2023-07-13T22:05:57.316 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:05:57.316 INFO:teuthology.orchestra.run.smithi026.stdout:Saving cluster configuration to /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config directory 2023-07-13T22:05:57.317 INFO:teuthology.orchestra.run.smithi026.stdout:Enabling autotune for osd_memory_target 2023-07-13T22:05:58.729 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.010+0000 7fb07f3fa700 1 Processor -- start 2023-07-13T22:05:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.011+0000 7fb07f3fa700 1 -- start start 2023-07-13T22:05:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.011+0000 7fb07f3fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.011+0000 7fb07f3fa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb078079710 con 0x7fb07807acd0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.011+0000 7fb07d196700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb07d196700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43598/0 (socket says 172.21.15.26:43598) 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb07d196700 1 -- 172.21.15.26:0/2253436930 learned_addr learned my addr 172.21.15.26:0/2253436930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb07d196700 1 -- 172.21.15.26:0/2253436930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078079850 con 0x7fb07807acd0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb07d196700 1 --2- 172.21.15.26:0/2253436930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098a0 tx=0x7fb074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a25707fb7646179 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb06ffff700 1 -- 172.21.15.26:0/2253436930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb074004160 con 0x7fb07807acd0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb06ffff700 1 -- 172.21.15.26:0/2253436930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0740042c0 con 0x7fb07807acd0 2023-07-13T22:05:58.735 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.012+0000 7fb06ffff700 1 -- 172.21.15.26:0/2253436930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0740119b0 con 0x7fb07807acd0 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2253436930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 msgr2=0x7fb0780791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2253436930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098a0 tx=0x7fb074009bb0 comp rx=0 tx=0).stop 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2253436930 shutdown_connections 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2253436930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb0780791d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2253436930 >> 172.21.15.26:0/2253436930 conn(0x7fb0781009f0 msgr2=0x7fb078102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2253436930 shutdown_connections 2023-07-13T22:05:58.736 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.013+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2253436930 wait complete. 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.014+0000 7fb07f3fa700 1 Processor -- start 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.014+0000 7fb07f3fa700 1 -- start start 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07f3fa700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07f3fa700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07819b8c0 con 0x7fb07807acd0 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07d196700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07d196700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43606/0 (socket says 172.21.15.26:43606) 2023-07-13T22:05:58.737 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07d196700 1 -- 172.21.15.26:0/2994163425 learned_addr learned my addr 172.21.15.26:0/2994163425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07d196700 1 -- 172.21.15.26:0/2994163425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb074009550 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.015+0000 7fb07d196700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb074003e60 tx=0x7fb074011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.016+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb074003af0 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.016+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb074003c50 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.016+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07819bac0 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.016+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb074024800 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.016+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07819bee0 con 0x7fb07807acd0 2023-07-13T22:05:58.738 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.017+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fb07401ab80 con 0x7fb07807acd0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.017+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb078061b40 con 0x7fb07807acd0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.017+0000 7fb06e7fc700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 0x7fb06403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.017+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb07404c9f0 con 0x7fb07807acd0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.018+0000 7fb07c995700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 0x7fb06403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.018+0000 7fb07c995700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 0x7fb06403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.022+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb07401a950 con 0x7fb07807acd0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.048+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb07401a430 con 0x7fb07807acd0 2023-07-13T22:05:58.739 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.230+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb07819e920 con 0x7fb07807acd0 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.230+0000 7fb06e7fc700 1 -- 172.21.15.26:0/2994163425 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb07401a950 con 0x7fb07807acd0 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 msgr2=0x7fb06403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 0x7fb06403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb068006fd0 tx=0x7fb068006e40 comp rx=0 tx=0).stop 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 msgr2=0x7fb07819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb074003e60 tx=0x7fb074011980 comp rx=0 tx=0).stop 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 shutdown_connections 2023-07-13T22:05:58.740 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb064038760 0x7fb06403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:58.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 --2- 172.21.15.26:0/2994163425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb07807acd0 0x7fb07819b380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:05:58.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 >> 172.21.15.26:0/2994163425 conn(0x7fb0781009f0 msgr2=0x7fb0781020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:05:58.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 shutdown_connections 2023-07-13T22:05:58.741 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:58.233+0000 7fb07f3fa700 1 -- 172.21.15.26:0/2994163425 wait complete. 2023-07-13T22:05:59.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:05:59 smithi026 ceph-mon[95647]: mgrmap e13: x(active, since 6s) 2023-07-13T22:06:00.117 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.310+0000 7fda73ab9700 1 Processor -- start 2023-07-13T22:06:00.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.310+0000 7fda73ab9700 1 -- start start 2023-07-13T22:06:00.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.310+0000 7fda73ab9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:00.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.310+0000 7fda73ab9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c079710 con 0x7fda6c07acd0 2023-07-13T22:06:00.118 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda71855700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda71855700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43614/0 (socket says 172.21.15.26:43614) 2023-07-13T22:06:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda71855700 1 -- 172.21.15.26:0/3892408418 learned_addr learned my addr 172.21.15.26:0/3892408418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda71855700 1 -- 172.21.15.26:0/3892408418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c079850 con 0x7fda6c07acd0 2023-07-13T22:06:00.119 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda71855700 1 --2- 172.21.15.26:0/3892408418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda680098a0 tx=0x7fda68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ebc12051f9c8ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:00.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.311+0000 7fda70853700 1 -- 172.21.15.26:0/3892408418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda68004160 con 0x7fda6c07acd0 2023-07-13T22:06:00.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda70853700 1 -- 172.21.15.26:0/3892408418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda680042c0 con 0x7fda6c07acd0 2023-07-13T22:06:00.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda70853700 1 -- 172.21.15.26:0/3892408418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda680119b0 con 0x7fda6c07acd0 2023-07-13T22:06:00.120 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 -- 172.21.15.26:0/3892408418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 msgr2=0x7fda6c0791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:00.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 --2- 172.21.15.26:0/3892408418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fda680098a0 tx=0x7fda68009bb0 comp rx=0 tx=0).stop 2023-07-13T22:06:00.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 -- 172.21.15.26:0/3892408418 shutdown_connections 2023-07-13T22:06:00.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 --2- 172.21.15.26:0/3892408418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c0791d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:00.121 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 -- 172.21.15.26:0/3892408418 >> 172.21.15.26:0/3892408418 conn(0x7fda6c1009f0 msgr2=0x7fda6c102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:00.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 -- 172.21.15.26:0/3892408418 shutdown_connections 2023-07-13T22:06:00.122 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.312+0000 7fda73ab9700 1 -- 172.21.15.26:0/3892408418 wait complete. 2023-07-13T22:06:00.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.313+0000 7fda73ab9700 1 Processor -- start 2023-07-13T22:06:00.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda73ab9700 1 -- start start 2023-07-13T22:06:00.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda73ab9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:00.123 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda73ab9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c19b8c0 con 0x7fda6c07acd0 2023-07-13T22:06:00.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda71855700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:00.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda71855700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43616/0 (socket says 172.21.15.26:43616) 2023-07-13T22:06:00.124 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda71855700 1 -- 172.21.15.26:0/2446907938 learned_addr learned my addr 172.21.15.26:0/2446907938 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:00.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.314+0000 7fda71855700 1 -- 172.21.15.26:0/2446907938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda68009550 con 0x7fda6c07acd0 2023-07-13T22:06:00.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda71855700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda68003e60 tx=0x7fda68011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:00.125 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda68003af0 con 0x7fda6c07acd0 2023-07-13T22:06:00.126 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c19bac0 con 0x7fda6c07acd0 2023-07-13T22:06:00.128 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c19bee0 con 0x7fda6c07acd0 2023-07-13T22:06:00.128 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda68003c50 con 0x7fda6c07acd0 2023-07-13T22:06:00.128 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.315+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda68024760 con 0x7fda6c07acd0 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.316+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fda6801a430 con 0x7fda6c07acd0 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.316+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c061b40 con 0x7fda6c07acd0 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.317+0000 7fda62ffd700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 0x7fda5803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.317+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda6804cb40 con 0x7fda6c07acd0 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.317+0000 7fda71054700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 0x7fda5803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:00.129 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.317+0000 7fda71054700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 0x7fda5803ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006fd0 tx=0x7fda5c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.322+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fda6801ab40 con 0x7fda6c07acd0 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.637+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fda6c19e920 con 0x7fda6c07acd0 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.640+0000 7fda62ffd700 1 -- 172.21.15.26:0/2446907938 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fda6802cb00 con 0x7fda6c07acd0 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.642+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 msgr2=0x7fda5803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.642+0000 7fda73ab9700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 0x7fda5803ad00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006fd0 tx=0x7fda5c006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.642+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 msgr2=0x7fda6c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.642+0000 7fda73ab9700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fda68003e60 tx=0x7fda68011980 comp rx=0 tx=0).stop 2023-07-13T22:06:00.130 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 shutdown_connections 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda58038850 0x7fda5803ad00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 --2- 172.21.15.26:0/2446907938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda6c07acd0 0x7fda6c19b380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 >> 172.21.15.26:0/2446907938 conn(0x7fda6c1009f0 msgr2=0x7fda6c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 shutdown_connections 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr 2023-07-13T22:05:59.643+0000 7fda73ab9700 1 -- 172.21.15.26:0/2446907938 wait complete. 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-13T22:06:00.131 INFO:teuthology.orchestra.run.smithi026.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout:Or, if you are only running a single cluster on this host: 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-13T22:06:00.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: ceph telemetry on 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout:For more information see: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-13T22:06:00.133 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:00.134 INFO:teuthology.orchestra.run.smithi026.stdout:Bootstrap complete. 2023-07-13T22:06:00.176 INFO:tasks.cephadm:Fetching config... 2023-07-13T22:06:00.177 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:00.177 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-13T22:06:00.220 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-13T22:06:00.221 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:00.221 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-13T22:06:00.307 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-13T22:06:00.307 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:00.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-13T22:06:00.418 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-13T22:06:00.419 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:00.419 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-13T22:06:00.502 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-13T22:06:00.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-13T22:06:00.647 INFO:teuthology.orchestra.run.smithi026.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:00.671 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:00 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2446907938' entity='client.admin' 2023-07-13T22:06:00.681 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-13T22:06:00.725 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:00.738 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-13T22:06:00.784 INFO:teuthology.orchestra.run.smithi121.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaiawHXTxqJmYc3ViP6hJkRrM59EYrK8yojbd8ekGAkPVkz3Tt5HM3bx0VP8Q5GXGFRlyYiqz8WN+DkrOrCs7ECy2BWCm+Owlt1TYNZSI+u5uva25LkCYyEHBXi94XXFKRA1sveaiRwT8KU0/hSKjFxXrQE25IGTZJdvEJBukHv9H7ewGgRUQkClVB+D82HRTWqwzEYF+Kdz8iWHhs/BQ5Io32Pzpd06oUapq2h0UqYoVSkzoVYS1yQhdmuiHI9WJsPKGpXXsdnSv/xiNMWKhHfhXzSmkPRWh4tXY5XEpwCd1G6Ei//ite7l7Ld1tFsPomdRxGB2LchnnNnZADNdfnEzdmzLo3ekcb7ZzghYJe0W8ovUNP02Oyt3nzyQRcWlzPFO6Px3qbR375HpqXgLKPishZLDjNchoPm/Crl3StU0i1CyM0rMU0gHe8WfDSEhlZ0u3pMgusmborO+24eBj57dm8yn/v10YIsSDJmiEN2frlY7slpeR5cdRIqSJKpW8= ceph-23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:00.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-13T22:06:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 -- 172.21.15.26:0/493859835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 msgr2=0x7f01c80fe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 --2- 172.21.15.26:0/493859835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c80fe2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009910 tx=0x7f01b8009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 -- 172.21.15.26:0/493859835 shutdown_connections 2023-07-13T22:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 --2- 172.21.15.26:0/493859835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c80fe2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 -- 172.21.15.26:0/493859835 >> 172.21.15.26:0/493859835 conn(0x7f01c80f9a70 msgr2=0x7f01c80fbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:03.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 -- 172.21.15.26:0/493859835 shutdown_connections 2023-07-13T22:06:03.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.929+0000 7f01cdb43700 1 -- 172.21.15.26:0/493859835 wait complete. 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.930+0000 7f01cdb43700 1 Processor -- start 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.930+0000 7f01cdb43700 1 -- start start 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.930+0000 7f01cdb43700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.930+0000 7f01cdb43700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8196e60 con 0x7f01c80fdee0 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.931+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.931+0000 7f01c77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34722/0 (socket says 172.21.15.26:34722) 2023-07-13T22:06:03.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.931+0000 7f01c77fe700 1 -- 172.21.15.26:0/500686477 learned_addr learned my addr 172.21.15.26:0/500686477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:03.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.931+0000 7f01c77fe700 1 -- 172.21.15.26:0/500686477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80095f0 con 0x7f01c80fdee0 2023-07-13T22:06:03.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.931+0000 7f01c77fe700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005f50 tx=0x7f01b80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.932+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b801d070 con 0x7f01c80fdee0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.932+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c8197060 con 0x7f01c80fdee0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.932+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01b8022470 con 0x7f01c80fdee0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.932+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01b800f5f0 con 0x7f01c80fdee0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.932+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c8197500 con 0x7f01c80fdee0 2023-07-13T22:06:03.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.933+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f01b800ba90 con 0x7f01c80fdee0 2023-07-13T22:06:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.933+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c81903a0 con 0x7f01c80fdee0 2023-07-13T22:06:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.934+0000 7f01c4ff9700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 0x7f01b003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:03.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.934+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f01b804cbb0 con 0x7f01c80fdee0 2023-07-13T22:06:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.934+0000 7f01c6ffd700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 0x7f01b003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.935+0000 7f01c6ffd700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 0x7f01b003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:03.939+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f01b802bba0 con 0x7f01c80fdee0 2023-07-13T22:06:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.145+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f01c802c850 con 0x7f01c80fdee0 2023-07-13T22:06:04.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.148+0000 7f01c4ff9700 1 -- 172.21.15.26:0/500686477 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f01b8027030 con 0x7f01c80fdee0 2023-07-13T22:06:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 msgr2=0x7f01b003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 0x7f01b003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01bc006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 msgr2=0x7f01c8196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f01b8005f50 tx=0x7f01b80057c0 comp rx=0 tx=0).stop 2023-07-13T22:06:04.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 shutdown_connections 2023-07-13T22:06:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01b00387b0 0x7f01b003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 --2- 172.21.15.26:0/500686477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c80fdee0 0x7f01c8196920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 >> 172.21.15.26:0/500686477 conn(0x7f01c80f9a70 msgr2=0x7f01c80fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 shutdown_connections 2023-07-13T22:06:04.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:04.151+0000 7f01cdb43700 1 -- 172.21.15.26:0/500686477 wait complete. 2023-07-13T22:06:04.618 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-13T22:06:04.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-13T22:06:04.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:04.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi026", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/500686477' entity='client.admin' 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:04.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:05.350 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.840+0000 7f7b36fe8700 1 -- 172.21.15.26:0/2906183700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 msgr2=0x7f7b300ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.840+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/2906183700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b300ff9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009910 tx=0x7f7b24009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.841+0000 7f7b36fe8700 1 -- 172.21.15.26:0/2906183700 shutdown_connections 2023-07-13T22:06:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.841+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/2906183700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b300ff9f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.841+0000 7f7b36fe8700 1 -- 172.21.15.26:0/2906183700 >> 172.21.15.26:0/2906183700 conn(0x7f7b300fab90 msgr2=0x7f7b300fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.841+0000 7f7b36fe8700 1 -- 172.21.15.26:0/2906183700 shutdown_connections 2023-07-13T22:06:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.841+0000 7f7b36fe8700 1 -- 172.21.15.26:0/2906183700 wait complete. 2023-07-13T22:06:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.842+0000 7f7b36fe8700 1 Processor -- start 2023-07-13T22:06:06.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.842+0000 7f7b36fe8700 1 -- start start 2023-07-13T22:06:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.842+0000 7f7b36fe8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:06.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.842+0000 7f7b36fe8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30196eb0 con 0x7f7b300ff5e0 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.842+0000 7f7b34d84700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b34d84700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34740/0 (socket says 172.21.15.26:34740) 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b34d84700 1 -- 172.21.15.26:0/4142903808 learned_addr learned my addr 172.21.15.26:0/4142903808 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b34d84700 1 -- 172.21.15.26:0/4142903808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240095f0 con 0x7f7b300ff5e0 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b34d84700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24004f40 tx=0x7f7b240057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:06.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b2401d070 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b2400bc10 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b24003a70 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.843+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b301970b0 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.844+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30197550 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.845+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f7b24003d30 con 0x7f7b300ff5e0 2023-07-13T22:06:06.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.845+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b301905b0 con 0x7f7b300ff5e0 2023-07-13T22:06:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.845+0000 7f7b2dffb700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 0x7f7b2003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.845+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7b2404c570 con 0x7f7b300ff5e0 2023-07-13T22:06:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.845+0000 7f7b2ffff700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 0x7f7b2003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:06.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.846+0000 7f7b2ffff700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 0x7f7b2003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c006fd0 tx=0x7f7b1c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:06.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:06.850+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7b2402abf0 con 0x7f7b300ff5e0 2023-07-13T22:06:07.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.065+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b300609e0 con 0x7f7b200387b0 2023-07-13T22:06:07.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.069+0000 7f7b2dffb700 1 -- 172.21.15.26:0/4142903808 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7b300609e0 con 0x7f7b200387b0 2023-07-13T22:06:07.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 msgr2=0x7f7b2003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:07.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 0x7f7b2003ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c006fd0 tx=0x7f7b1c006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 msgr2=0x7f7b30196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b24004f40 tx=0x7f7b240057c0 comp rx=0 tx=0).stop 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 shutdown_connections 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7b200387b0 0x7f7b2003ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 --2- 172.21.15.26:0/4142903808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300ff5e0 0x7f7b30196970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.071+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 >> 172.21.15.26:0/4142903808 conn(0x7f7b300fab90 msgr2=0x7f7b300fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.072+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 shutdown_connections 2023-07-13T22:06:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:07.072+0000 7f7b36fe8700 1 -- 172.21.15.26:0/4142903808 wait complete. 2023-07-13T22:06:07.528 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2023-07-13T22:06:07.529 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:06:07.529 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2023-07-13T22:06:07.549 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:06:07.550 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:07.609 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2023-07-13T22:06:07.609 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch host add smithi117 2023-07-13T22:06:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: Updating smithi026:/etc/ceph/ceph.conf 2023-07-13T22:06:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:08 smithi026 ceph-mon[95647]: Updating smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:09.299 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:09 smithi026 ceph-mon[95647]: Updating smithi026:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 -- 172.21.15.26:0/3314830266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 msgr2=0x7f6b20105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 --2- 172.21.15.26:0/3314830266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b20105a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009910 tx=0x7f6b10009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 -- 172.21.15.26:0/3314830266 shutdown_connections 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 --2- 172.21.15.26:0/3314830266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b20105a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 -- 172.21.15.26:0/3314830266 >> 172.21.15.26:0/3314830266 conn(0x7f6b200fb130 msgr2=0x7f6b200fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:09.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 -- 172.21.15.26:0/3314830266 shutdown_connections 2023-07-13T22:06:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.946+0000 7f6b267e7700 1 -- 172.21.15.26:0/3314830266 wait complete. 2023-07-13T22:06:09.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.947+0000 7f6b267e7700 1 Processor -- start 2023-07-13T22:06:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.947+0000 7f6b267e7700 1 -- start start 2023-07-13T22:06:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b267e7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b267e7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019cea0 con 0x7f6b20105630 2023-07-13T22:06:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:09.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b1ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34746/0 (socket says 172.21.15.26:34746) 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b1ffff700 1 -- 172.21.15.26:0/4023397462 learned_addr learned my addr 172.21.15.26:0/4023397462 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.948+0000 7f6b1ffff700 1 -- 172.21.15.26:0/4023397462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b100095f0 con 0x7f6b20105630 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b1ffff700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b1001d070 con 0x7f6b20105630 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1000bc10 con 0x7f6b20105630 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2019d100 con 0x7f6b20105630 2023-07-13T22:06:09.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b10003910 con 0x7f6b20105630 2023-07-13T22:06:09.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.949+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b201969e0 con 0x7f6b20105630 2023-07-13T22:06:09.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.950+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6b1000bd80 con 0x7f6b20105630 2023-07-13T22:06:09.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.950+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20061b40 con 0x7f6b20105630 2023-07-13T22:06:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.950+0000 7f6b1d7fa700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 0x7f6b0803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.950+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b1004c600 con 0x7f6b20105630 2023-07-13T22:06:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.951+0000 7f6b1f7fe700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 0x7f6b0803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:09.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.951+0000 7f6b1f7fe700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 0x7f6b0803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006fd0 tx=0x7f6b14006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:09.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:09.955+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6b10029360 con 0x7f6b20105630 2023-07-13T22:06:10.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:10.165+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b201974a0 con 0x7f6b080387b0 2023-07-13T22:06:10.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:10 smithi026 ceph-mon[95647]: Updating smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:10.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:10.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:10.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:11.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:11 smithi026 ceph-mon[95647]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:12.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.508+0000 7f6b1d7fa700 1 -- 172.21.15.26:0/4023397462 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f6b201974a0 con 0x7f6b080387b0 2023-07-13T22:06:12.504 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi117' with addr '172.21.15.117' 2023-07-13T22:06:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 msgr2=0x7f6b0803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 0x7f6b0803ac60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006fd0 tx=0x7f6b14006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:12.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 msgr2=0x7f6b2019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 shutdown_connections 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6b080387b0 0x7f6b0803ac60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 --2- 172.21.15.26:0/4023397462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b20105630 0x7f6b2019c960 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.510+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 >> 172.21.15.26:0/4023397462 conn(0x7f6b200fb130 msgr2=0x7f6b200fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.511+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 shutdown_connections 2023-07-13T22:06:12.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:12.511+0000 7f6b267e7700 1 -- 172.21.15.26:0/4023397462 wait complete. 2023-07-13T22:06:12.652 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:12 smithi026 ceph-mon[95647]: Deploying cephadm binary to smithi117 2023-07-13T22:06:12.652 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:12 smithi026 ceph-mon[95647]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:13.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch host ls --format=json 2023-07-13T22:06:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:13.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:13.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:13 smithi026 ceph-mon[95647]: Added host smithi117 2023-07-13T22:06:13.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:13.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:14.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:14 smithi026 ceph-mon[95647]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:14.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:15.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 -- 172.21.15.26:0/1940488360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 msgr2=0x7f2e24072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 --2- 172.21.15.26:0/1940488360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e24072c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009910 tx=0x7f2e14009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 -- 172.21.15.26:0/1940488360 shutdown_connections 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 --2- 172.21.15.26:0/1940488360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e24072c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 -- 172.21.15.26:0/1940488360 >> 172.21.15.26:0/1940488360 conn(0x7f2e240fb0d0 msgr2=0x7f2e240fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 -- 172.21.15.26:0/1940488360 shutdown_connections 2023-07-13T22:06:15.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.312+0000 7f2e2a999700 1 -- 172.21.15.26:0/1940488360 wait complete. 2023-07-13T22:06:15.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.313+0000 7f2e2a999700 1 Processor -- start 2023-07-13T22:06:15.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e2a999700 1 -- start start 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e2a999700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e2a999700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419cef0 con 0x7f2e240728b0 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e23fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59900/0 (socket says 172.21.15.26:59900) 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.314+0000 7f2e23fff700 1 -- 172.21.15.26:0/4279431886 learned_addr learned my addr 172.21.15.26:0/4279431886 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e23fff700 1 -- 172.21.15.26:0/4279431886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140095f0 con 0x7f2e240728b0 2023-07-13T22:06:15.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e23fff700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005850 tx=0x7f2e14004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:15.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e1401d070 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e14004370 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e14003ab0 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2419d0f0 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.315+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2419d510 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.316+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f2e1400b6f0 con 0x7f2e240728b0 2023-07-13T22:06:15.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.316+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24061b40 con 0x7f2e240728b0 2023-07-13T22:06:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.317+0000 7f2e217fa700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 0x7f2e0c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.317+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2e1404c600 con 0x7f2e240728b0 2023-07-13T22:06:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.317+0000 7f2e237fe700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 0x7f2e0c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:15.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.318+0000 7f2e237fe700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 0x7f2e0c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:15.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.322+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2e1402a590 con 0x7f2e240728b0 2023-07-13T22:06:15.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.535+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e24108b90 con 0x7f2e0c0387b0 2023-07-13T22:06:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.537+0000 7f2e217fa700 1 -- 172.21.15.26:0/4279431886 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f2e24108b90 con 0x7f2e0c0387b0 2023-07-13T22:06:15.532 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:15.533 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 msgr2=0x7f2e0c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 0x7f2e0c03ac60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 msgr2=0x7f2e2419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e14005850 tx=0x7f2e14004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 shutdown_connections 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2e0c0387b0 0x7f2e0c03ac60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 --2- 172.21.15.26:0/4279431886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e240728b0 0x7f2e2419c9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 >> 172.21.15.26:0/4279431886 conn(0x7f2e240fb0d0 msgr2=0x7f2e241060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 shutdown_connections 2023-07-13T22:06:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:15.539+0000 7f2e2a999700 1 -- 172.21.15.26:0/4279431886 wait complete. 2023-07-13T22:06:16.053 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi121 2023-07-13T22:06:16.053 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:06:16.054 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.conf 2023-07-13T22:06:16.074 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:06:16.074 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:16.132 INFO:tasks.cephadm:Adding host smithi121 to orchestrator... 2023-07-13T22:06:16.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch host add smithi121 2023-07-13T22:06:16.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:16.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:15 smithi026 ceph-mon[95647]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:16.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:15 smithi026 ceph-mon[95647]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:06:16.859 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.417+0000 7f4de1072700 1 -- 172.21.15.26:0/2743125728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 msgr2=0x7f4ddc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.417+0000 7f4de1072700 1 --2- 172.21.15.26:0/2743125728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc072c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009910 tx=0x7f4dcc009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.417+0000 7f4de1072700 1 -- 172.21.15.26:0/2743125728 shutdown_connections 2023-07-13T22:06:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.417+0000 7f4de1072700 1 --2- 172.21.15.26:0/2743125728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc072c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.417+0000 7f4de1072700 1 -- 172.21.15.26:0/2743125728 >> 172.21.15.26:0/2743125728 conn(0x7f4ddc0fb130 msgr2=0x7f4ddc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.418+0000 7f4de1072700 1 -- 172.21.15.26:0/2743125728 shutdown_connections 2023-07-13T22:06:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.418+0000 7f4de1072700 1 -- 172.21.15.26:0/2743125728 wait complete. 2023-07-13T22:06:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.419+0000 7f4de1072700 1 Processor -- start 2023-07-13T22:06:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.419+0000 7f4de1072700 1 -- start start 2023-07-13T22:06:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.419+0000 7f4de1072700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.419+0000 7f4de1072700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc19cf40 con 0x7f4ddc0728b0 2023-07-13T22:06:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.420+0000 7f4ddad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.420+0000 7f4ddad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59924/0 (socket says 172.21.15.26:59924) 2023-07-13T22:06:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.420+0000 7f4ddad9d700 1 -- 172.21.15.26:0/3409342210 learned_addr learned my addr 172.21.15.26:0/3409342210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.420+0000 7f4ddad9d700 1 -- 172.21.15.26:0/3409342210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc0095f0 con 0x7f4ddc0728b0 2023-07-13T22:06:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.421+0000 7f4ddad9d700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006010 tx=0x7f4dcc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.422+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc01d070 con 0x7f4ddc0728b0 2023-07-13T22:06:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.423+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc19d140 con 0x7f4ddc0728b0 2023-07-13T22:06:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.423+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc19d560 con 0x7f4ddc0728b0 2023-07-13T22:06:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dcc0044a0 con 0x7f4ddc0728b0 2023-07-13T22:06:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4dcc003a10 con 0x7f4ddc0728b0 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4dcc003c30 con 0x7f4ddc0728b0 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4dc3fff700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 0x7f4dc403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4dcc04d500 con 0x7f4ddc0728b0 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.424+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc8005320 con 0x7f4ddc0728b0 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.425+0000 7f4dda59c700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 0x7f4dc403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.425+0000 7f4dda59c700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 0x7f4dc403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.430+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4dcc026080 con 0x7f4ddc0728b0 2023-07-13T22:06:18.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:18.641+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dc8000bf0 con 0x7f4dc4038800 2023-07-13T22:06:18.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:18.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: Updating smithi117:/etc/ceph/ceph.conf 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:18.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:19.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:19 smithi026 ceph-mon[95647]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:19.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:19 smithi026 ceph-mon[95647]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:19.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:19 smithi026 ceph-mon[95647]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:19.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:19 smithi026 ceph-mon[95647]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:20.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:20 smithi026 ceph-mon[95647]: Deploying cephadm binary to smithi121 2023-07-13T22:06:20.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:20 smithi026 ceph-mon[95647]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.992+0000 7f4dc3fff700 1 -- 172.21.15.26:0/3409342210 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4dc8000bf0 con 0x7f4dc4038800 2023-07-13T22:06:20.989 INFO:teuthology.orchestra.run.smithi026.stdout:Added host 'smithi121' with addr '172.21.15.121' 2023-07-13T22:06:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 msgr2=0x7f4dc403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 0x7f4dc403acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0006fd0 tx=0x7f4dd0006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 msgr2=0x7f4ddc19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006010 tx=0x7f4dcc004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 shutdown_connections 2023-07-13T22:06:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dc4038800 0x7f4dc403acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 --2- 172.21.15.26:0/3409342210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ddc0728b0 0x7f4ddc19ca00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 >> 172.21.15.26:0/3409342210 conn(0x7f4ddc0fb130 msgr2=0x7f4ddc106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 shutdown_connections 2023-07-13T22:06:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:20.995+0000 7f4de1072700 1 -- 172.21.15.26:0/3409342210 wait complete. 2023-07-13T22:06:21.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch host ls --format=json 2023-07-13T22:06:22.190 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:22.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:22.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:21 smithi026 ceph-mon[95647]: Added host smithi121 2023-07-13T22:06:22.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:22.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:21 smithi026 ceph-mon[95647]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:22.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:23.438 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 -- 172.21.15.26:0/2276225562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 msgr2=0x7fc1e4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:23.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 --2- 172.21.15.26:0/2276225562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e4072c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009910 tx=0x7fc1e0009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:23.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 -- 172.21.15.26:0/2276225562 shutdown_connections 2023-07-13T22:06:23.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 --2- 172.21.15.26:0/2276225562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e4072c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:23.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 -- 172.21.15.26:0/2276225562 >> 172.21.15.26:0/2276225562 conn(0x7fc1e40fb130 msgr2=0x7fc1e40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:23.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 -- 172.21.15.26:0/2276225562 shutdown_connections 2023-07-13T22:06:23.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.691+0000 7fc1ec793700 1 -- 172.21.15.26:0/2276225562 wait complete. 2023-07-13T22:06:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.692+0000 7fc1ec793700 1 Processor -- start 2023-07-13T22:06:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.692+0000 7fc1ec793700 1 -- start start 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ec793700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ec793700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e419cf40 con 0x7fc1e40728b0 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ea52f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ea52f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47984/0 (socket says 172.21.15.26:47984) 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ea52f700 1 -- 172.21.15.26:0/3548437421 learned_addr learned my addr 172.21.15.26:0/3548437421 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.693+0000 7fc1ea52f700 1 -- 172.21.15.26:0/3548437421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00095f0 con 0x7fc1e40728b0 2023-07-13T22:06:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1ea52f700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006010 tx=0x7fc1e0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e001d070 con 0x7fc1e40728b0 2023-07-13T22:06:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e00044a0 con 0x7fc1e40728b0 2023-07-13T22:06:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e419d140 con 0x7fc1e40728b0 2023-07-13T22:06:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e0003a70 con 0x7fc1e40728b0 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.694+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e419d560 con 0x7fc1e40728b0 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.695+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4061b40 con 0x7fc1e40728b0 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.695+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fc1e0003c60 con 0x7fc1e40728b0 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.695+0000 7fc1db7fe700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 0x7fc1d003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.696+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1e004c660 con 0x7fc1e40728b0 2023-07-13T22:06:23.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.696+0000 7fc1e9d2e700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 0x7fc1d003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:23.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.696+0000 7fc1e9d2e700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 0x7fc1d003acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4006fd0 tx=0x7fc1d4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.701+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc1e0005c00 con 0x7fc1e40728b0 2023-07-13T22:06:23.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.916+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1e4108c50 con 0x7fc1d0038800 2023-07-13T22:06:23.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.917+0000 7fc1db7fe700 1 -- 172.21.15.26:0/3548437421 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fc1e4108c50 con 0x7fc1d0038800 2023-07-13T22:06:23.914 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:06:23.914 INFO:teuthology.orchestra.run.smithi026.stdout:[{"addr": "172.21.15.26", "hostname": "smithi026", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}] 2023-07-13T22:06:23.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.919+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 msgr2=0x7fc1d003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:23.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.919+0000 7fc1ec793700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 0x7fc1d003acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4006fd0 tx=0x7fc1d4006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.919+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 msgr2=0x7fc1e419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.919+0000 7fc1ec793700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006010 tx=0x7fc1e0004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 shutdown_connections 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc1d0038800 0x7fc1d003acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 --2- 172.21.15.26:0/3548437421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40728b0 0x7fc1e419ca00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 >> 172.21.15.26:0/3548437421 conn(0x7fc1e40fb130 msgr2=0x7fc1e4106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:23.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 shutdown_connections 2023-07-13T22:06:23.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:23.920+0000 7fc1ec793700 1 -- 172.21.15.26:0/3548437421 wait complete. 2023-07-13T22:06:24.368 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-13T22:06:24.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd crush tunables default 2023-07-13T22:06:24.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:24 smithi026 ceph-mon[95647]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:24.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:25.104 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:25.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:25 smithi026 ceph-mon[95647]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:06:26.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.617+0000 7f2bd4731700 1 -- 172.21.15.26:0/1621237673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 msgr2=0x7f2bcc105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.617+0000 7f2bd4731700 1 --2- 172.21.15.26:0/1621237673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc105a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009910 tx=0x7f2bc8009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.618+0000 7f2bd4731700 1 -- 172.21.15.26:0/1621237673 shutdown_connections 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.618+0000 7f2bd4731700 1 --2- 172.21.15.26:0/1621237673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc105a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.618+0000 7f2bd4731700 1 -- 172.21.15.26:0/1621237673 >> 172.21.15.26:0/1621237673 conn(0x7f2bcc0fb130 msgr2=0x7f2bcc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.618+0000 7f2bd4731700 1 -- 172.21.15.26:0/1621237673 shutdown_connections 2023-07-13T22:06:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.618+0000 7f2bd4731700 1 -- 172.21.15.26:0/1621237673 wait complete. 2023-07-13T22:06:26.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.619+0000 7f2bd4731700 1 Processor -- start 2023-07-13T22:06:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.619+0000 7f2bd4731700 1 -- start start 2023-07-13T22:06:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.619+0000 7f2bd4731700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.619+0000 7f2bd4731700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc19cea0 con 0x7f2bcc105630 2023-07-13T22:06:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bd24cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:26.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bd24cd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48002/0 (socket says 172.21.15.26:48002) 2023-07-13T22:06:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bd24cd700 1 -- 172.21.15.26:0/4139942799 learned_addr learned my addr 172.21.15.26:0/4139942799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bd24cd700 1 -- 172.21.15.26:0/4139942799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80095f0 con 0x7f2bcc105630 2023-07-13T22:06:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bd24cd700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8006010 tx=0x7f2bc8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.620+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bc801d070 con 0x7f2bcc105630 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.621+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc800bc10 con 0x7f2bcc105630 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.621+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bc8003910 con 0x7f2bcc105630 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.621+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc19d100 con 0x7f2bcc105630 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.621+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc1969e0 con 0x7f2bcc105630 2023-07-13T22:06:26.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.622+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f2bc800bd80 con 0x7f2bcc105630 2023-07-13T22:06:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.622+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc061b40 con 0x7f2bcc105630 2023-07-13T22:06:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.622+0000 7f2bc37fe700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 0x7f2bb803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.622+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2bc804c4a0 con 0x7f2bcc105630 2023-07-13T22:06:26.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.623+0000 7f2bd1ccc700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 0x7f2bb803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:26.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.623+0000 7f2bd1ccc700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 0x7f2bb803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:26.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.628+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2bc8029360 con 0x7f2bcc105630 2023-07-13T22:06:26.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:26.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:26.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:26.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:26.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:26.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:06:26.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:26.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:26.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:26.856+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f2bcc197ef0 con 0x7f2bcc105630 2023-07-13T22:06:27.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:27.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:27.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:27 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-13T22:06:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.772+0000 7f2bc37fe700 1 -- 172.21.15.26:0/4139942799 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2bc8026020 con 0x7f2bcc105630 2023-07-13T22:06:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 msgr2=0x7f2bb803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 0x7f2bb803acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 msgr2=0x7f2bcc19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8006010 tx=0x7f2bc8004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 shutdown_connections 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb8038800 0x7f2bb803acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 --2- 172.21.15.26:0/4139942799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc105630 0x7f2bcc19c960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 >> 172.21.15.26:0/4139942799 conn(0x7f2bcc0fb130 msgr2=0x7f2bcc0fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 shutdown_connections 2023-07-13T22:06:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:27.775+0000 7f2bd4731700 1 -- 172.21.15.26:0/4139942799 wait complete. 2023-07-13T22:06:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2023-07-13T22:06:28.185 INFO:tasks.cephadm:Adding mon.a on smithi026 2023-07-13T22:06:28.185 INFO:tasks.cephadm:Adding mon.b on smithi117 2023-07-13T22:06:28.185 INFO:tasks.cephadm:Adding mon.c on smithi121 2023-07-13T22:06:28.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch apply mon '3;smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c' 2023-07-13T22:06:28.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:28 smithi026 ceph-mon[95647]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:28.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-13T22:06:28.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:28 smithi026 ceph-mon[95647]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:28.770 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:29.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 -- 172.21.15.121:0/1887750594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 msgr2=0x7f0380072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:29.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 --2- 172.21.15.121:0/1887750594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f0380072c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0374009910 tx=0x7f0374009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:29.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 -- 172.21.15.121:0/1887750594 shutdown_connections 2023-07-13T22:06:29.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 --2- 172.21.15.121:0/1887750594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f0380072c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:29.868 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 -- 172.21.15.121:0/1887750594 >> 172.21.15.121:0/1887750594 conn(0x7f03800fb130 msgr2=0x7f03800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.870+0000 7f038623f700 1 -- 172.21.15.121:0/1887750594 shutdown_connections 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.871+0000 7f038623f700 1 -- 172.21.15.121:0/1887750594 wait complete. 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.871+0000 7f038623f700 1 Processor -- start 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.871+0000 7f038623f700 1 -- start start 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f038623f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f038623f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038019d000 con 0x7f0380072850 2023-07-13T22:06:29.869 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f037f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:29.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f037f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.121:55546/0 (socket says 172.21.15.121:55546) 2023-07-13T22:06:29.870 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f037f7fe700 1 -- 172.21.15.121:0/3203136820 learned_addr learned my addr 172.21.15.121:0/3203136820 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.872+0000 7f037f7fe700 1 -- 172.21.15.121:0/3203136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03740095f0 con 0x7f0380072850 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f037f7fe700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0374005850 tx=0x7f0374004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f037401d070 con 0x7f0380072850 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038019d200 con 0x7f0380072850 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0374004370 con 0x7f0380072850 2023-07-13T22:06:29.871 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0374003ab0 con 0x7f0380072850 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.873+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038019d620 con 0x7f0380072850 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.874+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0380061b40 con 0x7f0380072850 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.874+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f037400b6f0 con 0x7f0380072850 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.874+0000 7f037cff9700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 0x7f036803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.874+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f037404c550 con 0x7f0380072850 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.874+0000 7f037effd700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 0x7f036803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:29.872 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.875+0000 7f037effd700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 0x7f036803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0370006fd0 tx=0x7f0370006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:29.875 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:29.879+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f037402a590 con 0x7f0380072850 2023-07-13T22:06:30.069 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.073+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0380108c40 con 0x7f03680387b0 2023-07-13T22:06:30.076 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.080+0000 7f037cff9700 1 -- 172.21.15.121:0/3203136820 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0380108c40 con 0x7f03680387b0 2023-07-13T22:06:30.076 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mon update... 2023-07-13T22:06:30.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 msgr2=0x7f036803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:30.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 0x7f036803ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0370006fd0 tx=0x7f0370006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:30.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 msgr2=0x7f038019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:30.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0374005850 tx=0x7f0374004ff0 comp rx=0 tx=0).stop 2023-07-13T22:06:30.078 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 shutdown_connections 2023-07-13T22:06:30.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03680387b0 0x7f036803ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:30.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 --2- 172.21.15.121:0/3203136820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0380072850 0x7f038019cac0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:30.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 >> 172.21.15.121:0/3203136820 conn(0x7f03800fb130 msgr2=0x7f03801061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:30.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 shutdown_connections 2023-07-13T22:06:30.079 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:30.082+0000 7f038623f700 1 -- 172.21.15.121:0/3203136820 wait complete. 2023-07-13T22:06:30.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:29 smithi026 ceph-mon[95647]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:30.443 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.b.service 2023-07-13T22:06:30.446 DEBUG:teuthology.orchestra.run.smithi121:mon.c> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.c.service 2023-07-13T22:06:30.448 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-13T22:06:30.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-13T22:06:30.477 INFO:journalctl@ceph.mon.c.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:06:30.483 INFO:journalctl@ceph.mon.b.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:06:31.210 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:31.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:31.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:31.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:31.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:31.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:31 smithi026 ceph-mon[95647]: Deploying daemon mon.c on smithi121 2023-07-13T22:06:32.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:32 smithi026 ceph-mon[95647]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 -- 172.21.15.121:0/1577145139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 msgr2=0x7f6cd80fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/1577145139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80fe820 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009910 tx=0x7f6cd4009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 -- 172.21.15.121:0/1577145139 shutdown_connections 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/1577145139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80fe820 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 -- 172.21.15.121:0/1577145139 >> 172.21.15.121:0/1577145139 conn(0x7f6cd8074f10 msgr2=0x7f6cd8075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:32.925 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 -- 172.21.15.121:0/1577145139 shutdown_connections 2023-07-13T22:06:32.926 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 -- 172.21.15.121:0/1577145139 wait complete. 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.928+0000 7f6ce04c3700 1 Processor -- start 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6ce04c3700 1 -- start start 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6ce04c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6ce04c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd8101350 con 0x7f6cd80fe450 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6cde25f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:32.927 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6cde25f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.121:55552/0 (socket says 172.21.15.121:55552) 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.929+0000 7f6cde25f700 1 -- 172.21.15.121:0/3235094362 learned_addr learned my addr 172.21.15.121:0/3235094362 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6cde25f700 1 -- 172.21.15.121:0/3235094362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40095f0 con 0x7f6cd80fe450 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6cde25f700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4004f40 tx=0x7f6cd4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cd401d070 con 0x7f6cd80fe450 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd80ff910 con 0x7f6cd80fe450 2023-07-13T22:06:32.928 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd4004370 con 0x7f6cd80fe450 2023-07-13T22:06:32.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd80ffe10 con 0x7f6cd80fe450 2023-07-13T22:06:32.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.930+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cd40039a0 con 0x7f6cd80fe450 2023-07-13T22:06:32.929 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.931+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd8106c70 con 0x7f6cd80fe450 2023-07-13T22:06:32.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.931+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6cd4003b00 con 0x7f6cd80fe450 2023-07-13T22:06:32.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.932+0000 7f6ccf7fe700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 0x7f6cc403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:32.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.932+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6cd404c720 con 0x7f6cd80fe450 2023-07-13T22:06:32.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.932+0000 7f6cdda5e700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 0x7f6cc403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:32.930 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.932+0000 7f6cdda5e700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 0x7f6cc403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006fd0 tx=0x7f6cc8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:32.934 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:32.937+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6cd4005c00 con 0x7f6cd80fe450 2023-07-13T22:06:33.235 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.239+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6cd81000a0 con 0x7f6cd80fe450 2023-07-13T22:06:33.237 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.240+0000 7f6ccf7fe700 1 -- 172.21.15.121:0/3235094362 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f6cd4026070 con 0x7f6cd80fe450 2023-07-13T22:06:33.237 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:06:33.237 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","modified":"2023-07-13T22:04:50.317415Z","created":"2023-07-13T22:04:50.317415Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-13T22:06:33.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 msgr2=0x7f6cc403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:33.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 0x7f6cc403ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8006fd0 tx=0x7f6cc8006e40 comp rx=0 tx=0).stop 2023-07-13T22:06:33.239 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 msgr2=0x7f6cd80ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4004f40 tx=0x7f6cd4005020 comp rx=0 tx=0).stop 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 shutdown_connections 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc40387b0 0x7f6cc403ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 --2- 172.21.15.121:0/3235094362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cd80fe450 0x7f6cd80ff3d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 >> 172.21.15.121:0/3235094362 conn(0x7f6cd8074f10 msgr2=0x7f6cd8103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 shutdown_connections 2023-07-13T22:06:33.240 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:33.242+0000 7f6ce04c3700 1 -- 172.21.15.121:0/3235094362 wait complete. 2023-07-13T22:06:33.241 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2023-07-13T22:06:33.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 systemd[1]: Starting Ceph mon.c for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:06:33.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:33 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/3235094362' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:33.716 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 podman[129120]: 2023-07-13T22:06:34.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: set uid:gid to 167:167 (ceph:ceph) 2023-07-13T22:06:34.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-13T22:06:34.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pidfile_write: ignore empty --pid-file 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: load: jerasure load: lrc 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: RocksDB version: 7.9.2 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Git sha 0 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: DB SUMMARY 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: DB Session ID: N07JG71UA87QTUH3IA7I 2023-07-13T22:06:34.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: CURRENT file: CURRENT 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: IDENTITY file: IDENTITY 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.error_if_exists: 0 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.create_if_missing: 0 2023-07-13T22:06:34.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.paranoid_checks: 1 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.env: 0x55da7b468720 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.fs: PosixFileSystem 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.info_log: 0x55da7c27d280 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_file_opening_threads: 16 2023-07-13T22:06:34.004 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.statistics: (nil) 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.use_fsync: 0 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_log_file_size: 0 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.keep_log_file_num: 1000 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.recycle_log_file_num: 0 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_fallocate: 1 2023-07-13T22:06:34.005 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_mmap_reads: 0 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_mmap_writes: 0 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.use_direct_reads: 0 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.create_missing_column_families: 0 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.db_log_dir: 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.wal_dir: 2023-07-13T22:06:34.006 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.advise_random_on_open: 1 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.db_write_buffer_size: 0 2023-07-13T22:06:34.007 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.write_buffer_manager: 0x55da7c19e3c0 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.rate_limiter: (nil) 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.wal_recovery_mode: 2 2023-07-13T22:06:34.008 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enable_thread_tracking: 0 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enable_pipelined_write: 0 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.unordered_write: 0 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.row_cache: None 2023-07-13T22:06:34.009 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.wal_filter: None 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_ingest_behind: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.two_write_queues: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.manual_wal_flush: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.wal_compression: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.atomic_flush: 0 2023-07-13T22:06:34.010 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.log_readahead_size: 0 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.best_efforts_recovery: 0 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-13T22:06:34.011 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.allow_data_in_errors: 0 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.db_host_id: __hostname__ 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 bash[129120]: a08963ebc4f412a05f75beca93e6d17b99dbb139513269de9ffdc8fd0e0202c6 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_background_jobs: 2 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_background_compactions: -1 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_subcompactions: 1 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-13T22:06:34.012 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_total_wal_size: 0 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-13T22:06:34.013 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_open_files: -1 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bytes_per_sync: 0 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_readahead_size: 0 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_background_flushes: -1 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Compression algorithms supported: 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-13T22:06:34.014 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kZSTD supported: 0 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kXpressCompression supported: 0 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kLZ4HCCompression supported: 1 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kZlibCompression supported: 1 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kSnappyCompression supported: 1 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kLZ4Compression supported: 1 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: kBZip2Compression supported: 0 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-13T22:06:34.015 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.merge_operator: 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_filter: None 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_filter_factory: None 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.sst_partitioner_factory: None 2023-07-13T22:06:34.016 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55da7c27d380) 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: cache_index_and_filter_blocks: 1 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: pin_top_level_index_and_filter: 1 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: index_type: 0 2023-07-13T22:06:34.017 INFO:journalctl@ceph.mon.c.smithi121.stdout: data_block_index_type: 0 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: index_shortening: 1 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: checksum: 4 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: no_block_cache: 0 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_cache: 0x55da7c23d1f0 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_cache_name: BinnedLRUCache 2023-07-13T22:06:34.018 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_cache_options: 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: capacity : 536870912 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: num_shard_bits : 4 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: strict_capacity_limit : 0 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: high_pri_pool_ratio: 0.000 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_cache_compressed: (nil) 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: persistent_cache: (nil) 2023-07-13T22:06:34.019 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_size: 4096 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_size_deviation: 10 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_restart_interval: 16 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: index_block_restart_interval: 1 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: metadata_block_size: 4096 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: partition_filters: 0 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: use_delta_encoding: 1 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: filter_policy: bloomfilter 2023-07-13T22:06:34.020 INFO:journalctl@ceph.mon.c.smithi121.stdout: whole_key_filtering: 1 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: verify_compression: 0 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: read_amp_bytes_per_bit: 0 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: format_version: 5 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: enable_index_compression: 1 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: block_align: 0 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: max_auto_readahead_size: 262144 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: prepopulate_block_cache: 0 2023-07-13T22:06:34.021 INFO:journalctl@ceph.mon.c.smithi121.stdout: initial_auto_readahead_size: 8192 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout: num_file_reads_for_auto_readahead: 2 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.write_buffer_size: 33554432 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_write_buffer_number: 2 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression: NoCompression 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression: Disabled 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.prefix_extractor: nullptr 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-13T22:06:34.022 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.num_levels: 7 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-13T22:06:34.023 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-13T22:06:34.024 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-13T22:06:34.025 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.level: 32767 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.strategy: 0 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.enabled: false 2023-07-13T22:06:34.026 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.target_file_size_base: 67108864 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-13T22:06:34.027 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-13T22:06:34.028 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.arena_block_size: 1048576 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-13T22:06:34.029 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.disable_auto_compactions: 0 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-13T22:06:34.030 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.table_properties_collectors: 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.inplace_update_support: 0 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-13T22:06:34.031 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.bloom_locality: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.max_successive_merges: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.paranoid_file_checks: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.force_consistency_checks: 1 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.report_bg_io_stats: 0 2023-07-13T22:06:34.032 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.ttl: 2592000 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enable_blob_files: false 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.min_blob_size: 0 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_file_size: 268435456 2023-07-13T22:06:34.033 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.blob_file_starting_level: 0 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-13T22:06:34.034 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d1f15bac-d229-4ccb-a9ad-decadd9cd1e8 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285993794612, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285993795010, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689285993, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d1f15bac-d229-4ccb-a9ad-decadd9cd1e8", "db_session_id": "N07JG71UA87QTUH3IA7I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689285993795077, "job": 1, "event": "recovery_finished"} 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-13T22:06:34.035 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55da7c2d6000 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: DB pointer 0x55da7c2c2000 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout: ** DB Stats ** 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:06:34.036 INFO:journalctl@ceph.mon.c.smithi121.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: ** Compaction Stats [default] ** 2023-07-13T22:06:34.037 INFO:journalctl@ceph.mon.c.smithi121.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: ** Compaction Stats [default] ** 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:06:34.038 INFO:journalctl@ceph.mon.c.smithi121.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-13T22:06:34.039 INFO:journalctl@ceph.mon.c.smithi121.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: Block cache BinnedLRUCache@0x55da7c23d1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-13T22:06:34.040 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: using public_addr v2:172.21.15.121:0/0 -> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: starting mon.c rank -1 at public addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] at bind addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(???) e0 preinit fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).mds e1 new map 2023-07-13T22:06:34.041 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: e1 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: legacy client fscid: -1 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout: No filesystems configured 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-13T22:06:34.042 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:34.043 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Deploying cephadm binary to smithi121 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.044 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Added host smithi121 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:06:34.045 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:34.046 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.047 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.048 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: Deploying daemon mon.c on smithi121 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:34.049 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/3235094362' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:34.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-13T22:06:34.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 ceph-mon[129187]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-13T22:06:34.050 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:33 smithi121 systemd[1]: Started Ceph mon.c for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:06:34.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-13T22:06:34.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-13T22:06:35.316 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:06:39.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:39.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: mon.a calling monitor election 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: mon.c calling monitor election 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: fsmap 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: mgrmap e13: x(active, since 47s) 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: overall HEALTH_OK 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:39.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:39.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: mon.a calling monitor election 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: mon.c calling monitor election 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:39.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: fsmap 2023-07-13T22:06:39.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:39.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: mgrmap e13: x(active, since 47s) 2023-07-13T22:06:39.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: overall HEALTH_OK 2023-07-13T22:06:39.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:39.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:39.506 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.509+0000 7f79bd94e700 1 -- 172.21.15.121:0/3283654960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a00035c0 msgr2=0x7f79a0005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:39.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.509+0000 7f79bd94e700 1 --2- 172.21.15.121:0/3283654960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a00035c0 0x7f79a0005a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79b8100ba0 tx=0x7f79ac00fe60 comp rx=0 tx=0).stop 2023-07-13T22:06:39.507 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 -- 172.21.15.121:0/3283654960 shutdown_connections 2023-07-13T22:06:39.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 --2- 172.21.15.121:0/3283654960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a00035c0 0x7f79a0005a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 -- 172.21.15.121:0/3283654960 >> 172.21.15.121:0/3283654960 conn(0x7f79b80fb210 msgr2=0x7f79b80fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:39.508 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 -- 172.21.15.121:0/3283654960 shutdown_connections 2023-07-13T22:06:39.509 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 -- 172.21.15.121:0/3283654960 wait complete. 2023-07-13T22:06:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.510+0000 7f79bd94e700 1 Processor -- start 2023-07-13T22:06:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79bd94e700 1 -- start start 2023-07-13T22:06:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79bd94e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:39.513 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79bd94e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 0x7f79b81949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79bd94e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b819b3c0 con 0x7f79b819ae90 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79bd94e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b819b530 con 0x7f79a00035c0 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40510/0 (socket says 172.21.15.121:40510) 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 -- 172.21.15.121:0/1634885708 learned_addr learned my addr 172.21.15.121:0/1634885708 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:06:39.514 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 -- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 msgr2=0x7f79b819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 -- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 msgr2=0x7f79b819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.511+0000 7f79b6ffd700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b6ffd700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b67fc700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 0x7f79b81949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b67fc700 1 -- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 msgr2=0x7f79b819a950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b67fc700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.515 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b67fc700 1 -- 172.21.15.121:0/1634885708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac00f8b0 con 0x7f79b819ae90 2023-07-13T22:06:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79b67fc700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 0x7f79b81949d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f79a800b500 tx=0x7f79a800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f79a8011630 con 0x7f79b819ae90 2023-07-13T22:06:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a8011c50 con 0x7f79b819ae90 2023-07-13T22:06:39.516 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f79a8010cc0 con 0x7f79b819ae90 2023-07-13T22:06:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.512+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b8194fd0 con 0x7f79b819ae90 2023-07-13T22:06:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.513+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b81955a0 con 0x7f79b819ae90 2023-07-13T22:06:39.517 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.514+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f79a801a410 con 0x7f79b819ae90 2023-07-13T22:06:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.514+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b8061b40 con 0x7f79b819ae90 2023-07-13T22:06:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.514+0000 7f79bc94c700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 0x7f79a403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.514+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f79a804aac0 con 0x7f79b819ae90 2023-07-13T22:06:39.518 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.514+0000 7f79b6ffd700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 0x7f79a403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:39.519 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.515+0000 7f79b6ffd700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 0x7f79a403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79ac005ed0 tx=0x7f79ac001e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:39.520 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.520+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f79a8018310 con 0x7f79b819ae90 2023-07-13T22:06:39.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.783+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f79b819f7c0 con 0x7f79b819ae90 2023-07-13T22:06:39.780 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.783+0000 7f79bc94c700 1 -- 172.21.15.121:0/1634885708 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f79a8018310 con 0x7f79b819ae90 2023-07-13T22:06:39.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:06:39.781 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":2,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","modified":"2023-07-13T22:06:33.823994Z","created":"2023-07-13T22:04:50.317415Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-13T22:06:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 msgr2=0x7f79a403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 0x7f79a403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f79ac005ed0 tx=0x7f79ac001e70 comp rx=0 tx=0).stop 2023-07-13T22:06:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 msgr2=0x7f79b81949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 0x7f79b81949d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f79a800b500 tx=0x7f79a800b8c0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.782 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 shutdown_connections 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79a00035c0 0x7f79b819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f79a40385f0 0x7f79a403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 --2- 172.21.15.121:0/1634885708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79b819ae90 0x7f79b81949d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 >> 172.21.15.121:0/1634885708 conn(0x7f79b80fb210 msgr2=0x7f79b8103260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 shutdown_connections 2023-07-13T22:06:39.783 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:39.785+0000 7f79bd94e700 1 -- 172.21.15.121:0/1634885708 wait complete. 2023-07-13T22:06:39.784 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 2 2023-07-13T22:06:40.140 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: Deploying daemon mon.b on smithi117 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/1634885708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:40.141 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:40.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:40.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:40.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:40.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: Deploying daemon mon.b on smithi117 2023-07-13T22:06:40.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:40.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/1634885708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:40.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.164 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-13T22:06:41.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph mon dump -f json 2023-07-13T22:06:41.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 systemd[1]: Starting Ceph mon.b for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:06:41.726 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:06:41.898 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 podman[129034]: 2023-07-13T22:06:41.898 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: set uid:gid to 167:167 (ceph:ceph) 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: ceph version 18.0.0-4757-g3fc0089c (3fc0089cc1082313a98853d48b892acdc1385b8b) reef (dev), process ceph-mon, pid 7 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pidfile_write: ignore empty --pid-file 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 bash[129034]: 91c70bb935e6e2ebfed6191342f3e0c8bd321eb76c20f5278986a06dc6945f3c 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: load: jerasure load: lrc 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: RocksDB version: 7.9.2 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Git sha 0 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Compile date 2023-07-13 18:29:45 2023-07-13T22:06:41.899 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: DB SUMMARY 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: DB Session ID: ENF9M182HPU2KTMIGP6P 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: CURRENT file: CURRENT 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: IDENTITY file: IDENTITY 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-13T22:06:41.900 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.error_if_exists: 0 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.create_if_missing: 0 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.paranoid_checks: 1 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.env: 0x565232ef1720 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.fs: PosixFileSystem 2023-07-13T22:06:41.901 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.info_log: 0x565235691280 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_file_opening_threads: 16 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.statistics: (nil) 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.use_fsync: 0 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_log_file_size: 0 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-13T22:06:41.902 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.keep_log_file_num: 1000 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.recycle_log_file_num: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_fallocate: 1 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_mmap_reads: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_mmap_writes: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.use_direct_reads: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.create_missing_column_families: 0 2023-07-13T22:06:41.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.db_log_dir: 2023-07-13T22:06:41.905 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.wal_dir: 2023-07-13T22:06:41.905 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-13T22:06:41.905 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.advise_random_on_open: 1 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.db_write_buffer_size: 0 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.write_buffer_manager: 0x5652355b23c0 2023-07-13T22:06:41.906 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.rate_limiter: (nil) 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.wal_recovery_mode: 2 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enable_thread_tracking: 0 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enable_pipelined_write: 0 2023-07-13T22:06:41.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.unordered_write: 0 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.row_cache: None 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.wal_filter: None 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-13T22:06:41.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_ingest_behind: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.two_write_queues: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.manual_wal_flush: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.wal_compression: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.atomic_flush: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-13T22:06:41.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.log_readahead_size: 0 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.best_efforts_recovery: 0 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.allow_data_in_errors: 0 2023-07-13T22:06:41.910 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.db_host_id: __hostname__ 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_background_jobs: 2 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_background_compactions: -1 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_subcompactions: 1 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-13T22:06:41.911 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_total_wal_size: 0 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_open_files: -1 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bytes_per_sync: 0 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-13T22:06:41.912 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_readahead_size: 0 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_background_flushes: -1 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Compression algorithms supported: 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kZSTD supported: 0 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kXpressCompression supported: 0 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kLZ4HCCompression supported: 1 2023-07-13T22:06:41.913 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kZlibCompression supported: 1 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kSnappyCompression supported: 1 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kLZ4Compression supported: 1 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: kBZip2Compression supported: 0 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-13T22:06:41.914 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.merge_operator: 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_filter: None 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_filter_factory: None 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.sst_partitioner_factory: None 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-13T22:06:41.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x565235691380) 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks: 1 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_top_level_index_and_filter: 1 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_type: 0 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_index_type: 0 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_shortening: 1 2023-07-13T22:06:41.916 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: checksum: 4 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: no_block_cache: 0 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache: 0x5652356511f0 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_name: BinnedLRUCache 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_options: 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: capacity : 536870912 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: num_shard_bits : 4 2023-07-13T22:06:41.917 INFO:journalctl@ceph.mon.b.smithi117.stdout: strict_capacity_limit : 0 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: high_pri_pool_ratio: 0.000 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_compressed: (nil) 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: persistent_cache: (nil) 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size: 4096 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size_deviation: 10 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_restart_interval: 16 2023-07-13T22:06:41.918 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_block_restart_interval: 1 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: metadata_block_size: 4096 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: partition_filters: 0 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: use_delta_encoding: 1 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: filter_policy: bloomfilter 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: whole_key_filtering: 1 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: verify_compression: 0 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: read_amp_bytes_per_bit: 0 2023-07-13T22:06:41.919 INFO:journalctl@ceph.mon.b.smithi117.stdout: format_version: 5 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_index_compression: 1 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_align: 0 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: max_auto_readahead_size: 262144 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: prepopulate_block_cache: 0 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: initial_auto_readahead_size: 8192 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.write_buffer_size: 33554432 2023-07-13T22:06:41.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_write_buffer_number: 2 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression: NoCompression 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression: Disabled 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.prefix_extractor: nullptr 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.num_levels: 7 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-13T22:06:41.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-13T22:06:41.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.level: 32767 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.strategy: 0 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-13T22:06:41.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.enabled: false 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.target_file_size_base: 67108864 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-13T22:06:41.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-13T22:06:41.925 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.arena_block_size: 1048576 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-13T22:06:41.926 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.disable_auto_compactions: 0 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-13T22:06:41.927 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.table_properties_collectors: 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.inplace_update_support: 0 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-13T22:06:41.928 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.bloom_locality: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.max_successive_merges: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.paranoid_file_checks: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.force_consistency_checks: 1 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.report_bg_io_stats: 0 2023-07-13T22:06:41.929 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.ttl: 2592000 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enable_blob_files: false 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.min_blob_size: 0 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_file_size: 268435456 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-13T22:06:41.930 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.blob_file_starting_level: 0 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-13T22:06:41.931 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 74b15ff1-bc42-49d4-9a97-9e2cbae8f741 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689286001878477, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689286001878890, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689286001, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "74b15ff1-bc42-49d4-9a97-9e2cbae8f741", "db_session_id": "ENF9M182HPU2KTMIGP6P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689286001878955, "job": 1, "event": "recovery_finished"} 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-13T22:06:41.932 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5652356ea000 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: DB pointer 0x5652356d6000 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** DB Stats ** 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:41.933 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:06:41.934 INFO:journalctl@ceph.mon.b.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-13T22:06:41.935 INFO:journalctl@ceph.mon.b.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-13T22:06:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: Block cache BinnedLRUCache@0x5652356511f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: starting mon.b rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(???) e0 preinit fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).mds e1 new map 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-13T22:06:41.938 INFO:journalctl@ceph.mon.b.smithi117.stdout: e1 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout: legacy client fscid: -1 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout: No filesystems configured 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-13T22:06:41.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-13T22:06:41.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Deploying cephadm binary to smithi121 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Added host smithi121 2023-07-13T22:06:41.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:41.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.client.admin.keyring 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-13T22:06:41.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4139942799' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Saving service mon spec with placement smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Deploying daemon mon.c on smithi121 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/3235094362' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:41.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.a calling monitor election 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.c calling monitor election 2023-07-13T22:06:41.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: monmap e2: 2 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: fsmap 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:41.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mgrmap e13: x(active, since 47s) 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: overall HEALTH_OK 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: Deploying daemon mon.b on smithi117 2023-07-13T22:06:41.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/1634885708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expand map: {default=false} 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta from 'false' to 'false' 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expanded map: {default=false} 2023-07-13T22:06:41.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expand map: {default=info} 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta from 'info' to 'info' 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expanded map: {default=info} 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expand map: {default=daemon} 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expanded map: {default=daemon} 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expand map: {default=debug} 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta from 'debug' to 'debug' 2023-07-13T22:06:41.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 ceph-mon[129074]: expand_channel_meta expanded map: {default=debug} 2023-07-13T22:06:42.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:41 smithi117 systemd[1]: Started Ceph mon.b for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:06:47.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: mon.c calling monitor election 2023-07-13T22:06:47.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:47.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: mon.a calling monitor election 2023-07-13T22:06:47.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:47.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:47.260 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.260 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.261 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:47.261 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.261 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.261 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:47.261 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: fsmap 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: mgrmap e13: x(active, since 55s) 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: mon.b (rank 2) addr [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] is down (out of quorum) 2023-07-13T22:06:47.262 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.263 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.267 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 -- 172.21.15.121:0/1288174866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 msgr2=0x7f122c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 --2- 172.21.15.121:0/1288174866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f122c005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f12440fe540 tx=0x7f123400fe60 comp rx=0 tx=0).stop 2023-07-13T22:06:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 -- 172.21.15.121:0/1288174866 shutdown_connections 2023-07-13T22:06:47.268 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 --2- 172.21.15.121:0/1288174866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f122c005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 -- 172.21.15.121:0/1288174866 >> 172.21.15.121:0/1288174866 conn(0x7f12440fa130 msgr2=0x7f12440fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:47.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 -- 172.21.15.121:0/1288174866 shutdown_connections 2023-07-13T22:06:47.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.264+0000 7f124a84e700 1 -- 172.21.15.121:0/1288174866 wait complete. 2023-07-13T22:06:47.269 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.265+0000 7f124a84e700 1 Processor -- start 2023-07-13T22:06:47.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.265+0000 7f124a84e700 1 -- start start 2023-07-13T22:06:47.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f124419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:47.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:47.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:47.270 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124419b5a0 con 0x7f122c0035c0 2023-07-13T22:06:47.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441a1870 con 0x7f1244195010 2023-07-13T22:06:47.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f124a84e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12441a19b0 con 0x7f124419af90 2023-07-13T22:06:47.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:47.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40540/0 (socket says 172.21.15.121:40540) 2023-07-13T22:06:47.271 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 -- 172.21.15.121:0/287423130 learned_addr learned my addr 172.21.15.121:0/287423130 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:06:47.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f1248deb700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:47.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f1243fff700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f124419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:47.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 msgr2=0x7f1244194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-13T22:06:47.272 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 msgr2=0x7f1244194ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-13T22:06:47.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:06:47.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f1248deb700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 msgr2=0x7f12441954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-13T22:06:47.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f12437fe700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:06:47.273 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.266+0000 7f1248deb700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 msgr2=0x7f12441954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-13T22:06:47.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1248deb700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-13T22:06:47.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1248deb700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-13T22:06:47.274 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 msgr2=0x7f1244194ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:06:47.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 msgr2=0x7f12441954b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:06:47.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.275 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 -- 172.21.15.121:0/287423130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123400f8b0 con 0x7f122c0035c0 2023-07-13T22:06:47.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f1243fff700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f124419aa50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1234015040 tx=0x7f1234012d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:47.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1234004130 con 0x7f122c0035c0 2023-07-13T22:06:47.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1234004290 con 0x7f122c0035c0 2023-07-13T22:06:47.276 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340278b0 con 0x7f122c0035c0 2023-07-13T22:06:47.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.267+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12441a1c30 con 0x7f122c0035c0 2023-07-13T22:06:47.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.268+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12441a2130 con 0x7f122c0035c0 2023-07-13T22:06:47.277 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.268+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1234024050 con 0x7f122c0035c0 2023-07-13T22:06:47.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.268+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244061b40 con 0x7f122c0035c0 2023-07-13T22:06:47.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.269+0000 7f12417fa700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 0x7f123003b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:47.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.269+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1234000e40 con 0x7f122c0035c0 2023-07-13T22:06:47.278 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.269+0000 7f12437fe700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 0x7f123003b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:47.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.269+0000 7f12437fe700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 0x7f123003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1238009180 tx=0x7f1238008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:47.279 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.274+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1234026730 con 0x7f122c0035c0 2023-07-13T22:06:47.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: mon.c calling monitor election 2023-07-13T22:06:47.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:47.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: mon.a calling monitor election 2023-07-13T22:06:47.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:47.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: fsmap 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: mgrmap e13: x(active, since 55s) 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-13T22:06:47.460 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: mon.b (rank 2) addr [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] is down (out of quorum) 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:47.549 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.553+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1244195fe0 con 0x7f122c0035c0 2023-07-13T22:06:47.550 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.554+0000 7f12417fa700 1 -- 172.21.15.121:0/287423130 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f123402b020 con 0x7f122c0035c0 2023-07-13T22:06:47.550 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-07-13T22:06:47.551 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":3,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","modified":"2023-07-13T22:06:41.907906Z","created":"2023-07-13T22:04:50.317415Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:3300","nonce":0},{"type":"v1","addr":"172.21.15.26:6789","nonce":0}]},"addr":"172.21.15.26:6789/0","public_addr":"172.21.15.26:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-13T22:06:47.553 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 msgr2=0x7f123003b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 0x7f123003b130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1238009180 tx=0x7f1238008040 comp rx=0 tx=0).stop 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 msgr2=0x7f124419aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f124419aa50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1234015040 tx=0x7f1234012d70 comp rx=0 tx=0).stop 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 shutdown_connections 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1230038c80 0x7f123003b130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.554 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f122c0035c0 0x7f124419aa50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1244195010 0x7f12441954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 --2- 172.21.15.121:0/287423130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f124419af90 0x7f1244194ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 >> 172.21.15.121:0/287423130 conn(0x7f12440fa130 msgr2=0x7f1244102150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 shutdown_connections 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:47.556+0000 7f124a84e700 1 -- 172.21.15.121:0/287423130 wait complete. 2023-07-13T22:06:47.555 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 3 2023-07-13T22:06:47.876 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-13T22:06:47.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph config generate-minimal-conf 2023-07-13T22:06:48.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:48.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:47 smithi026 ceph-mon[95647]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:48.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:47 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/287423130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:48.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:48.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:48.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:47 smithi121 ceph-mon[129187]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:48.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:47 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/287423130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:48.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:48.619 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:49.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: mon.c calling monitor election 2023-07-13T22:06:49.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:06:49.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: mon.a calling monitor election 2023-07-13T22:06:49.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:06:49.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:49.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:49.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-13T22:06:49.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:49.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: fsmap 2023-07-13T22:06:49.205 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:49.205 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: mgrmap e13: x(active, since 55s) 2023-07-13T22:06:49.205 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-07-13T22:06:49.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-07-13T22:06:49.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-07-13T22:06:49.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: mon.b (rank 2) addr [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] is down (out of quorum) 2023-07-13T22:06:49.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/287423130' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-13T22:06:49.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:49.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mon.b calling monitor election 2023-07-13T22:06:49.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi026:/etc/ceph/ceph.conf 2023-07-13T22:06:49.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi117:/etc/ceph/ceph.conf 2023-07-13T22:06:49.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:49.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mon.b calling monitor election 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mon.c calling monitor election 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mon.a calling monitor election 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: fsmap 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:49.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: mgrmap e13: x(active, since 57s) 2023-07-13T22:06:49.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-13T22:06:49.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Cluster is now healthy 2023-07-13T22:06:49.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: overall HEALTH_OK 2023-07-13T22:06:49.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:49.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:49.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.961 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: Updating smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:49.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:49.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:49.962 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mon.b calling monitor election 2023-07-13T22:06:50.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi026:/etc/ceph/ceph.conf 2023-07-13T22:06:50.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi117:/etc/ceph/ceph.conf 2023-07-13T22:06:50.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mon.b calling monitor election 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mon.c calling monitor election 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mon.a calling monitor election 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: fsmap 2023-07-13T22:06:50.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: mgrmap e13: x(active, since 57s) 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Cluster is now healthy 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: overall HEALTH_OK 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: Updating smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:50.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mon.b calling monitor election 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi026:/etc/ceph/ceph.conf 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi117:/etc/ceph/ceph.conf 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi121:/etc/ceph/ceph.conf 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mon.b calling monitor election 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mon.c calling monitor election 2023-07-13T22:06:50.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mon.a calling monitor election 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: monmap e3: 3 mons at {a=[v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0]} removed_ranks: {} 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: fsmap 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: osdmap e4: 0 total, 0 up, 0 in 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: mgrmap e13: x(active, since 57s) 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-07-13T22:06:50.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Cluster is now healthy 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: overall HEALTH_OK 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.253 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: Updating smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/config/ceph.conf 2023-07-13T22:06:50.254 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:50.254 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.254 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.254 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:50.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 -- 172.21.15.26:0/3984321921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 msgr2=0x7f55c8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:50.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/3984321921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c8072c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009910 tx=0x7f55b8009c20 comp rx=0 tx=0).stop 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 -- 172.21.15.26:0/3984321921 shutdown_connections 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/3984321921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c8072c80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 -- 172.21.15.26:0/3984321921 >> 172.21.15.26:0/3984321921 conn(0x7f55c80fb130 msgr2=0x7f55c80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 -- 172.21.15.26:0/3984321921 shutdown_connections 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.469+0000 7f55ccfb6700 1 -- 172.21.15.26:0/3984321921 wait complete. 2023-07-13T22:06:50.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.470+0000 7f55ccfb6700 1 Processor -- start 2023-07-13T22:06:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.470+0000 7f55ccfb6700 1 -- start start 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c819b7b0 0x7f55c819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c81a0160 0x7f55c81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c819be80 con 0x7f55c80728b0 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81a7ed0 con 0x7f55c819b7b0 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55ccfb6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c81a8010 con 0x7f55c81a0160 2023-07-13T22:06:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55c659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44612/0 (socket says 172.21.15.26:44612) 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55c659c700 1 -- 172.21.15.26:0/2327601088 learned_addr learned my addr 172.21.15.26:0/2327601088 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55c6d9d700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c81a0160 0x7f55c81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.471+0000 7f55c5d9b700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c819b7b0 0x7f55c819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 -- 172.21.15.26:0/2327601088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c81a0160 msgr2=0x7f55c81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c81a0160 0x7f55c81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 -- 172.21.15.26:0/2327601088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c819b7b0 msgr2=0x7f55c819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c819b7b0 0x7f55c819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 -- 172.21.15.26:0/2327601088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80095f0 con 0x7f55c80728b0 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55c659c700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55b8004cc0 tx=0x7f55b8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b801d070 con 0x7f55c80728b0 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55b800be20 con 0x7f55c80728b0 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800f460 con 0x7f55c80728b0 2023-07-13T22:06:50.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c81a8260 con 0x7f55c80728b0 2023-07-13T22:06:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.472+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c81a86b0 con 0x7f55c80728b0 2023-07-13T22:06:50.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.473+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f55b800b960 con 0x7f55c80728b0 2023-07-13T22:06:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.474+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c8061b40 con 0x7f55c80728b0 2023-07-13T22:06:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.474+0000 7f55b77fe700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 0x7f55b003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.474+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55b804c9d0 con 0x7f55c80728b0 2023-07-13T22:06:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.474+0000 7f55c5d9b700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 0x7f55b003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:50.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.475+0000 7f55c5d9b700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 0x7f55b003b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009850 tx=0x7f55bc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:50.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.480+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55b800ee10 con 0x7f55c80728b0 2023-07-13T22:06:50.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.696+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f55c81a8970 con 0x7f55c80728b0 2023-07-13T22:06:50.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.696+0000 7f55b77fe700 1 -- 172.21.15.26:0/2327601088 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f55b8027020 con 0x7f55c80728b0 2023-07-13T22:06:50.693 INFO:teuthology.orchestra.run.smithi026.stdout:# minimal ceph.conf for 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:50.693 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2023-07-13T22:06:50.693 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:06:50.694 INFO:teuthology.orchestra.run.smithi026.stdout: mon_host = [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2023-07-13T22:06:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.698+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 msgr2=0x7f55b003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.698+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 0x7f55b003b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009850 tx=0x7f55bc008040 comp rx=0 tx=0).stop 2023-07-13T22:06:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.698+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 msgr2=0x7f55c819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:50.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.698+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f55b8004cc0 tx=0x7f55b8004cf0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 shutdown_connections 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f55b0038c30 0x7f55b003b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c80728b0 0x7f55c819b270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55c81a0160 0x7f55c81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 --2- 172.21.15.26:0/2327601088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55c819b7b0 0x7f55c819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:50.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 >> 172.21.15.26:0/2327601088 conn(0x7f55c80fb130 msgr2=0x7f55c80fb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 shutdown_connections 2023-07-13T22:06:50.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:06:50.699+0000 7f55ccfb6700 1 -- 172.21.15.26:0/2327601088 wait complete. 2023-07-13T22:06:51.000 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: Reconfiguring mon.a (unknown last config time)... 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: Reconfiguring daemon mon.a on smithi026 2023-07-13T22:06:51.001 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2327601088' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.002 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:06:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:06:50.907+0000 7fb1c3c30700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-13T22:06:51.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:51.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: Reconfiguring mon.a (unknown last config time)... 2023-07-13T22:06:51.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:51.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:51.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: Reconfiguring daemon mon.a on smithi026 2023-07-13T22:06:51.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:50 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2327601088' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:06:51.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: Reconfiguring mon.a (unknown last config time)... 2023-07-13T22:06:51.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:51.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:51.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: Reconfiguring daemon mon.a on smithi026 2023-07-13T22:06:51.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:50 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2327601088' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:51.545 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-13T22:06:51.546 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:51.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.conf 2023-07-13T22:06:51.620 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:51.620 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:51.730 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:06:51.730 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2023-07-13T22:06:51.766 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:06:51.767 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:51.837 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:06:51.838 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.conf 2023-07-13T22:06:51.874 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:06:51.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-13T22:06:51.947 INFO:tasks.cephadm:Adding mgr.x on smithi026 2023-07-13T22:06:51.948 INFO:tasks.cephadm:Adding mgr.y on smithi117 2023-07-13T22:06:51.948 INFO:tasks.cephadm:Adding mgr.z on smithi121 2023-07-13T22:06:51.949 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch apply mgr '3;smithi026=x;smithi117=y;smithi121=z' 2023-07-13T22:06:52.535 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:06:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:52.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:52.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:52.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:52.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:52.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:52.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:53.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: Reconfiguring mon.b (monmap changed)... 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: Reconfiguring daemon mon.b on smithi117 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:53.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:53.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: Reconfiguring mon.b (monmap changed)... 2023-07-13T22:06:53.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: Reconfiguring daemon mon.b on smithi117 2023-07-13T22:06:53.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:53.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:53.747 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 -- 172.21.15.121:0/650085897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44500fdd90 msgr2=0x7f44500fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 --2- 172.21.15.121:0/650085897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44500fdd90 0x7f44500fe1a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f444000aa20 tx=0x7f444000ad30 comp rx=0 tx=0).stop 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 -- 172.21.15.121:0/650085897 shutdown_connections 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 --2- 172.21.15.121:0/650085897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44500fdd90 0x7f44500fe1a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 -- 172.21.15.121:0/650085897 >> 172.21.15.121:0/650085897 conn(0x7f44500f9320 msgr2=0x7f44500fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 -- 172.21.15.121:0/650085897 shutdown_connections 2023-07-13T22:06:53.748 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 -- 172.21.15.121:0/650085897 wait complete. 2023-07-13T22:06:53.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.751+0000 7f445657e700 1 Processor -- start 2023-07-13T22:06:53.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 -- start start 2023-07-13T22:06:53.750 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 0x7f4450194b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4450195510 0x7f445019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019ce20 con 0x7f4450195510 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019cf90 con 0x7f44501950a0 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f445657e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019d150 con 0x7f44500fdd90 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f444f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.121:43304/0 (socket says 172.21.15.121:43304) 2023-07-13T22:06:53.751 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f444f7fe700 1 -- 172.21.15.121:0/2841233292 learned_addr learned my addr 172.21.15.121:0/2841233292 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f444ffff700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 0x7f4450194b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.752+0000 7f4454b1b700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4450195510 0x7f445019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 -- 172.21.15.121:0/2841233292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 msgr2=0x7f445019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 -- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4450195510 msgr2=0x7f445019c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:53.752 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4450195510 0x7f445019c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 -- 172.21.15.121:0/2841233292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444000a700 con 0x7f44500fdd90 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444f7fe700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444ffff700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 0x7f4450194b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444001d070 con 0x7f44500fdd90 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44400056e0 con 0x7f44500fdd90 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445019d3d0 con 0x7f44500fdd90 2023-07-13T22:06:53.753 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000e450 con 0x7f44500fdd90 2023-07-13T22:06:53.754 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.753+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445019d940 con 0x7f44500fdd90 2023-07-13T22:06:53.754 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: Reconfiguring mon.b (monmap changed)... 2023-07-13T22:06:53.755 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: Reconfiguring daemon mon.b on smithi117 2023-07-13T22:06:53.755 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.755 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:53.755 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-13T22:06:53.756 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-13T22:06:53.756 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:53.757 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.754+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f444000e5b0 con 0x7f44500fdd90 2023-07-13T22:06:53.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.754+0000 7f444d7fa700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 0x7f443803b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:06:53.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.754+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4440012070 con 0x7f44500fdd90 2023-07-13T22:06:53.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.754+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445018c090 con 0x7f44500fdd90 2023-07-13T22:06:53.758 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.754+0000 7f444f7fe700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 0x7f443803b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:06:53.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.755+0000 7f444f7fe700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 0x7f443803b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4444009f70 tx=0x7f44440092c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:06:53.759 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.759+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f444002b440 con 0x7f44500fdd90 2023-07-13T22:06:53.967 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.971+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f4450060950 con 0x7f4438038c30 2023-07-13T22:06:53.974 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.978+0000 7f444d7fa700 1 -- 172.21.15.121:0/2841233292 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4450060950 con 0x7f4438038c30 2023-07-13T22:06:53.975 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mgr update... 2023-07-13T22:06:53.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 msgr2=0x7f443803b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 0x7f443803b0e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4444009f70 tx=0x7f44440092c0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 msgr2=0x7f4450194b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 0x7f4450194b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440004c00 comp rx=0 tx=0).stop 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 shutdown_connections 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4438038c30 0x7f443803b0e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4450195510 0x7f445019c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44500fdd90 0x7f4450194b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 --2- 172.21.15.121:0/2841233292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44501950a0 0x7f445019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:06:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 >> 172.21.15.121:0/2841233292 conn(0x7f44500f9320 msgr2=0x7f44500fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:06:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 shutdown_connections 2023-07-13T22:06:53.978 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:06:53.980+0000 7f445657e700 1 -- 172.21.15.121:0/2841233292 wait complete. 2023-07-13T22:06:54.582 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: Reconfiguring mon.c (monmap changed)... 2023-07-13T22:06:54.582 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: Reconfiguring daemon mon.c on smithi121 2023-07-13T22:06:54.583 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:54.583 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.583 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.583 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.676 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.y.service 2023-07-13T22:06:54.678 DEBUG:teuthology.orchestra.run.smithi121:mgr.z> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.z.service 2023-07-13T22:06:54.680 INFO:tasks.cephadm:Deploying OSDs... 2023-07-13T22:06:54.681 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:06:54.681 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2023-07-13T22:06:54.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: Reconfiguring mon.c (monmap changed)... 2023-07-13T22:06:54.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: Reconfiguring daemon mon.c on smithi121 2023-07-13T22:06:54.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:54.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: Reconfiguring mon.c (monmap changed)... 2023-07-13T22:06:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: Reconfiguring daemon mon.c on smithi121 2023-07-13T22:06:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:54.709 INFO:journalctl@ceph.mgr.z.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:06:54.712 INFO:journalctl@ceph.mgr.y.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:06:54.730 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-13T22:06:54.731 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2023-07-13T22:06:54.818 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-13T22:06:54.818 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 446 Links: 1 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-07-13 22:06:02.990237969 +0000 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-07-13 21:59:29.950967905 +0000 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-07-13 21:59:29.950967905 +0000 2023-07-13T22:06:54.819 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-07-13 21:59:29.926968290 +0000 2023-07-13T22:06:54.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-13T22:06:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-07-13T22:06:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-07-13T22:06:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000207373 s, 2.5 MB/s 2023-07-13T22:06:54.891 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-13T22:06:54.938 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2023-07-13T22:06:54.982 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-13T22:06:54.982 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:54.982 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-13T22:06:54.982 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:54.983 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:54.983 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-07-13 22:06:02.990237969 +0000 2023-07-13T22:06:54.983 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-07-13 21:59:30.825953853 +0000 2023-07-13T22:06:54.983 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-07-13 21:59:30.825953853 +0000 2023-07-13T22:06:54.983 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-07-13 21:59:30.803954206 +0000 2023-07-13T22:06:54.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-13T22:06:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-07-13T22:06:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-07-13T22:06:55.090 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000193956 s, 2.6 MB/s 2023-07-13T22:06:55.091 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-13T22:06:55.138 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2023-07-13T22:06:55.222 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-13T22:06:55.222 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:55.222 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-13T22:06:55.222 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:55.223 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:55.223 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-07-13 22:06:02.990237969 +0000 2023-07-13T22:06:55.223 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-07-13 21:59:31.711939625 +0000 2023-07-13T22:06:55.223 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-07-13 21:59:31.711939625 +0000 2023-07-13T22:06:55.223 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-07-13 21:59:31.689939978 +0000 2023-07-13T22:06:55.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-13T22:06:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-07-13T22:06:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-07-13T22:06:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000173579 s, 2.9 MB/s 2023-07-13T22:06:55.329 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-13T22:06:55.375 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2023-07-13T22:06:55.460 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-13T22:06:55.460 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:55.460 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-13T22:06:55.460 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:55.460 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:55.461 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-07-13 22:06:02.990237969 +0000 2023-07-13T22:06:55.461 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-07-13 21:59:32.598925380 +0000 2023-07-13T22:06:55.461 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-07-13 21:59:32.598925380 +0000 2023-07-13T22:06:55.461 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-07-13 21:59:32.576925734 +0000 2023-07-13T22:06:55.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-13T22:06:55.565 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-07-13T22:06:55.566 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-07-13T22:06:55.567 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.00019369 s, 2.6 MB/s 2023-07-13T22:06:55.568 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-13T22:06:55.614 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-07-13T22:06:55.614 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2023-07-13T22:06:55.636 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-13T22:06:55.637 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2023-07-13T22:06:55.664 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:55.665 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: Saving service mgr spec with placement smithi026=x;smithi117=y;smithi121=z;count:3 2023-07-13T22:06:55.665 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:55.665 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.665 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:55.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:55.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:55.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:55.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:55.667 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.667 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:55 smithi121 ceph-mon[129187]: Deploying daemon mgr.z on smithi121 2023-07-13T22:06:55.696 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 379484 Links: 1 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-07-13 22:06:41.318775917 +0000 2023-07-13T22:06:55.697 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-07-13 21:55:57.680527793 +0000 2023-07-13T22:06:55.698 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-07-13 21:55:57.680527793 +0000 2023-07-13T22:06:55.698 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-07-13T22:06:55.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-13T22:06:55.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:55.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: Saving service mgr spec with placement smithi026=x;smithi117=y;smithi121=z;count:3 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:55.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:55.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:55 smithi117 ceph-mon[129074]: Deploying daemon mgr.z on smithi121 2023-07-13T22:06:55.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi026=x;smithi117=y;smithi121=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: Saving service mgr spec with placement smithi026=x;smithi117=y;smithi121=z;count:3 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:55.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:55.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:55.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:55.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:55 smithi026 ceph-mon[95647]: Deploying daemon mgr.z on smithi121 2023-07-13T22:06:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-07-13T22:06:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-07-13T22:06:55.729 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000214501 s, 2.4 MB/s 2023-07-13T22:06:55.730 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-13T22:06:55.792 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2023-07-13T22:06:55.852 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-13T22:06:55.853 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:55.853 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 372732 Links: 1 2023-07-13T22:06:55.853 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:55.853 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:55.854 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-07-13 22:06:41.318775917 +0000 2023-07-13T22:06:55.854 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-07-13 21:55:58.287515262 +0000 2023-07-13T22:06:55.854 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-07-13 21:55:58.287515262 +0000 2023-07-13T22:06:55.854 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-07-13T22:06:55.855 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-13T22:06:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-07-13T22:06:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-07-13T22:06:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000223827 s, 2.3 MB/s 2023-07-13T22:06:55.921 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-13T22:06:55.981 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2023-07-13T22:06:56.000 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:55 smithi121 systemd[1]: Starting Ceph mgr.z for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 376749 Links: 1 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.039 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-07-13 22:06:41.318775917 +0000 2023-07-13T22:06:56.040 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-07-13 21:55:58.840503846 +0000 2023-07-13T22:06:56.040 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-07-13 21:55:58.840503846 +0000 2023-07-13T22:06:56.040 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-07-13T22:06:56.040 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-13T22:06:56.105 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-07-13T22:06:56.105 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-07-13T22:06:56.105 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000238805 s, 2.1 MB/s 2023-07-13T22:06:56.106 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-13T22:06:56.166 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2023-07-13T22:06:56.224 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-13T22:06:56.224 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 6h/6d Inode: 384145 Links: 1 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-07-13 22:06:41.318775917 +0000 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-07-13 21:55:59.471490820 +0000 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-07-13 21:55:59.471490820 +0000 2023-07-13T22:06:56.225 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2023-07-13T22:06:56.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-13T22:06:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-07-13T22:06:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-07-13T22:06:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000228579 s, 2.2 MB/s 2023-07-13T22:06:56.292 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-13T22:06:56.351 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-07-13T22:06:56.351 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2023-07-13T22:06:56.370 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-13T22:06:56.370 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2023-07-13T22:06:56.393 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 podman[131016]: 2023-07-13T22:06:56.411 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 381032 Links: 1 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-13 22:06:55.839675310 +0000 2023-07-13T22:06:56.412 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-13 21:55:47.424337370 +0000 2023-07-13T22:06:56.413 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-13 21:55:47.424337370 +0000 2023-07-13T22:06:56.413 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-13T22:06:56.413 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-13T22:06:56.485 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-13T22:06:56.486 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-13T22:06:56.486 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000226259 s, 2.3 MB/s 2023-07-13T22:06:56.488 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-13T22:06:56.546 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 381112 Links: 1 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.605 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-13 22:06:55.839675310 +0000 2023-07-13T22:06:56.606 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-13 21:55:48.001329622 +0000 2023-07-13T22:06:56.606 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-13 21:55:48.001329622 +0000 2023-07-13T22:06:56.606 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-13T22:06:56.606 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-13T22:06:56.670 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-mon[129187]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:56.671 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 bash[131016]: 5756aa9662a6f81efb6c546aa483c3d2b9a1d9c0157ceebe9385132f303148f0 2023-07-13T22:06:56.671 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-mgr[131075]: -- 172.21.15.121:0/809827460 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558d10af31e0 con 0x558d10ac9000 2023-07-13T22:06:56.671 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 systemd[1]: Started Ceph mgr.z for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:06:56.671 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:56.527+0000 7fb30d951200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:06:56.671 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:56.614+0000 7fb30d951200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:06:56.673 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-13T22:06:56.673 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-13T22:06:56.674 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.00020608 s, 2.5 MB/s 2023-07-13T22:06:56.675 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-13T22:06:56.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:56 smithi117 ceph-mon[129074]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:56.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:56 smithi026 ceph-mon[95647]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:56.733 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2023-07-13T22:06:56.790 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-13T22:06:56.790 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.790 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 379298 Links: 1 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-13 22:06:55.839675310 +0000 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-13 21:55:48.559322129 +0000 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-13 21:55:48.559322129 +0000 2023-07-13T22:06:56.791 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-13T22:06:56.792 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-13T22:06:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-13T22:06:56.856 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-13T22:06:56.857 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000181331 s, 2.8 MB/s 2023-07-13T22:06:56.858 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-13T22:06:56.915 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2023-07-13T22:06:56.971 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-13T22:06:56.972 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-13T22:06:56.972 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 379378 Links: 1 2023-07-13T22:06:56.972 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:06:56.973 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-07-13T22:06:56.973 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-07-13 22:06:55.839675310 +0000 2023-07-13T22:06:56.973 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-07-13 21:55:49.112314702 +0000 2023-07-13T22:06:56.973 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-07-13 21:55:49.112314702 +0000 2023-07-13T22:06:56.973 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-07-13T22:06:56.973 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-13T22:06:56.979 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:56.709+0000 7fb30d951200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:06:56.979 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:56.806+0000 7fb30d951200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:06:56.980 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:56.982+0000 7fb30d951200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:06:57.010 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-07-13T22:06:57.010 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-07-13T22:06:57.010 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.0002269 s, 2.3 MB/s 2023-07-13T22:06:57.010 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-13T22:06:57.067 INFO:tasks.cephadm:Deploying osd.0 on smithi026 with /dev/vg_nvme/lv_4... 2023-07-13T22:06:57.068 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-13T22:06:57.607 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.607 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.607 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-mon[129187]: Deploying daemon mgr.y on smithi117 2023-07-13T22:06:57.608 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:57.295+0000 7fb30d951200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:06:57.609 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:57.383+0000 7fb30d951200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:06:57.609 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:57.515+0000 7fb30d951200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:06:57.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:57.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:57.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:57.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:57.796 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:57 smithi117 ceph-mon[129074]: Deploying daemon mgr.y on smithi117 2023-07-13T22:06:57.808 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:06:57.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:06:57.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:57 smithi026 ceph-mon[95647]: Deploying daemon mgr.y on smithi117 2023-07-13T22:06:58.000 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:57.610+0000 7fb30d951200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:06:58.200 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 systemd[1]: Starting Ceph mgr.y for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:06:58.500 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:58.025+0000 7fb30d951200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:06:58.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 podman[130509]: 2023-07-13T22:06:58.588 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:58 smithi026 ceph-mon[95647]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:58.873 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-mon[129074]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:58.873 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 bash[130509]: b92160516541e81f6fa5fc46eb3be8377ffcba9226cda11570b09f48fb84d99d 2023-07-13T22:06:58.873 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-mgr[130550]: -- 172.21.15.117:0/971827300 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b3276cf1e0 con 0x55b3276a4800 2023-07-13T22:06:58.873 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 systemd[1]: Started Ceph mgr.y for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:06:58.874 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:58.707+0000 7f96912a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-13T22:06:58.874 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:58.785+0000 7f96912a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-13T22:06:58.874 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:58.877+0000 7f96912a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-13T22:06:59.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:58 smithi121 ceph-mon[129187]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:59.000 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:58.594+0000 7fb30d951200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:06:59.000 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:58.763+0000 7fb30d951200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:06:59.148 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:58.976+0000 7f96912a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-13T22:06:59.148 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:59.151+0000 7f96912a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-13T22:06:59.712 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-13T22:06:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-13T22:06:59.713 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-07-13T22:06:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-07-13T22:06:59.714 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.031769 s, 330 MB/s 2023-07-13T22:06:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-07-13T22:06:59.804 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:59.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-mon[129074]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:59.806 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:59.476+0000 7f96912a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-13T22:06:59.806 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:59.571+0000 7f96912a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-13T22:06:59.806 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:59.704+0000 7f96912a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-13T22:06:59.844 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:59.576+0000 7fb30d951200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:06:59.844 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:59.667+0000 7fb30d951200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:59.845 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-mon[129187]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:06:59.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:06:59.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:06:59.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:06:59 smithi026 ceph-mon[95647]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:00.137 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:06:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:06:59.847+0000 7fb30d951200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:07:00.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi026:vg_nvme/lv_4 2023-07-13T22:07:00.199 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:06:59 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:06:59.808+0000 7f96912a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-13T22:07:00.500 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:00.140+0000 7fb30d951200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:07:00.500 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:00.373+0000 7fb30d951200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:07:00.699 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:00 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:00.252+0000 7f96912a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-13T22:07:00.784 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:00.525+0000 7fb30d951200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:07:00.941 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:07:01.199 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:00 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:00.843+0000 7f96912a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-13T22:07:01.200 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:01.012+0000 7f96912a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-13T22:07:01.250 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:00.788+0000 7fb30d951200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:07:01.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.675 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:01.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: Reconfiguring mgr.x (unknown last config time)... 2023-07-13T22:07:01.677 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:07:01.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:07:01.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: Reconfiguring daemon mgr.x on smithi026 2023-07-13T22:07:01.679 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: Reconfiguring mgr.x (unknown last config time)... 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:07:01.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: Reconfiguring daemon mgr.x on smithi026 2023-07-13T22:07:01.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.750 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:01.481+0000 7fb30d951200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:07:01.750 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:01.610+0000 7fb30d951200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:07:01.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:01.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: Reconfiguring mgr.x (unknown last config time)... 2023-07-13T22:07:01.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-13T22:07:01.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-13T22:07:01.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:01.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: Reconfiguring daemon mgr.x on smithi026 2023-07-13T22:07:01.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:02.025 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:01.769+0000 7f96912a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:07:02.025 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:01 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:01.859+0000 7f96912a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-13T22:07:02.122 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:01.799+0000 7fb30d951200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:07:02.122 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:01.913+0000 7fb30d951200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:07:02.471 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:02.125+0000 7fb30d951200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:07:02.563 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:02 smithi026 ceph-mon[95647]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:02.670 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-mon[129074]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:02.670 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:02.029+0000 7f96912a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-13T22:07:02.671 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:02.304+0000 7f96912a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-13T22:07:02.671 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:02.527+0000 7f96912a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-13T22:07:02.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:02 smithi121 ceph-mon[129187]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:02.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 -- 172.21.15.26:0/3717233819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8101720 msgr2=0x7f02b8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3717233819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8101720 0x7f02b8107fe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b0a0 tx=0x7f02b000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 -- 172.21.15.26:0/3717233819 shutdown_connections 2023-07-13T22:07:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3717233819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b80ffb90 0x7f02b80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3717233819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8101720 0x7f02b8107fe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3717233819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b8100d90 0x7f02b81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 -- 172.21.15.26:0/3717233819 >> 172.21.15.26:0/3717233819 conn(0x7f02b80fb1a0 msgr2=0x7f02b80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:07:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 -- 172.21.15.26:0/3717233819 shutdown_connections 2023-07-13T22:07:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.981+0000 7f02bdd80700 1 -- 172.21.15.26:0/3717233819 wait complete. 2023-07-13T22:07:02.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.982+0000 7f02bdd80700 1 Processor -- start 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.982+0000 7f02bdd80700 1 -- start start 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 0x7f02b8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b8101720 0x7f02b819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b819f790 con 0x7f02b8101720 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b819f8d0 con 0x7f02b80ffb90 2023-07-13T22:07:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02bdd80700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b819fa90 con 0x7f02b8100d90 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 0x7f02b8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 0x7f02b8197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60018/0 (socket says 172.21.15.26:60018) 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57220/0 (socket says 172.21.15.26:57220) 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b77fe700 1 -- 172.21.15.26:0/3033331314 learned_addr learned my addr 172.21.15.26:0/3033331314 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:07:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b7fff700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b8101720 0x7f02b819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 -- 172.21.15.26:0/3033331314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 msgr2=0x7f02b8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 0x7f02b8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 -- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b8101720 msgr2=0x7f02b819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b8101720 0x7f02b819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 -- 172.21.15.26:0/3033331314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b0009d00 con 0x7f02b8100d90 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.983+0000 7f02b6ffd700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e9f0 tx=0x7f02ac00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.984+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00c9c0 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.984+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02ac010430 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.984+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac018680 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.984+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b819fd70 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.984+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b81a01c0 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.985+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b810b420 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.985+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f02ac00cb20 con 0x7f02b8100d90 2023-07-13T22:07:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.985+0000 7f02b4ff9700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 0x7f02a003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.985+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02ac014070 con 0x7f02b8100d90 2023-07-13T22:07:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.986+0000 7f02b77fe700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 0x7f02a003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.986+0000 7f02b77fe700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 0x7f02a003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02b8199120 tx=0x7f02a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:02.990+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f02ac016330 con 0x7f02b8100d90 2023-07-13T22:07:03.200 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:02.674+0000 7f96912a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-13T22:07:03.200 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:02.934+0000 7f96912a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-13T22:07:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:03.206+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f02b8107d60 con 0x7f02a00389e0 2023-07-13T22:07:03.250 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:07:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z[131032]: 2023-07-13T22:07:02.985+0000 7fb30d951200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:07:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:03.789+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f02ac010860 con 0x7f02b8100d90 2023-07-13T22:07:03.942 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:03.626+0000 7f96912a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-13T22:07:03.943 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:03.758+0000 7f96912a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-13T22:07:03.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:03.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:03.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:03.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:03.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:03.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: Standby manager daemon z started 2023-07-13T22:07:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-13T22:07:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-13T22:07:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:03.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:03.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:03.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:03.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:03.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-mon[129074]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:04.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:04.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:04.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:04.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: Standby manager daemon z started 2023-07-13T22:07:04.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-13T22:07:04.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:04.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-13T22:07:04.002 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:04.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:04.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:04.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:04.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:04.003 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:03 smithi121 ceph-mon[129187]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:04.199 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:03 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:03.947+0000 7f96912a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-13T22:07:04.200 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:04 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:04.059+0000 7f96912a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:04.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: Standby manager daemon z started 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.121:0/1077189722' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:04.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:04.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:04.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:04.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:03 smithi026 ceph-mon[95647]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:04.699 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:04 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:04.272+0000 7f96912a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-13T22:07:04.818 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:04 smithi026 ceph-mon[95647]: mgrmap e14: x(active, since 72s), standbys: z 2023-07-13T22:07:04.818 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-13T22:07:05.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:04 smithi117 ceph-mon[129074]: mgrmap e14: x(active, since 72s), standbys: z 2023-07-13T22:07:05.114 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-13T22:07:05.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:04 smithi121 ceph-mon[129187]: mgrmap e14: x(active, since 72s), standbys: z 2023-07-13T22:07:05.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-13T22:07:05.449 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y[130525]: 2023-07-13T22:07:05.118+0000 7f96912a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-13T22:07:05.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:05.799+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mon.1 v2:172.21.15.121:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f02ac01d210 con 0x7f02b8100d90 2023-07-13T22:07:06.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: Standby manager daemon y started 2023-07-13T22:07:06.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-13T22:07:06.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:06.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-13T22:07:06.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:06.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:05 smithi117 ceph-mon[129074]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:06.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: Standby manager daemon y started 2023-07-13T22:07:06.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-13T22:07:06.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:06.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-13T22:07:06.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:06.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:05 smithi026 ceph-mon[95647]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:06.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: Standby manager daemon y started 2023-07-13T22:07:06.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-13T22:07:06.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-13T22:07:06.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-13T22:07:06.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: from='mgr.? 172.21.15.117:0/1242053001' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-13T22:07:06.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:05 smithi121 ceph-mon[129187]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:07.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-07-13T22:07:07.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-13T22:07:07.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]: dispatch 2023-07-13T22:07:07.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]': finished 2023-07-13T22:07:07.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: osdmap e5: 1 total, 0 up, 1 in 2023-07-13T22:07:07.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:07.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-07-13T22:07:07.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-13T22:07:07.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]: dispatch 2023-07-13T22:07:07.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]': finished 2023-07-13T22:07:07.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: osdmap e5: 1 total, 0 up, 1 in 2023-07-13T22:07:07.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:07.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-07-13T22:07:07.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-13T22:07:07.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]: dispatch 2023-07-13T22:07:07.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3353754543' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b"}]': finished 2023-07-13T22:07:07.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: osdmap e5: 1 total, 0 up, 1 in 2023-07-13T22:07:07.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:08.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:07 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/142086638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:08.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:07 smithi117 ceph-mon[129074]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:08.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/142086638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:08.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:07 smithi026 ceph-mon[95647]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:07 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/142086638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:07 smithi121 ceph-mon[129187]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:10.152 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:09 smithi026 ceph-mon[95647]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:10.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:09 smithi117 ceph-mon[129074]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:10.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:09 smithi121 ceph-mon[129187]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:12.585 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:12 smithi026 ceph-mon[95647]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:12.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:12 smithi117 ceph-mon[129074]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:12.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:12 smithi121 ceph-mon[129187]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:14.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:14 smithi117 ceph-mon[129074]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:14.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-13T22:07:14.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:14 smithi026 ceph-mon[95647]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-13T22:07:14.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:14.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:14 smithi121 ceph-mon[129187]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:14.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-13T22:07:14.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:15 smithi117 ceph-mon[129074]: Deploying daemon osd.0 on smithi026 2023-07-13T22:07:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:15 smithi026 ceph-mon[95647]: Deploying daemon osd.0 on smithi026 2023-07-13T22:07:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:15 smithi121 ceph-mon[129187]: Deploying daemon osd.0 on smithi026 2023-07-13T22:07:16.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:16 smithi117 ceph-mon[129074]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:16.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:16 smithi026 ceph-mon[95647]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:16.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:16 smithi121 ceph-mon[129187]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:18.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:18 smithi117 ceph-mon[129074]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:18.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:18 smithi026 ceph-mon[95647]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:18.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:18 smithi121 ceph-mon[129187]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:19.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:19 smithi026 ceph-mon[95647]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:20.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:19 smithi117 ceph-mon[129074]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:20.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:19 smithi121 ceph-mon[129187]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:21.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:21.305 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.306 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:21.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:21.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:22.171 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:22 smithi026 ceph-mon[95647]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:22.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:22 smithi117 ceph-mon[129074]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:22.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:22 smithi121 ceph-mon[129187]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:23.366 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 0 on host 'smithi026' 2023-07-13T22:07:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.370+0000 7f02b4ff9700 1 -- 172.21.15.26:0/3033331314 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f02b8107d60 con 0x7f02a00389e0 2023-07-13T22:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 msgr2=0x7f02a003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 0x7f02a003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02b8199120 tx=0x7f02a8009260 comp rx=0 tx=0).stop 2023-07-13T22:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 msgr2=0x7f02b8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00e9f0 tx=0x7f02ac00ed00 comp rx=0 tx=0).stop 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 shutdown_connections 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f02a00389e0 0x7f02a003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02b8101720 0x7f02b819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02b80ffb90 0x7f02b8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 --2- 172.21.15.26:0/3033331314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02b8100d90 0x7f02b8198180 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.372+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 >> 172.21.15.26:0/3033331314 conn(0x7f02b80fb1a0 msgr2=0x7f02b8106640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.373+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 shutdown_connections 2023-07-13T22:07:23.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:07:23.373+0000 7f02bdd80700 1 -- 172.21.15.26:0/3033331314 wait complete. 2023-07-13T22:07:23.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:23.779 DEBUG:teuthology.orchestra.run.smithi026:osd.0> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.0.service 2023-07-13T22:07:23.784 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_4... 2023-07-13T22:07:23.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-13T22:07:23.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:23.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:23.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:23.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:23.863 INFO:journalctl@ceph.osd.0.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:07:24.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:24.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:24.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:24.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:24.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:24.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:24.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:24.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:24.207 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:07:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:07:23.998+0000 7f13a7aba640 -1 osd.0 0 log_to_monitors true 2023-07-13T22:07:24.345 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:07:24.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:24 smithi117 ceph-mon[129074]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:24.821 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:24 smithi117 ceph-mon[129074]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-13T22:07:24.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:24 smithi026 ceph-mon[95647]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:24.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:24 smithi026 ceph-mon[95647]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-13T22:07:25.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:24 smithi121 ceph-mon[129187]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:25.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:24 smithi121 ceph-mon[129187]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-13T22:07:25.864 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-13T22:07:25.864 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-13T22:07:25.864 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2023-07-13T22:07:25.864 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2023-07-13T22:07:25.864 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147745 s, 710 MB/s 2023-07-13T22:07:25.865 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2023-07-13T22:07:26.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:25 smithi117 ceph-mon[129074]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-13T22:07:26.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:25 smithi117 ceph-mon[129074]: osdmap e6: 1 total, 0 up, 1 in 2023-07-13T22:07:26.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:25 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:26.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:25 smithi117 ceph-mon[129074]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:07:26.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:25 smithi117 ceph-mon[129074]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:26.207 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:07:25.916+0000 7f139c931700 -1 osd.0 0 waiting for initial osdmap 2023-07-13T22:07:26.208 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:07:25.924+0000 7f1395720700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:07:26.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-mon[95647]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-13T22:07:26.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-mon[95647]: osdmap e6: 1 total, 0 up, 1 in 2023-07-13T22:07:26.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:26.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-mon[95647]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:07:26.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:25 smithi026 ceph-mon[95647]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:26.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:25 smithi121 ceph-mon[129187]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-13T22:07:26.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:25 smithi121 ceph-mon[129187]: osdmap e6: 1 total, 0 up, 1 in 2023-07-13T22:07:26.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:25 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:26.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:25 smithi121 ceph-mon[129187]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:07:26.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:25 smithi121 ceph-mon[129187]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:26.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2023-07-13T22:07:26.888 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:07:27.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:07:27.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: osdmap e7: 1 total, 0 up, 1 in 2023-07-13T22:07:27.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi026 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi026 to 17272M 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:27.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:27.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:07:27.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: osdmap e7: 1 total, 0 up, 1 in 2023-07-13T22:07:27.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi026 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi026 to 17272M 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:27.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:27.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:07:27.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: osdmap e7: 1 total, 0 up, 1 in 2023-07-13T22:07:27.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:27.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi026 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi026 to 17272M 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:27.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1790888598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80fea80 msgr2=0x7fafd80fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1790888598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80fea80 0x7fafd80fee90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009990 tx=0x7fafc8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.032 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1790888598 shutdown_connections 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1790888598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 0x7fafd8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1790888598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80fea80 0x7fafd80fee90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1790888598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80ffc80 0x7fafd81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1790888598 >> 172.21.15.117:0/1790888598 conn(0x7fafd80fa0b0 msgr2=0x7fafd80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1790888598 shutdown_connections 2023-07-13T22:07:28.033 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.036+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1790888598 wait complete. 2023-07-13T22:07:28.035 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 Processor -- start 2023-07-13T22:07:28.035 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 -- start start 2023-07-13T22:07:28.035 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80fea80 0x7fafd8194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:28.035 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:28.036 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 0x7fafd819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:28.036 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819c3c0 con 0x7fafd80fea80 2023-07-13T22:07:28.036 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819c500 con 0x7fafd80ffc80 2023-07-13T22:07:28.036 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.037+0000 7fafdf9a9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819c690 con 0x7fafd8100610 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59054/0 (socket says 172.21.15.117:59054) 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 -- 172.21.15.117:0/1279880284 learned_addr learned my addr 172.21.15.117:0/1279880284 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdd745700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80fea80 0x7fafd8194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafddf46700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 0x7fafd819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 -- 172.21.15.117:0/1279880284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 msgr2=0x7fafd819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:28.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 0x7fafd819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 -- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80fea80 msgr2=0x7fafd8194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80fea80 0x7fafd8194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 -- 172.21.15.117:0/1279880284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80095f0 con 0x7fafd80ffc80 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdcf44700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d9e0 tx=0x7fafd400dda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400ecd0 con 0x7fafd80ffc80 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd8068030 con 0x7fafd80ffc80 2023-07-13T22:07:28.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd400ee30 con 0x7fafd80ffc80 2023-07-13T22:07:28.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4014800 con 0x7fafd80ffc80 2023-07-13T22:07:28.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.038+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd8068530 con 0x7fafd80ffc80 2023-07-13T22:07:28.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.039+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd810a2e0 con 0x7fafd80ffc80 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.040+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fafd4016440 con 0x7fafd80ffc80 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.041+0000 7fafce7fc700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 0x7fafc40a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.041+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fafd40cd3e0 con 0x7fafd80ffc80 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.041+0000 7fafdd745700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 0x7fafc40a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.042+0000 7fafdd745700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 0x7fafc40a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:28.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.044+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fafd4098590 con 0x7fafd80ffc80 2023-07-13T22:07:28.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:07:28.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:07:28.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] boot 2023-07-13T22:07:28.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: osdmap e8: 1 total, 1 up, 1 in 2023-07-13T22:07:28.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:28.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:27 smithi117 ceph-mon[129074]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:28.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:07:28.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:07:28.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] boot 2023-07-13T22:07:28.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: osdmap e8: 1 total, 1 up, 1 in 2023-07-13T22:07:28.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:28.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:27 smithi026 ceph-mon[95647]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:28.232 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:28.237+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fafd80687b0 con 0x7fafc40a1140 2023-07-13T22:07:28.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:07:28.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:07:28.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: osd.0 [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] boot 2023-07-13T22:07:28.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: osdmap e8: 1 total, 1 up, 1 in 2023-07-13T22:07:28.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-13T22:07:28.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:27 smithi121 ceph-mon[129187]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-13T22:07:29.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:28 smithi117 ceph-mon[129074]: osdmap e9: 1 total, 1 up, 1 in 2023-07-13T22:07:29.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:28 smithi117 ceph-mon[129074]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:29.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:29.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:29.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:28 smithi026 ceph-mon[95647]: osdmap e9: 1 total, 1 up, 1 in 2023-07-13T22:07:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:28 smithi026 ceph-mon[95647]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:29.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:29.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:29.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:28 smithi121 ceph-mon[129187]: osdmap e9: 1 total, 1 up, 1 in 2023-07-13T22:07:29.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:28 smithi121 ceph-mon[129187]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:29.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:29.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:29.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:30.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:29 smithi117 ceph-mon[129074]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:30.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:29 smithi026 ceph-mon[95647]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:30.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:29 smithi121 ceph-mon[129187]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:31.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:30 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/1639497462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:30 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:30 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]': finished 2023-07-13T22:07:31.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:30 smithi117 ceph-mon[129074]: osdmap e10: 2 total, 1 up, 2 in 2023-07-13T22:07:31.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:31.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:30 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/1639497462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:30 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:30 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]': finished 2023-07-13T22:07:31.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:30 smithi026 ceph-mon[95647]: osdmap e10: 2 total, 1 up, 2 in 2023-07-13T22:07:31.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:31.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:30 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/1639497462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:30 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]: dispatch 2023-07-13T22:07:31.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:30 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b5fb060a-61be-4007-809f-69c2e19ed034"}]': finished 2023-07-13T22:07:31.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:30 smithi121 ceph-mon[129187]: osdmap e10: 2 total, 1 up, 2 in 2023-07-13T22:07:31.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:32.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:31 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/2260703950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:32.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:31 smithi117 ceph-mon[129074]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:32.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:31 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/2260703950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:32.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:31 smithi026 ceph-mon[95647]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:32.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:31 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/2260703950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:32.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:31 smithi121 ceph-mon[129187]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:34.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:34 smithi117 ceph-mon[129074]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:34.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:34 smithi026 ceph-mon[95647]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:34.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:34 smithi121 ceph-mon[129187]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:36.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:36 smithi117 ceph-mon[129074]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:36.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:36 smithi026 ceph-mon[95647]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:36.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:36 smithi121 ceph-mon[129187]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:37.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-13T22:07:37.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:37.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-13T22:07:37.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-13T22:07:37.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:38.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:38 smithi117 ceph-mon[129074]: Deploying daemon osd.1 on smithi117 2023-07-13T22:07:38.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:38 smithi117 ceph-mon[129074]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:38.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:38 smithi026 ceph-mon[95647]: Deploying daemon osd.1 on smithi117 2023-07-13T22:07:38.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:38 smithi026 ceph-mon[95647]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:38.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:38 smithi121 ceph-mon[129187]: Deploying daemon osd.1 on smithi117 2023-07-13T22:07:38.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:38 smithi121 ceph-mon[129187]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:40.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:39 smithi117 ceph-mon[129074]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:40.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:39 smithi026 ceph-mon[95647]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:40.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:39 smithi121 ceph-mon[129187]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:42.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:42 smithi117 ceph-mon[129074]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:42.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:42.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:42.608 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:42.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:42 smithi026 ceph-mon[95647]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:42.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:42.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:42.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:42.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:42 smithi121 ceph-mon[129187]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:43.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:43.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:43.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:44.398 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.403+0000 7fafce7fc700 1 -- 172.21.15.117:0/1279880284 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fafd80687b0 con 0x7fafc40a1140 2023-07-13T22:07:44.398 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2023-07-13T22:07:44.400 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 msgr2=0x7fafc40a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:44.401 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 0x7fafc40a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fafc8006010 tx=0x7fafc800b540 comp rx=0 tx=0).stop 2023-07-13T22:07:44.401 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 msgr2=0x7fafd8194d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:44.401 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafd400d9e0 tx=0x7fafd400dda0 comp rx=0 tx=0).stop 2023-07-13T22:07:44.402 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 shutdown_connections 2023-07-13T22:07:44.402 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fafc40a1140 0x7fafc40a35f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:44.403 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafd80fea80 0x7fafd8194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:44.403 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafd8100610 0x7fafd819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:44.403 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 --2- 172.21.15.117:0/1279880284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fafd80ffc80 0x7fafd8194d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:44.403 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 >> 172.21.15.117:0/1279880284 conn(0x7fafd80fa0b0 msgr2=0x7fafd80fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:07:44.404 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 shutdown_connections 2023-07-13T22:07:44.404 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:07:44.405+0000 7fafdf9a9700 1 -- 172.21.15.117:0/1279880284 wait complete. 2023-07-13T22:07:44.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:44.767 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.1.service 2023-07-13T22:07:44.769 INFO:tasks.cephadm:Deploying osd.2 on smithi121 with /dev/vg_nvme/lv_4... 2023-07-13T22:07:44.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-13T22:07:44.797 INFO:journalctl@ceph.osd.1.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:07:44.800 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:44.800 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.801 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:44.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:44.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:44.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:44.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:44.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:07:45.329 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:07:45.591 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:45 smithi117 ceph-mon[129074]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:45.591 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:45 smithi117 ceph-mon[129074]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:45.591 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:07:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:07:45.463+0000 7f7c7b652640 -1 osd.1 0 log_to_monitors true 2023-07-13T22:07:45.816 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:45 smithi121 ceph-mon[129187]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:45.817 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:45 smithi121 ceph-mon[129187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:45.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:45 smithi026 ceph-mon[95647]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:45.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:45 smithi026 ceph-mon[95647]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-13T22:07:46.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:46.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-13T22:07:46.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: osdmap e11: 2 total, 1 up, 2 in 2023-07-13T22:07:46.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:46.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:46.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:46 smithi121 ceph-mon[129187]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:46.782 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-13T22:07:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-13T22:07:46.783 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2023-07-13T22:07:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2023-07-13T22:07:46.783 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145844 s, 719 MB/s 2023-07-13T22:07:46.783 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2023-07-13T22:07:46.949 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:07:46.529+0000 7f7c704c9700 -1 osd.1 0 waiting for initial osdmap 2023-07-13T22:07:46.949 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:07:46.533+0000 7f7c6aabb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: osdmap e11: 2 total, 1 up, 2 in 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:46.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:46 smithi117 ceph-mon[129074]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:46.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:46.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-13T22:07:46.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: osdmap e11: 2 total, 1 up, 2 in 2023-07-13T22:07:46.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:46.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: from='osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:46.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:46 smithi026 ceph-mon[95647]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:07:47.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi121:vg_nvme/lv_4 2023-07-13T22:07:47.763 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:07:47.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:07:47.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: osdmap e12: 2 total, 1 up, 2 in 2023-07-13T22:07:47.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:47.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:47.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:07:47.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: osdmap e12: 2 total, 1 up, 2 in 2023-07-13T22:07:47.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:47.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: osdmap e12: 2 total, 1 up, 2 in 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.964 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.965 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:07:47.965 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:47.965 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:47.965 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:07:47.965 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:07:48.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:07:48.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:07:48.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi117 2023-07-13T22:07:48.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi117 to 17168M 2023-07-13T22:07:48.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:48.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] boot 2023-07-13T22:07:48.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: osdmap e13: 2 total, 2 up, 2 in 2023-07-13T22:07:48.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:48.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 -- 172.21.15.121:0/1123594204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0feb00 msgr2=0x7f335c0fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:48.937 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 --2- 172.21.15.121:0/1123594204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0feb00 0x7f335c0fef10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3344009990 tx=0x7f3344009ca0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 -- 172.21.15.121:0/1123594204 shutdown_connections 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 --2- 172.21.15.121:0/1123594204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 --2- 172.21.15.121:0/1123594204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0feb00 0x7f335c0fef10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 --2- 172.21.15.121:0/1123594204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0ffd00 0x7f335c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 -- 172.21.15.121:0/1123594204 >> 172.21.15.121:0/1123594204 conn(0x7f335c0fa130 msgr2=0x7f335c0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 -- 172.21.15.121:0/1123594204 shutdown_connections 2023-07-13T22:07:48.938 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.941+0000 7f33613c3700 1 -- 172.21.15.121:0/1123594204 wait complete. 2023-07-13T22:07:48.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 Processor -- start 2023-07-13T22:07:48.940 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 -- start start 2023-07-13T22:07:48.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0feb00 0x7f335c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:48.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:48.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:48.941 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19f710 con 0x7f335c100690 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19f880 con 0x7f335c0feb00 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f33613c3700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19fa40 con 0x7f335c0ffd00 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f335a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40580/0 (socket says 172.21.15.121:40580) 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.942+0000 7f335a7fc700 1 -- 172.21.15.121:0/1662767753 learned_addr learned my addr 172.21.15.121:0/1662767753 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:07:48.942 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335affd700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0feb00 0x7f335c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:48.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335b7fe700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:48.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 -- 172.21.15.121:0/1662767753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0feb00 msgr2=0x7f335c0694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:48.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0feb00 0x7f335c0694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.943 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 -- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 msgr2=0x7f335c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:07:48.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:07:48.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 -- 172.21.15.121:0/1662767753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440095f0 con 0x7f335c0ffd00 2023-07-13T22:07:48.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335b7fe700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:07:48.944 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f335a7fc700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f334c00b530 tx=0x7f334c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:48.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c011830 con 0x7f335c0ffd00 2023-07-13T22:07:48.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334c011e50 con 0x7f335c0ffd00 2023-07-13T22:07:48.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00f2e0 con 0x7f335c0ffd00 2023-07-13T22:07:48.945 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c19fd20 con 0x7f335c0ffd00 2023-07-13T22:07:48.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.943+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c1a0240 con 0x7f335c0ffd00 2023-07-13T22:07:48.946 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.944+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c106190 con 0x7f335c0ffd00 2023-07-13T22:07:48.947 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.945+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f334c010380 con 0x7f335c0ffd00 2023-07-13T22:07:48.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.946+0000 7f3353fff700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 0x7f33480a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:07:48.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.946+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f334c0cb0d0 con 0x7f335c0ffd00 2023-07-13T22:07:48.948 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.947+0000 7f335affd700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 0x7f33480a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:07:48.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.947+0000 7f335affd700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 0x7f33480a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f334400b5c0 tx=0x7f3344005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:07:48.949 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:48.950+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f334c096070 con 0x7f335c0ffd00 2023-07-13T22:07:48.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:07:48.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:07:48.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi117 2023-07-13T22:07:48.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi117 to 17168M 2023-07-13T22:07:48.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:48.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] boot 2023-07-13T22:07:48.951 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: osdmap e13: 2 total, 2 up, 2 in 2023-07-13T22:07:48.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:48.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:07:48.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:07:48.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi117 2023-07-13T22:07:48.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi117 to 17168M 2023-07-13T22:07:48.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-13T22:07:48.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: osd.1 [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] boot 2023-07-13T22:07:48.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: osdmap e13: 2 total, 2 up, 2 in 2023-07-13T22:07:48.958 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-13T22:07:49.139 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:07:49.143+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f335c060950 con 0x7f33480a1140 2023-07-13T22:07:50.224 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: osdmap e14: 2 total, 2 up, 2 in 2023-07-13T22:07:50.224 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:50.225 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:50.225 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:50.225 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:50.225 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:50 smithi121 ceph-mon[129187]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:50.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: osdmap e14: 2 total, 2 up, 2 in 2023-07-13T22:07:50.459 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:50.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:50.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:50.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:50.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:50 smithi117 ceph-mon[129074]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: osdmap e14: 2 total, 2 up, 2 in 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:50.461 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:50 smithi026 ceph-mon[95647]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/3827475784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]': finished 2023-07-13T22:07:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: osdmap e15: 3 total, 2 up, 3 in 2023-07-13T22:07:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:07:52.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:52 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/840957788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/3827475784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]': finished 2023-07-13T22:07:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: osdmap e15: 3 total, 2 up, 3 in 2023-07-13T22:07:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:07:52.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:52 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/840957788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/3827475784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]: dispatch 2023-07-13T22:07:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79f28811-85d8-4f49-a4eb-c8a30a9b606b"}]': finished 2023-07-13T22:07:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: osdmap e15: 3 total, 2 up, 3 in 2023-07-13T22:07:52.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:07:52.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:52 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/840957788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:07:54.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:54 smithi117 ceph-mon[129074]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:54.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:54 smithi026 ceph-mon[95647]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:54.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:54 smithi121 ceph-mon[129187]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:56.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:56 smithi117 ceph-mon[129074]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:56.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:56 smithi026 ceph-mon[95647]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:56.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:56 smithi121 ceph-mon[129187]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:58.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:58 smithi117 ceph-mon[129074]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:58.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:58 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-13T22:07:58.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:58 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:58.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:58 smithi026 ceph-mon[95647]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:58.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:58 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-13T22:07:58.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:58 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:58.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:58 smithi121 ceph-mon[129187]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:07:58.719 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:58 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-13T22:07:58.719 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:58 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:07:59.645 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:07:59 smithi121 ceph-mon[129187]: Deploying daemon osd.2 on smithi121 2023-07-13T22:07:59.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:07:59 smithi117 ceph-mon[129074]: Deploying daemon osd.2 on smithi121 2023-07-13T22:07:59.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:07:59 smithi026 ceph-mon[95647]: Deploying daemon osd.2 on smithi121 2023-07-13T22:08:00.598 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:00 smithi121 ceph-mon[129187]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:00 smithi117 ceph-mon[129074]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:00 smithi026 ceph-mon[95647]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:02.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:02 smithi117 ceph-mon[129074]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:02.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:02 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:02 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:02 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:02.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:02 smithi026 ceph-mon[95647]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:02.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:02 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:02 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:02 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:02.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:02 smithi121 ceph-mon[129187]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:02.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:02 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:02 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:02.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:02 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:04.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:04.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:04.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:04.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:04.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:04.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:04.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:04.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:04.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:04.998 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 2 on host 'smithi121' 2023-07-13T22:08:04.999 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.003+0000 7f3353fff700 1 -- 172.21.15.121:0/1662767753 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f335c060950 con 0x7f33480a1140 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 msgr2=0x7f33480a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 0x7f33480a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f334400b5c0 tx=0x7f3344005fd0 comp rx=0 tx=0).stop 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 msgr2=0x7f335c19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f334c00b530 tx=0x7f334c00b840 comp rx=0 tx=0).stop 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 shutdown_connections 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33480a1140 0x7f33480a35f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:05.001 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f335c100690 0x7f335c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f335c0feb00 0x7f335c0694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 --2- 172.21.15.121:0/1662767753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f335c0ffd00 0x7f335c19cb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 >> 172.21.15.121:0/1662767753 conn(0x7f335c0fa130 msgr2=0x7f335c1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 shutdown_connections 2023-07-13T22:08:05.002 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:05.005+0000 7f33613c3700 1 -- 172.21.15.121:0/1662767753 wait complete. 2023-07-13T22:08:05.354 DEBUG:teuthology.orchestra.run.smithi121:osd.2> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.2.service 2023-07-13T22:08:05.356 INFO:tasks.cephadm:Deploying osd.3 on smithi026 with /dev/vg_nvme/lv_3... 2023-07-13T22:08:05.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-13T22:08:05.388 INFO:journalctl@ceph.osd.2.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:08:06.098 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:08:06.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:06.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:05 smithi117 ceph-mon[129074]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:06.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:06.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:05 smithi026 ceph-mon[95647]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:06.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:06.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:06.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:05 smithi121 ceph-mon[129187]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:06.583 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:08:06.401+0000 7f5316041640 -1 osd.2 0 log_to_monitors true 2023-07-13T22:08:07.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:06 smithi117 ceph-mon[129074]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:07.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:06 smithi117 ceph-mon[129074]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:07.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:06 smithi026 ceph-mon[95647]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:07.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:06 smithi026 ceph-mon[95647]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:07.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:06 smithi121 ceph-mon[129187]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:07.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:06 smithi121 ceph-mon[129187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-13T22:08:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-13T22:08:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-13T22:08:08.030 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-07-13T22:08:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-07-13T22:08:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.030857 s, 340 MB/s 2023-07-13T22:08:08.031 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 2023-07-13T22:08:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-07-13T22:08:08.249 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:08:07.969+0000 7f530aeb8700 -1 osd.2 0 waiting for initial osdmap 2023-07-13T22:08:08.249 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:08:07.973+0000 7f53074ae700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:08:08.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: osdmap e16: 3 total, 2 up, 3 in 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi121 2023-07-13T22:08:08.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:08.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi121 to 17168M 2023-07-13T22:08:08.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:08.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:08.252 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: osdmap e16: 3 total, 2 up, 3 in 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:08.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi121 2023-07-13T22:08:08.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:08.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi121 to 17168M 2023-07-13T22:08:08.451 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.451 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:08.451 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:08.451 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-13T22:08:08.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: osdmap e16: 3 total, 2 up, 3 in 2023-07-13T22:08:08.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi121 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:08.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi121 to 17168M 2023-07-13T22:08:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:08.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:08.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi026:vg_nvme/lv_3 2023-07-13T22:08:09.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:08 smithi121 ceph-mon[129187]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:08:09.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:08 smithi121 ceph-mon[129187]: osdmap e17: 3 total, 2 up, 3 in 2023-07-13T22:08:09.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:08 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:09.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:08 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:08:09.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:08 smithi117 ceph-mon[129074]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:08:09.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:08 smithi117 ceph-mon[129074]: osdmap e17: 3 total, 2 up, 3 in 2023-07-13T22:08:09.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:08 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:09.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:08 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:09.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:08 smithi026 ceph-mon[95647]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:08:09.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:08 smithi026 ceph-mon[95647]: osdmap e17: 3 total, 2 up, 3 in 2023-07-13T22:08:09.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:09.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:08:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:08:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] boot 2023-07-13T22:08:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: osdmap e18: 3 total, 3 up, 3 in 2023-07-13T22:08:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:10.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:10.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:10.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:08:10.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:08:10.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] boot 2023-07-13T22:08:10.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: osdmap e18: 3 total, 3 up, 3 in 2023-07-13T22:08:10.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:10.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:10.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:10.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: osd.2 [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] boot 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: osdmap e18: 3 total, 3 up, 3 in 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-13T22:08:10.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 -- 172.21.15.26:0/918619536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640101720 msgr2=0x7f4640107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 --2- 172.21.15.26:0/918619536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640101720 0x7f4640107fe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f463800b0a0 tx=0x7f463800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 -- 172.21.15.26:0/918619536 shutdown_connections 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 --2- 172.21.15.26:0/918619536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640101720 0x7f4640107fe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 --2- 172.21.15.26:0/918619536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46400fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 --2- 172.21.15.26:0/918619536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640100d90 0x7f46401011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 -- 172.21.15.26:0/918619536 >> 172.21.15.26:0/918619536 conn(0x7f46400fb1a0 msgr2=0x7f46400fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 -- 172.21.15.26:0/918619536 shutdown_connections 2023-07-13T22:08:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.628+0000 7f46462b1700 1 -- 172.21.15.26:0/918619536 wait complete. 2023-07-13T22:08:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.629+0000 7f46462b1700 1 Processor -- start 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 -- start start 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46401959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 0x7f464019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464019d570 con 0x7f4640100d90 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464019d6b0 con 0x7f46400ffb90 2023-07-13T22:08:10.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f46462b1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464019d840 con 0x7f4640101720 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f463f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f463ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46401959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f463ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46401959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:33068/0 (socket says 172.21.15.26:33068) 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f463f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57404/0 (socket says 172.21.15.26:57404) 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f463f7fe700 1 -- 172.21.15.26:0/4118613155 learned_addr learned my addr 172.21.15.26:0/4118613155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.630+0000 7f464484e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 0x7f464019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 -- 172.21.15.26:0/4118613155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 msgr2=0x7f46401959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:10.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46401959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 -- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 msgr2=0x7f4640195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 -- 172.21.15.26:0/4118613155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009d00 con 0x7f4640101720 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f463f7fe700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f464484e700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 0x7f464019cfa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f463800b070 tx=0x7f4638007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463800e040 con 0x7f4640101720 2023-07-13T22:08:10.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4638004c50 con 0x7f4640101720 2023-07-13T22:08:10.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46380052a0 con 0x7f4640101720 2023-07-13T22:08:10.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.631+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464019dac0 con 0x7f4640101720 2023-07-13T22:08:10.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.632+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464019dee0 con 0x7f4640101720 2023-07-13T22:08:10.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.632+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46401070d0 con 0x7f4640101720 2023-07-13T22:08:10.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.634+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f463801d470 con 0x7f4640101720 2023-07-13T22:08:10.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.636+0000 7f463d7fa700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 0x7f46280a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:10.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.636+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f46380ccfe0 con 0x7f4640101720 2023-07-13T22:08:10.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.636+0000 7f463ffff700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 0x7f46280a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:10.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.637+0000 7f463ffff700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 0x7f46280a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4640100bf0 tx=0x7f4630009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:10.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.638+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4638097c00 con 0x7f4640101720 2023-07-13T22:08:10.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:10.851+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4640060950 con 0x7f46280a1140 2023-07-13T22:08:11.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:11.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: osdmap e19: 3 total, 3 up, 3 in 2023-07-13T22:08:11.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:11.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:11.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:11.251 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:11.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:11.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: osdmap e19: 3 total, 3 up, 3 in 2023-07-13T22:08:11.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:11.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:11.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:11.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:11.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:11.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: osdmap e19: 3 total, 3 up, 3 in 2023-07-13T22:08:11.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:08:11.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:11.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:11.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:11.941 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:11 smithi026 sudo[111719]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:11.942 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:11 smithi026 sudo[111719]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:11.942 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:11 smithi026 sudo[111719]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:11.942 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:11 smithi026 sudo[111719]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:12.007+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4638097fd0 con 0x7f4640101720 2023-07-13T22:08:12.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:11 smithi026 ceph-mon[95647]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:12.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:12.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:11 smithi026 ceph-mon[95647]: osdmap e20: 3 total, 3 up, 3 in 2023-07-13T22:08:12.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:11 smithi026 ceph-mon[95647]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:12.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:11 smithi121 ceph-mon[129187]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:12.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:12.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:11 smithi121 ceph-mon[129187]: osdmap e20: 3 total, 3 up, 3 in 2023-07-13T22:08:12.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:11 smithi121 ceph-mon[129187]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:12.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:11 smithi117 ceph-mon[129074]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:12.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:08:12.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:11 smithi117 ceph-mon[129074]: osdmap e20: 3 total, 3 up, 3 in 2023-07-13T22:08:12.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:11 smithi117 ceph-mon[129074]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:12.456 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:12 smithi026 sudo[111769]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:12.457 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:12 smithi026 sudo[111769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:12.457 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:12 smithi026 sudo[111769]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:12.457 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:08:12 smithi026 sudo[111769]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:12.727 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136362]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:12.728 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:12.728 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136362]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:12.728 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136362]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:12.995 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136365]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:12.996 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136365]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:12.996 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136365]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:12.996 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:08:12 smithi117 sudo[136365]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:13.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:12 smithi026 ceph-mon[95647]: osdmap e21: 3 total, 3 up, 3 in 2023-07-13T22:08:13.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:12 smithi026 ceph-mon[95647]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-13T22:08:13.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:12 smithi121 ceph-mon[129187]: osdmap e21: 3 total, 3 up, 3 in 2023-07-13T22:08:13.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:12 smithi121 ceph-mon[129187]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-13T22:08:13.249 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.250 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.250 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137132]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:13.250 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:13.251 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137132]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:13.251 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137132]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:13.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:13 smithi117 ceph-mon[129074]: osdmap e21: 3 total, 3 up, 3 in 2023-07-13T22:08:13.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:13 smithi117 ceph-mon[129074]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-13T22:08:13.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:13 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:13 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:13.499 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137135]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:13.499 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:13.499 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137135]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:13.499 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:08:13 smithi121 sudo[137135]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:14.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:13 smithi026 sudo[111878]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:14.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:13 smithi026 sudo[111878]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:14.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:13 smithi026 sudo[111878]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:14.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:13 smithi026 sudo[111878]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137138]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137138]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:14.616 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137138]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:14.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:14.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:14 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:14.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 sudo[111882]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 sudo[111882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 sudo[111882]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 sudo[111882]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:14.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:14 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:14.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137141]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:14.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:14.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137141]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:14.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:14 smithi121 sudo[137141]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:15.346 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136368]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-13T22:08:15.346 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136368]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:15.347 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136368]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:15.347 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136368]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:15.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136371]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-13T22:08:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-13T22:08:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136371]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-13T22:08:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 sudo[136371]: pam_unix(sudo:session): session closed for user root 2023-07-13T22:08:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:15.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2058992306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.700 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]': finished 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: osdmap e22: 4 total, 3 up, 4 in 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.701 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:15 smithi117 ceph-mon[129074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:15.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2058992306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]': finished 2023-07-13T22:08:15.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: osdmap e22: 4 total, 3 up, 4 in 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.708 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:15 smithi026 ceph-mon[95647]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:15.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2058992306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "609be474-a324-4303-b916-e112222f2ab4"}]': finished 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: osdmap e22: 4 total, 3 up, 4 in 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:15.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-13T22:08:15.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-13T22:08:15.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-13T22:08:15.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-13T22:08:15.752 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:15 smithi121 ceph-mon[129187]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-13T22:08:16.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:16 smithi117 ceph-mon[129074]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:16.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:16 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2884638370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:16.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:16 smithi026 ceph-mon[95647]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:16.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:16 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2884638370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:16.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:16 smithi121 ceph-mon[129187]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:16.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:16 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2884638370' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:18.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:18 smithi117 ceph-mon[129074]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:18.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:18 smithi026 ceph-mon[95647]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:18.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:18 smithi121 ceph-mon[129187]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:20.587 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:20 smithi026 ceph-mon[95647]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 52 KiB/s wr, 4 op/s 2023-07-13T22:08:20.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:20 smithi117 ceph-mon[129074]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 52 KiB/s wr, 4 op/s 2023-07-13T22:08:20.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:20 smithi121 ceph-mon[129187]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 52 KiB/s wr, 4 op/s 2023-07-13T22:08:22.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:22 smithi117 ceph-mon[129074]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2023-07-13T22:08:22.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-13T22:08:22.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:22.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:22 smithi026 ceph-mon[95647]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2023-07-13T22:08:22.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-13T22:08:22.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:22.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:22 smithi121 ceph-mon[129187]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 37 KiB/s wr, 3 op/s 2023-07-13T22:08:22.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-13T22:08:22.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:23.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:23 smithi117 ceph-mon[129074]: Deploying daemon osd.3 on smithi026 2023-07-13T22:08:23.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:23 smithi026 ceph-mon[95647]: Deploying daemon osd.3 on smithi026 2023-07-13T22:08:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:23 smithi121 ceph-mon[129187]: Deploying daemon osd.3 on smithi026 2023-07-13T22:08:24.678 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:24 smithi026 ceph-mon[95647]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-07-13T22:08:24.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:24 smithi117 ceph-mon[129074]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-07-13T22:08:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:24 smithi121 ceph-mon[129187]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1 KiB/s rd, 29 KiB/s wr, 2 op/s 2023-07-13T22:08:26.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:26 smithi117 ceph-mon[129074]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 837 B/s rd, 24 KiB/s wr, 2 op/s 2023-07-13T22:08:26.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:26 smithi026 ceph-mon[95647]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 837 B/s rd, 24 KiB/s wr, 2 op/s 2023-07-13T22:08:26.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:26 smithi121 ceph-mon[129187]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 837 B/s rd, 24 KiB/s wr, 2 op/s 2023-07-13T22:08:28.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:28 smithi117 ceph-mon[129074]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:28.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:28.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:28 smithi026 ceph-mon[95647]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:28.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:28.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:28 smithi121 ceph-mon[129187]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:28.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:28.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:30 smithi117 ceph-mon[129074]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:30.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:30 smithi026 ceph-mon[95647]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:30 smithi121 ceph-mon[129187]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-07-13T22:08:31.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.770+0000 7f463d7fa700 1 -- 172.21.15.26:0/4118613155 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4640060950 con 0x7f46280a1140 2023-07-13T22:08:31.766 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 3 on host 'smithi026' 2023-07-13T22:08:31.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.773+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 msgr2=0x7f46280a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:31.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.773+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 0x7f46280a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4640100bf0 tx=0x7f4630009260 comp rx=0 tx=0).stop 2023-07-13T22:08:31.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 msgr2=0x7f464019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:31.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 0x7f464019cfa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f463800b070 tx=0x7f4638007930 comp rx=0 tx=0).stop 2023-07-13T22:08:31.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 shutdown_connections 2023-07-13T22:08:31.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4640101720 0x7f464019cfa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:31.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46400ffb90 0x7f46401959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:31.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f46280a1140 0x7f46280a35f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:31.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 --2- 172.21.15.26:0/4118613155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4640100d90 0x7f4640195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:31.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 >> 172.21.15.26:0/4118613155 conn(0x7f46400fb1a0 msgr2=0x7f4640104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:31.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 shutdown_connections 2023-07-13T22:08:31.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:08:31.774+0000 7f46462b1700 1 -- 172.21.15.26:0/4118613155 wait complete. 2023-07-13T22:08:32.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:32.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:32.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.286 DEBUG:teuthology.orchestra.run.smithi026:osd.3> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.3.service 2023-07-13T22:08:32.292 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2023-07-13T22:08:32.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-13T22:08:32.370 INFO:journalctl@ceph.osd.3.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:08:32.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:32.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:32.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:08:33.345 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:33 smithi117 ceph-mon[129074]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-13T22:08:33.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:33 smithi026 ceph-mon[95647]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-13T22:08:33.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:33 smithi121 ceph-mon[129187]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-13T22:08:34.329 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-13T22:08:34.329 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-13T22:08:34.329 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2023-07-13T22:08:34.329 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2023-07-13T22:08:34.329 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110388 s, 950 MB/s 2023-07-13T22:08:34.330 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2023-07-13T22:08:34.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:34 smithi117 ceph-mon[129074]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-13T22:08:34.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:34 smithi117 ceph-mon[129074]: osdmap e23: 4 total, 3 up, 4 in 2023-07-13T22:08:34.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:34 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:34.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:34 smithi117 ceph-mon[129074]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:08:34.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:34 smithi117 ceph-mon[129074]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:34.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-mon[95647]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-13T22:08:34.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-mon[95647]: osdmap e23: 4 total, 3 up, 4 in 2023-07-13T22:08:34.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:34.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-mon[95647]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:08:34.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-mon[95647]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:34.457 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:08:34.194+0000 7fa0d5644700 -1 osd.3 0 waiting for initial osdmap 2023-07-13T22:08:34.457 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:08:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:08:34.200+0000 7fa0d0437700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:08:34.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:34 smithi121 ceph-mon[129187]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-13T22:08:34.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:34 smithi121 ceph-mon[129187]: osdmap e23: 4 total, 3 up, 4 in 2023-07-13T22:08:34.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:34 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:34.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:34 smithi121 ceph-mon[129187]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:08:34.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:34 smithi121 ceph-mon[129187]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-07-13T22:08:34.705 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2023-07-13T22:08:35.258 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:08:35.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:08:35.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: osdmap e24: 4 total, 3 up, 4 in 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] boot 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: osdmap e25: 4 total, 4 up, 4 in 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:35.450 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:35 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:08:35.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: osdmap e24: 4 total, 3 up, 4 in 2023-07-13T22:08:35.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] boot 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: osdmap e25: 4 total, 4 up, 4 in 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:35.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:35.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:35 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:08:35.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: osdmap e24: 4 total, 3 up, 4 in 2023-07-13T22:08:35.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: osd.3 [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] boot 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: osdmap e25: 4 total, 4 up, 4 in 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:35.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:35.501 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:35.501 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:35 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:36.309 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:08:36.310 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:08:36.310 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi026 2023-07-13T22:08:36.310 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi026 to 8636M 2023-07-13T22:08:36.310 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: osdmap e26: 4 total, 4 up, 4 in 2023-07-13T22:08:36.310 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:36 smithi117 ceph-mon[129074]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:36.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:08:36.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:08:36.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi026 2023-07-13T22:08:36.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi026 to 8636M 2023-07-13T22:08:36.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: osdmap e26: 4 total, 4 up, 4 in 2023-07-13T22:08:36.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:36 smithi026 ceph-mon[95647]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:36.458 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 -- 172.21.15.117:0/714288527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 msgr2=0x7f881c106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 --2- 172.21.15.117:0/714288527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c106ed0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f881400b0a0 tx=0x7f881400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 -- 172.21.15.117:0/714288527 shutdown_connections 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 --2- 172.21.15.117:0/714288527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0fea80 0x7f881c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 --2- 172.21.15.117:0/714288527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c106ed0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 --2- 172.21.15.117:0/714288527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0ffc80 0x7f881c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.459 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 -- 172.21.15.117:0/714288527 >> 172.21.15.117:0/714288527 conn(0x7f881c0fa0b0 msgr2=0x7f881c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:36.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 -- 172.21.15.117:0/714288527 shutdown_connections 2023-07-13T22:08:36.460 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.464+0000 7f8822ea7700 1 -- 172.21.15.117:0/714288527 wait complete. 2023-07-13T22:08:36.461 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 Processor -- start 2023-07-13T22:08:36.461 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 -- start start 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 0x7f881c195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 0x7f881c196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c19d6c0 con 0x7f881c0ffc80 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c19d800 con 0x7f881c100610 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8822ea7700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c19d990 con 0x7f881c0fea80 2023-07-13T22:08:36.462 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8821444700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8821444700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59136/0 (socket says 172.21.15.117:59136) 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8821444700 1 -- 172.21.15.117:0/60124108 learned_addr learned my addr 172.21.15.117:0/60124108 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.465+0000 7f8820c43700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 0x7f881c195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f881bfff700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 0x7f881c196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 -- 172.21.15.117:0/60124108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 msgr2=0x7f881c195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 0x7f881c195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.463 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 -- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 msgr2=0x7f881c196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 0x7f881c196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 -- 172.21.15.117:0/60124108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814009d00 con 0x7f881c100610 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f881bfff700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 0x7f881c196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8820c43700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 0x7f881c195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8821444700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f881400b070 tx=0x7f8814004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881400e040 con 0x7f881c100610 2023-07-13T22:08:36.464 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c19dc10 con 0x7f881c100610 2023-07-13T22:08:36.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881401bd20 con 0x7f881c100610 2023-07-13T22:08:36.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814005900 con 0x7f881c100610 2023-07-13T22:08:36.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.466+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c19e080 con 0x7f881c100610 2023-07-13T22:08:36.465 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.467+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c1060a0 con 0x7f881c100610 2023-07-13T22:08:36.466 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.468+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f881401b860 con 0x7f881c100610 2023-07-13T22:08:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.469+0000 7f8819ffb700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 0x7f880c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.469+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f88140cdc50 con 0x7f881c100610 2023-07-13T22:08:36.511 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.469+0000 7f8820c43700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 0x7f880c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:36.512 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.470+0000 7f8820c43700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 0x7f880c0a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f881c0ffae0 tx=0x7f8810008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:36.512 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:08:36.512 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:08:36.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi026 2023-07-13T22:08:36.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi026 to 8636M 2023-07-13T22:08:36.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: osdmap e26: 4 total, 4 up, 4 in 2023-07-13T22:08:36.513 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:36 smithi121 ceph-mon[129187]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:36.513 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.472+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8814098590 con 0x7f881c100610 2023-07-13T22:08:36.657 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:36.663+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f881c060950 con 0x7f880c0a1150 2023-07-13T22:08:37.632 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:37 smithi117 ceph-mon[129074]: osdmap e27: 4 total, 4 up, 4 in 2023-07-13T22:08:37.633 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:37 smithi117 ceph-mon[129074]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:37.633 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:37.633 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:37.633 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:37.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:37 smithi026 ceph-mon[95647]: osdmap e27: 4 total, 4 up, 4 in 2023-07-13T22:08:37.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:37 smithi026 ceph-mon[95647]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:37.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:37.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:37.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:37 smithi121 ceph-mon[129187]: osdmap e27: 4 total, 4 up, 4 in 2023-07-13T22:08:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:37 smithi121 ceph-mon[129187]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:37.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:38.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:38 smithi117 ceph-mon[129074]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:38.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:38 smithi117 ceph-mon[129074]: osdmap e28: 4 total, 4 up, 4 in 2023-07-13T22:08:38.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:38 smithi026 ceph-mon[95647]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:38.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:38 smithi026 ceph-mon[95647]: osdmap e28: 4 total, 4 up, 4 in 2023-07-13T22:08:38.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:38 smithi121 ceph-mon[129187]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:38.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:38 smithi121 ceph-mon[129187]: osdmap e28: 4 total, 4 up, 4 in 2023-07-13T22:08:39.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: osdmap e29: 4 total, 4 up, 4 in 2023-07-13T22:08:39.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/3896542007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]': finished 2023-07-13T22:08:39.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: osdmap e30: 5 total, 4 up, 5 in 2023-07-13T22:08:39.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:39 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:39.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: osdmap e29: 4 total, 4 up, 4 in 2023-07-13T22:08:39.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/3896542007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]': finished 2023-07-13T22:08:39.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: osdmap e30: 5 total, 4 up, 5 in 2023-07-13T22:08:39.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:39.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: osdmap e29: 4 total, 4 up, 4 in 2023-07-13T22:08:39.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/3896542007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]: dispatch 2023-07-13T22:08:39.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "244086b9-04cf-4892-a23a-ca2d9050394a"}]': finished 2023-07-13T22:08:39.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: osdmap e30: 5 total, 4 up, 5 in 2023-07-13T22:08:39.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:40.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:40 smithi117 ceph-mon[129074]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:08:40.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:40 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/2104960366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:40.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:40 smithi026 ceph-mon[95647]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:08:40.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:40 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/2104960366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:40.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:40 smithi121 ceph-mon[129187]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:08:40.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:40 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/2104960366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:08:42.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:42 smithi117 ceph-mon[129074]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-13T22:08:42.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:42 smithi026 ceph-mon[95647]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-13T22:08:42.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:42 smithi121 ceph-mon[129187]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-13T22:08:44.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:44 smithi117 ceph-mon[129074]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:08:44.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:44 smithi026 ceph-mon[95647]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:08:44.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:44 smithi121 ceph-mon[129187]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:08:46.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:46 smithi117 ceph-mon[129074]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:08:46.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-13T22:08:46.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:46.449 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:46 smithi117 ceph-mon[129074]: Deploying daemon osd.4 on smithi117 2023-07-13T22:08:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:46 smithi026 ceph-mon[95647]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:08:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-13T22:08:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:46.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:46 smithi026 ceph-mon[95647]: Deploying daemon osd.4 on smithi117 2023-07-13T22:08:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:46 smithi121 ceph-mon[129187]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:08:46.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-13T22:08:46.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:46.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:46 smithi121 ceph-mon[129187]: Deploying daemon osd.4 on smithi117 2023-07-13T22:08:48.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:48 smithi117 ceph-mon[129074]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:08:48.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:48 smithi026 ceph-mon[95647]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:08:48.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:48 smithi121 ceph-mon[129187]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:08:50.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:50 smithi117 ceph-mon[129074]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:50.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:50.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:50 smithi026 ceph-mon[95647]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:50.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:50.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:50 smithi121 ceph-mon[129187]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:50.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:50.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:52.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:52.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:52.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:52.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:52.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.721 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.727+0000 7f8819ffb700 1 -- 172.21.15.117:0/60124108 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f881c060950 con 0x7f880c0a1150 2023-07-13T22:08:52.722 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2023-07-13T22:08:52.723 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 msgr2=0x7f880c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:52.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 0x7f880c0a3600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f881c0ffae0 tx=0x7f8810008040 comp rx=0 tx=0).stop 2023-07-13T22:08:52.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 msgr2=0x7f881c19d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:52.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f881400b070 tx=0x7f8814004db0 comp rx=0 tx=0).stop 2023-07-13T22:08:52.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 shutdown_connections 2023-07-13T22:08:52.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f881c100610 0x7f881c19d0f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:52.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f881c0fea80 0x7f881c195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:52.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f880c0a1150 0x7f880c0a3600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:52.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 --2- 172.21.15.117:0/60124108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f881c0ffc80 0x7f881c196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:52.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 >> 172.21.15.117:0/60124108 conn(0x7f881c0fa0b0 msgr2=0x7f881c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:52.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 shutdown_connections 2023-07-13T22:08:52.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:08:52.729+0000 7f8822ea7700 1 -- 172.21.15.117:0/60124108 wait complete. 2023-07-13T22:08:52.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:52.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:52.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:52.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:52.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.148 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.4.service 2023-07-13T22:08:53.149 INFO:tasks.cephadm:Deploying osd.5 on smithi121 with /dev/vg_nvme/lv_3... 2023-07-13T22:08:53.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-13T22:08:53.195 INFO:journalctl@ceph.osd.4.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:08:53.723 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:08:53.940 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.941 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.941 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:53.941 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:53 smithi121 ceph-mon[129187]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:53.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:53.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:53 smithi117 ceph-mon[129074]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:53.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:53.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:08:53.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:53 smithi026 ceph-mon[95647]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:54.414 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:08:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:08:54.068+0000 7fefcb26a640 -1 osd.4 0 log_to_monitors true 2023-07-13T22:08:54.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:54 smithi117 ceph-mon[129074]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:54.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:54 smithi117 ceph-mon[129074]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:54.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:54 smithi026 ceph-mon[95647]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:54.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:54 smithi026 ceph-mon[95647]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:54.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:54 smithi121 ceph-mon[129187]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:54.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:54 smithi121 ceph-mon[129187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-13T22:08:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-13T22:08:55.146 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-13T22:08:55.146 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2023-07-13T22:08:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2023-07-13T22:08:55.147 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110075 s, 953 MB/s 2023-07-13T22:08:55.147 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2023-07-13T22:08:55.477 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi121:vg_nvme/lv_3 2023-07-13T22:08:55.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-13T22:08:55.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: osdmap e31: 5 total, 4 up, 5 in 2023-07-13T22:08:55.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:55.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:55.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi117 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:55.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi117 to 8584M 2023-07-13T22:08:55.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:55.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:55.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:55.751 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.032 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:08:56.198 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:08:55.718+0000 7fefc00e1700 -1 osd.4 0 waiting for initial osdmap 2023-07-13T22:08:56.198 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:08:55.724+0000 7fefbc6d7700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: osdmap e31: 5 total, 4 up, 5 in 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:56.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi117 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi117 to 8584M 2023-07-13T22:08:56.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:56.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:56.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-13T22:08:56.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: osdmap e31: 5 total, 4 up, 5 in 2023-07-13T22:08:56.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:56.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:56.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi117 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:08:56.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi117 to 8584M 2023-07-13T22:08:56.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:56.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:08:56.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:08:56.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:56 smithi121 ceph-mon[129187]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:08:56.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:56 smithi121 ceph-mon[129187]: osdmap e32: 5 total, 4 up, 5 in 2023-07-13T22:08:56.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:56.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:56.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.089 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 -- 172.21.15.121:0/3277287041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f8100610 msgr2=0x7f16f8106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 --2- 172.21.15.121:0/3277287041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f8100610 0x7f16f8106ed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f16f000b0a0 tx=0x7f16f000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 -- 172.21.15.121:0/3277287041 shutdown_connections 2023-07-13T22:08:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 --2- 172.21.15.121:0/3277287041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16f80fea80 0x7f16f80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.090 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 --2- 172.21.15.121:0/3277287041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f8100610 0x7f16f8106ed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 --2- 172.21.15.121:0/3277287041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f80ffc80 0x7f16f81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 -- 172.21.15.121:0/3277287041 >> 172.21.15.121:0/3277287041 conn(0x7f16f80fa0b0 msgr2=0x7f16f80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:08:57.091 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 -- 172.21.15.121:0/3277287041 shutdown_connections 2023-07-13T22:08:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 -- 172.21.15.121:0/3277287041 wait complete. 2023-07-13T22:08:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.093+0000 7f16feed4700 1 Processor -- start 2023-07-13T22:08:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 -- start start 2023-07-13T22:08:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:57.092 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f80ffc80 0x7f16f819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16f8100610 0x7f16f819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f819d540 con 0x7f16f8100610 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f819d6b0 con 0x7f16f80ffc80 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f819d870 con 0x7f16f80fea80 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40654/0 (socket says 172.21.15.121:40654) 2023-07-13T22:08:57.093 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 -- 172.21.15.121:0/967928689 learned_addr learned my addr 172.21.15.121:0/967928689 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 -- 172.21.15.121:0/967928689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f80ffc80 msgr2=0x7f16f819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f80ffc80 0x7f16f819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 -- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16f8100610 msgr2=0x7f16f819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16f8100610 0x7f16f819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 -- 172.21.15.121:0/967928689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0009d00 con 0x7f16f80fea80 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16fcc70700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b500 tx=0x7f16ec00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:57.094 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec011830 con 0x7f16f80fea80 2023-07-13T22:08:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16ec011e50 con 0x7f16f80fea80 2023-07-13T22:08:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f819db50 con 0x7f16f80fea80 2023-07-13T22:08:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.094+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec00f2e0 con 0x7f16f80fea80 2023-07-13T22:08:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.095+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f819e120 con 0x7f16f80fea80 2023-07-13T22:08:57.095 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.096+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f8106020 con 0x7f16f80fea80 2023-07-13T22:08:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.096+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f16ec010580 con 0x7f16f80fea80 2023-07-13T22:08:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.097+0000 7f16f5ffb700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 0x7f16e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:08:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.097+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f16ec0cb2d0 con 0x7f16f80fea80 2023-07-13T22:08:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.098+0000 7f16f7fff700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 0x7f16e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:08:57.096 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.098+0000 7f16f7fff700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 0x7f16e80a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16f8069940 tx=0x7f16e4005c80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:08:57.097 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.102+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f16ec095a90 con 0x7f16f80fea80 2023-07-13T22:08:57.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:56 smithi117 ceph-mon[129074]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:08:57.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:56 smithi117 ceph-mon[129074]: osdmap e32: 5 total, 4 up, 5 in 2023-07-13T22:08:57.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:56 smithi026 ceph-mon[95647]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:08:57.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:56 smithi026 ceph-mon[95647]: osdmap e32: 5 total, 4 up, 5 in 2023-07-13T22:08:57.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:57.289 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:08:57.294+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f8060950 con 0x7f16e80a11d0 2023-07-13T22:08:57.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:08:57.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:08:57.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] boot 2023-07-13T22:08:57.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: osdmap e33: 5 total, 5 up, 5 in 2023-07-13T22:08:58.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:58.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:58.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:58.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:58.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:58.001 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:57 smithi121 ceph-mon[129187]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:58.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:08:58.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:08:58.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] boot 2023-07-13T22:08:58.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: osdmap e33: 5 total, 5 up, 5 in 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:58.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:57 smithi117 ceph-mon[129074]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:58.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:08:58.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:08:58.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: osd.4 [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] boot 2023-07-13T22:08:58.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: osdmap e33: 5 total, 5 up, 5 in 2023-07-13T22:08:58.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-13T22:08:58.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:08:58.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:08:58.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:08:58.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:08:58.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:57 smithi026 ceph-mon[95647]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-07-13T22:08:58.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:08:58 smithi121 ceph-mon[129187]: osdmap e34: 5 total, 5 up, 5 in 2023-07-13T22:08:59.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:08:58 smithi117 ceph-mon[129074]: osdmap e34: 5 total, 5 up, 5 in 2023-07-13T22:08:59.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:08:58 smithi026 ceph-mon[95647]: osdmap e34: 5 total, 5 up, 5 in 2023-07-13T22:09:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/997243130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]': finished 2023-07-13T22:09:00.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: osdmap e35: 6 total, 5 up, 6 in 2023-07-13T22:09:00.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:00.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:00 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/2645244887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/997243130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]': finished 2023-07-13T22:09:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: osdmap e35: 6 total, 5 up, 6 in 2023-07-13T22:09:00.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:00.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:00 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/2645244887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:00.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/997243130' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]: dispatch 2023-07-13T22:09:00.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1497c31-c3f2-4912-b954-a9996a18cfd6"}]': finished 2023-07-13T22:09:00.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: osdmap e35: 6 total, 5 up, 6 in 2023-07-13T22:09:00.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:00.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:00 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/2645244887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:02.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:02 smithi117 ceph-mon[129074]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:02.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:02 smithi026 ceph-mon[95647]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:02.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:02 smithi121 ceph-mon[129187]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:04.676 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:04 smithi121 ceph-mon[129187]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:04.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:04 smithi117 ceph-mon[129074]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:04.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:04 smithi026 ceph-mon[95647]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:06.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:06 smithi121 ceph-mon[129187]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:06.666 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-13T22:09:06.667 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:06.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:06 smithi117 ceph-mon[129074]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:06.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-13T22:09:06.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:06.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:06 smithi026 ceph-mon[95647]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:06.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-13T22:09:06.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:07.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:07 smithi117 ceph-mon[129074]: Deploying daemon osd.5 on smithi121 2023-07-13T22:09:07.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:07 smithi026 ceph-mon[95647]: Deploying daemon osd.5 on smithi121 2023-07-13T22:09:07.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:07 smithi121 ceph-mon[129187]: Deploying daemon osd.5 on smithi121 2023-07-13T22:09:08.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:08 smithi117 ceph-mon[129074]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:08.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:08 smithi026 ceph-mon[95647]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:08.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:08 smithi121 ceph-mon[129187]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:10.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:10 smithi117 ceph-mon[129074]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:10.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:10.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:10.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:10.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:10 smithi026 ceph-mon[95647]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:10.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:10.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:10.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:10.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:10 smithi121 ceph-mon[129187]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:10.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:10.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:10.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.654 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.659+0000 7f16f5ffb700 1 -- 172.21.15.121:0/967928689 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f16f8060950 con 0x7f16e80a11d0 2023-07-13T22:09:12.654 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 5 on host 'smithi121' 2023-07-13T22:09:12.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 msgr2=0x7f16e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:12.656 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 0x7f16e80a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f16f8069940 tx=0x7f16e4005c80 comp rx=0 tx=0).stop 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 msgr2=0x7f16f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b500 tx=0x7f16ec00b8c0 comp rx=0 tx=0).stop 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 shutdown_connections 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f16e80a11d0 0x7f16e80a3680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16f8100610 0x7f16f819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:12.657 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f80fea80 0x7f16f80693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 --2- 172.21.15.121:0/967928689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f80ffc80 0x7f16f819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 >> 172.21.15.121:0/967928689 conn(0x7f16f80fa0b0 msgr2=0x7f16f8103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:09:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 shutdown_connections 2023-07-13T22:09:12.658 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:09:12.662+0000 7f16feed4700 1 -- 172.21.15.121:0/967928689 wait complete. 2023-07-13T22:09:12.681 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:12.681 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.682 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.682 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:12.682 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:12.682 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:12.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:12.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:12.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:12.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:12.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:12.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:12.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.022 DEBUG:teuthology.orchestra.run.smithi121:osd.5> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.5.service 2023-07-13T22:09:13.024 INFO:tasks.cephadm:Deploying osd.6 on smithi026 with /dev/vg_nvme/lv_2... 2023-07-13T22:09:13.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-13T22:09:13.055 INFO:journalctl@ceph.osd.5.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:09:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:09:13.821 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:13 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:13.821 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:13 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.822 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:13 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:13 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:13.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:13 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:13 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:13.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:13.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:13 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:14.379 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:09:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:09:14.118+0000 7f292af83640 -1 osd.5 0 log_to_monitors true 2023-07-13T22:09:14.873 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:14 smithi121 ceph-mon[129187]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:14.873 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:14 smithi121 ceph-mon[129187]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:14.874 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:14 smithi121 ceph-mon[129187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:14.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:14 smithi026 ceph-mon[95647]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:14.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:14 smithi026 ceph-mon[95647]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:14.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:14 smithi026 ceph-mon[95647]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:14.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:14 smithi117 ceph-mon[129074]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:14.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:14 smithi117 ceph-mon[129074]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:14.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:14 smithi117 ceph-mon[129074]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-13T22:09:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-13T22:09:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-13T22:09:15.641 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-07-13T22:09:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-07-13T22:09:15.641 INFO:teuthology.orchestra.run.smithi026.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0346443 s, 303 MB/s 2023-07-13T22:09:15.642 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: osdmap e36: 6 total, 5 up, 6 in 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:15.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: osdmap e36: 6 total, 5 up, 6 in 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:15.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.998 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:09:15.630+0000 7f291fdfa700 -1 osd.5 0 waiting for initial osdmap 2023-07-13T22:09:15.998 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:09:15.637+0000 7f291abed700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: osdmap e36: 6 total, 5 up, 6 in 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:15.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:16.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:16.061 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi026:vg_nvme/lv_2 2023-07-13T22:09:16.796 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:09:16.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi121 2023-07-13T22:09:16.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi121 to 8584M 2023-07-13T22:09:16.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:16.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:09:16.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: osdmap e37: 6 total, 5 up, 6 in 2023-07-13T22:09:16.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] boot 2023-07-13T22:09:16.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: osdmap e38: 6 total, 6 up, 6 in 2023-07-13T22:09:16.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:16 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi121 2023-07-13T22:09:16.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi121 to 8584M 2023-07-13T22:09:16.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:16.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:09:16.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: osdmap e37: 6 total, 5 up, 6 in 2023-07-13T22:09:16.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] boot 2023-07-13T22:09:16.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: osdmap e38: 6 total, 6 up, 6 in 2023-07-13T22:09:16.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:16 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi121 2023-07-13T22:09:16.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi121 to 8584M 2023-07-13T22:09:16.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: osdmap e37: 6 total, 5 up, 6 in 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: osd.5 [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] boot 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: osdmap e38: 6 total, 6 up, 6 in 2023-07-13T22:09:16.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:16 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-13T22:09:17.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:17 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:09:17.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:17 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:09:17.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:17 smithi026 ceph-mon[95647]: osdmap e39: 6 total, 6 up, 6 in 2023-07-13T22:09:17.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:17 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:09:17.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:17 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:09:17.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:17 smithi117 ceph-mon[129074]: osdmap e39: 6 total, 6 up, 6 in 2023-07-13T22:09:17.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:17 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:09:17.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:17 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:09:17.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:17 smithi121 ceph-mon[129187]: osdmap e39: 6 total, 6 up, 6 in 2023-07-13T22:09:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 -- 172.21.15.26:0/2405959791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 msgr2=0x7f6ff80fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/2405959791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff80fffa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009990 tx=0x7f6fe8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 -- 172.21.15.26:0/2405959791 shutdown_connections 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/2405959791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff80fffa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/2405959791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 0x7f6ff8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/2405959791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 0x7f6ff81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 -- 172.21.15.26:0/2405959791 >> 172.21.15.26:0/2405959791 conn(0x7f6ff80fb1a0 msgr2=0x7f6ff80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:09:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 -- 172.21.15.26:0/2405959791 shutdown_connections 2023-07-13T22:09:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.192+0000 7f6ffd19d700 1 -- 172.21.15.26:0/2405959791 wait complete. 2023-07-13T22:09:18.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 Processor -- start 2023-07-13T22:09:18.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 -- start start 2023-07-13T22:09:18.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 0x7f6ff8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 0x7f6ff819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff819f670 con 0x7f6ff80ffb90 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff819f7b0 con 0x7f6ff8101720 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.193+0000 7f6ffd19d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff819f940 con 0x7f6ff8100d90 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46714/0 (socket says 172.21.15.26:46714) 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 -- 172.21.15.26:0/1718059598 learned_addr learned my addr 172.21.15.26:0/1718059598 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 0x7f6ff819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff659c700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 0x7f6ff8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 -- 172.21.15.26:0/1718059598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 msgr2=0x7f6ff8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 0x7f6ff8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 -- 172.21.15.26:0/1718059598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 msgr2=0x7f6ff819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 0x7f6ff819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 -- 172.21.15.26:0/1718059598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe80095f0 con 0x7f6ff80ffb90 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.194+0000 7f6ff6d9d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005740 tx=0x7f6fe8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.195+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe801d070 con 0x7f6ff80ffb90 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.195+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe8021410 con 0x7f6ff80ffb90 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.195+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff819fbc0 con 0x7f6ff80ffb90 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.195+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8017650 con 0x7f6ff80ffb90 2023-07-13T22:09:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.195+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff81a0030 con 0x7f6ff80ffb90 2023-07-13T22:09:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.196+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff810b3a0 con 0x7f6ff80ffb90 2023-07-13T22:09:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.198+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6fe8021580 con 0x7f6ff80ffb90 2023-07-13T22:09:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.200+0000 7f6fdffff700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 0x7f6fe00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.200+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f6fe80ccec0 con 0x7f6ff80ffb90 2023-07-13T22:09:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.201+0000 7f6ff659c700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 0x7f6fe00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.201+0000 7f6ff659c700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 0x7f6fe00a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009c60 tx=0x7f6fec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:09:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.205+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6fe80974a0 con 0x7f6ff80ffb90 2023-07-13T22:09:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:18.422+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff81076f0 con 0x7f6fe00a11d0 2023-07-13T22:09:18.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:18 smithi117 ceph-mon[129074]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:18.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:18.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:18.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:18 smithi026 ceph-mon[95647]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:18.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:18.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:18.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:18 smithi121 ceph-mon[129187]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:18.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:18.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:18.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:19.860 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:19 smithi026 ceph-mon[95647]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:19.861 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:19 smithi026 ceph-mon[95647]: osdmap e40: 6 total, 6 up, 6 in 2023-07-13T22:09:19.861 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:19 smithi026 ceph-mon[95647]: osdmap e41: 6 total, 6 up, 6 in 2023-07-13T22:09:19.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:19 smithi117 ceph-mon[129074]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:19.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:19 smithi117 ceph-mon[129074]: osdmap e40: 6 total, 6 up, 6 in 2023-07-13T22:09:19.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:19 smithi117 ceph-mon[129074]: osdmap e41: 6 total, 6 up, 6 in 2023-07-13T22:09:19.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:19 smithi121 ceph-mon[129187]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:19.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:19 smithi121 ceph-mon[129187]: osdmap e40: 6 total, 6 up, 6 in 2023-07-13T22:09:19.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:19 smithi121 ceph-mon[129187]: osdmap e41: 6 total, 6 up, 6 in 2023-07-13T22:09:20.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:20 smithi026 ceph-mon[95647]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:20.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:20 smithi026 ceph-mon[95647]: osdmap e42: 6 total, 6 up, 6 in 2023-07-13T22:09:20.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:20 smithi117 ceph-mon[129074]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:20 smithi117 ceph-mon[129074]: osdmap e42: 6 total, 6 up, 6 in 2023-07-13T22:09:20.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:20 smithi121 ceph-mon[129187]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:20.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:20 smithi121 ceph-mon[129187]: osdmap e42: 6 total, 6 up, 6 in 2023-07-13T22:09:21.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:21 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]: dispatch 2023-07-13T22:09:21.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:21 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]': finished 2023-07-13T22:09:21.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:21 smithi117 ceph-mon[129074]: osdmap e43: 7 total, 6 up, 7 in 2023-07-13T22:09:21.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:21.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:21 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]: dispatch 2023-07-13T22:09:21.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:21 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]': finished 2023-07-13T22:09:21.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:21 smithi026 ceph-mon[95647]: osdmap e43: 7 total, 6 up, 7 in 2023-07-13T22:09:21.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:21.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:21 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]: dispatch 2023-07-13T22:09:21.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:21 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3544356358' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6b07e8a-bab9-4cf6-8af9-86655ea5b994"}]': finished 2023-07-13T22:09:21.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:21 smithi121 ceph-mon[129187]: osdmap e43: 7 total, 6 up, 7 in 2023-07-13T22:09:21.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:22.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:22 smithi117 ceph-mon[129074]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:22.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3231426179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:22.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:22 smithi026 ceph-mon[95647]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:22.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3231426179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:22.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:22 smithi121 ceph-mon[129187]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:22.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3231426179' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:24 smithi117 ceph-mon[129074]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:24.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:24 smithi026 ceph-mon[95647]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:24.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:24 smithi121 ceph-mon[129187]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:26.915 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:26 smithi026 ceph-mon[95647]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:09:26.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:26 smithi117 ceph-mon[129074]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:09:26.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:26 smithi121 ceph-mon[129187]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:09:28.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:28 smithi117 ceph-mon[129074]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:09:28.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:28 smithi026 ceph-mon[95647]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:09:28.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:28 smithi121 ceph-mon[129187]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:09:29.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-13T22:09:29.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:29.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-13T22:09:29.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:29.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-13T22:09:29.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:30.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:30 smithi117 ceph-mon[129074]: Deploying daemon osd.6 on smithi026 2023-07-13T22:09:30.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:30 smithi117 ceph-mon[129074]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:09:30.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:30 smithi026 ceph-mon[95647]: Deploying daemon osd.6 on smithi026 2023-07-13T22:09:30.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:30 smithi026 ceph-mon[95647]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:09:30.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:30 smithi121 ceph-mon[129187]: Deploying daemon osd.6 on smithi026 2023-07-13T22:09:30.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:30 smithi121 ceph-mon[129187]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2023-07-13T22:09:32.681 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:32 smithi026 ceph-mon[95647]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-13T22:09:32.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:32 smithi117 ceph-mon[129074]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-13T22:09:32.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:32 smithi121 ceph-mon[129187]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-13T22:09:34.912 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:34 smithi026 ceph-mon[95647]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:34.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:34 smithi117 ceph-mon[129074]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:34.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:34 smithi121 ceph-mon[129187]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:36.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:36.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:36 smithi121 ceph-mon[129187]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:36.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.519 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:36.520 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:36 smithi026 ceph-mon[95647]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:36.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:36.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:36.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:36 smithi117 ceph-mon[129074]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-13T22:09:38.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:38.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:38.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:38.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:38 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:38.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:38.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:38.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:38 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:38.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:38.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:38.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:38.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:38 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.131+0000 7f6fdffff700 1 -- 172.21.15.26:0/1718059598 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6ff81076f0 con 0x7f6fe00a11d0 2023-07-13T22:09:39.125 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 6 on host 'smithi026' 2023-07-13T22:09:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 msgr2=0x7f6fe00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 0x7f6fe00a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009c60 tx=0x7f6fec009260 comp rx=0 tx=0).stop 2023-07-13T22:09:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 msgr2=0x7f6ff8197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8005740 tx=0x7f6fe8005770 comp rx=0 tx=0).stop 2023-07-13T22:09:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 shutdown_connections 2023-07-13T22:09:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6fe00a11d0 0x7f6fe00a3680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ff80ffb90 0x7f6ff8197ae0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ff8101720 0x7f6ff819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 --2- 172.21.15.26:0/1718059598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ff8100d90 0x7f6ff8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 >> 172.21.15.26:0/1718059598 conn(0x7f6ff80fb1a0 msgr2=0x7f6ff80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:09:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 shutdown_connections 2023-07-13T22:09:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:09:39.133+0000 7f6ffd19d700 1 -- 172.21.15.26:0/1718059598 wait complete. 2023-07-13T22:09:39.519 DEBUG:teuthology.orchestra.run.smithi026:osd.6> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.6.service 2023-07-13T22:09:39.523 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/vg_nvme/lv_2... 2023-07-13T22:09:39.524 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-13T22:09:39.596 INFO:journalctl@ceph.osd.6.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:09:40.086 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:09:40.324 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:40.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.325 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:40 smithi117 ceph-mon[129074]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:40.348 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.349 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.349 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:40.349 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.349 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.349 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:40 smithi026 ceph-mon[95647]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:40.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:40.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:40.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-13T22:09:40.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:40 smithi121 ceph-mon[129187]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:41.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-mon[95647]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-13T22:09:41.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-mon[95647]: osdmap e44: 7 total, 6 up, 7 in 2023-07-13T22:09:41.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:41.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-mon[95647]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.176 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-mon[95647]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.176 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:09:41.090+0000 7fc15cbf9700 -1 osd.6 0 waiting for initial osdmap 2023-07-13T22:09:41.177 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:09:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:09:41.105+0000 7fc1551e7700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:09:41.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:41 smithi117 ceph-mon[129074]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-13T22:09:41.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:41 smithi117 ceph-mon[129074]: osdmap e44: 7 total, 6 up, 7 in 2023-07-13T22:09:41.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:41 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:41.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:41 smithi117 ceph-mon[129074]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:41 smithi117 ceph-mon[129074]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:41 smithi121 ceph-mon[129187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-13T22:09:41.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:41 smithi121 ceph-mon[129187]: osdmap e44: 7 total, 6 up, 7 in 2023-07-13T22:09:41.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:41 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:41.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:41 smithi121 ceph-mon[129187]: from='osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:41 smithi121 ceph-mon[129187]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:09:41.544 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-13T22:09:41.545 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-13T22:09:41.545 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2023-07-13T22:09:41.545 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2023-07-13T22:09:41.545 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111692 s, 939 MB/s 2023-07-13T22:09:41.545 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 2023-07-13T22:09:41.546 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2023-07-13T22:09:41.928 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2023-07-13T22:09:42.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:42 smithi117 ceph-mon[129074]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:09:42.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:42 smithi117 ceph-mon[129074]: osdmap e45: 7 total, 6 up, 7 in 2023-07-13T22:09:42.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:42 smithi117 ceph-mon[129074]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:42.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:42 smithi026 ceph-mon[95647]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:09:42.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:42 smithi026 ceph-mon[95647]: osdmap e45: 7 total, 6 up, 7 in 2023-07-13T22:09:42.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:42 smithi026 ceph-mon[95647]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:42.480 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:09:42.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:42 smithi121 ceph-mon[129187]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:09:42.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:42 smithi121 ceph-mon[129187]: osdmap e45: 7 total, 6 up, 7 in 2023-07-13T22:09:42.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:42.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:42 smithi121 ceph-mon[129187]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] boot 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: osdmap e46: 7 total, 7 up, 7 in 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:43.232 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi026 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.233 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi026 to 5757M 2023-07-13T22:09:43.234 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.234 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:43.234 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:43.234 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:09:43.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] boot 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: osdmap e46: 7 total, 7 up, 7 in 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi026 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi026 to 5757M 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:43.457 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:09:43.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:09:43.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: osd.6 [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] boot 2023-07-13T22:09:43.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: osdmap e46: 7 total, 7 up, 7 in 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi026 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:09:43.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi026 to 5757M 2023-07-13T22:09:43.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:43.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:09:43.500 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:43.650 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 -- 172.21.15.117:0/4226032702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 msgr2=0x7fe34c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:43.650 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 --2- 172.21.15.117:0/4226032702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c105390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe338009990 tx=0x7fe338009ca0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.650 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 -- 172.21.15.117:0/4226032702 shutdown_connections 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 --2- 172.21.15.117:0/4226032702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c1058d0 0x7fe34c107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 --2- 172.21.15.117:0/4226032702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34c100690 0x7fe34c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 --2- 172.21.15.117:0/4226032702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c105390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 -- 172.21.15.117:0/4226032702 >> 172.21.15.117:0/4226032702 conn(0x7fe34c0fa130 msgr2=0x7fe34c0fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 -- 172.21.15.117:0/4226032702 shutdown_connections 2023-07-13T22:09:43.651 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.656+0000 7fe352d9f700 1 -- 172.21.15.117:0/4226032702 wait complete. 2023-07-13T22:09:43.653 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 Processor -- start 2023-07-13T22:09:43.653 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 -- start start 2023-07-13T22:09:43.653 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c100690 0x7fe34c0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:43.653 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34c1058d0 0x7fe34c0feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0ff300 con 0x7fe34c100690 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0ff470 con 0x7fe34c102fb0 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe352d9f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c19f910 con 0x7fe34c1058d0 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59252/0 (socket says 172.21.15.117:59252) 2023-07-13T22:09:43.654 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 -- 172.21.15.117:0/998189344 learned_addr learned my addr 172.21.15.117:0/998189344 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe350b3b700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c100690 0x7fe34c0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 -- 172.21.15.117:0/998189344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34c1058d0 msgr2=0x7fe34c0feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34c1058d0 0x7fe34c0feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 -- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c100690 msgr2=0x7fe34c0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c100690 0x7fe34c0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.657+0000 7fe34bfff700 1 -- 172.21.15.117:0/998189344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380095f0 con 0x7fe34c102fb0 2023-07-13T22:09:43.655 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe34bfff700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe3380059d0 tx=0x7fe33800f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33801d070 con 0x7fe34c102fb0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe33800fd60 con 0x7fe34c102fb0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c19fb90 con 0x7fe34c102fb0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338017650 con 0x7fe34c102fb0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.658+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c19ffa0 con 0x7fe34c102fb0 2023-07-13T22:09:43.656 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.659+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c0fbca0 con 0x7fe34c102fb0 2023-07-13T22:09:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.660+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe338004830 con 0x7fe34c102fb0 2023-07-13T22:09:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.660+0000 7fe349ffb700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 0x7fe33c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:09:43.657 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.661+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fe3380cd410 con 0x7fe34c102fb0 2023-07-13T22:09:43.658 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.661+0000 7fe350b3b700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 0x7fe33c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:09:43.658 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.661+0000 7fe350b3b700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 0x7fe33c0a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe340009bf0 tx=0x7fe340009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:09:43.658 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.664+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe338097900 con 0x7fe34c102fb0 2023-07-13T22:09:43.851 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:09:43.857+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe34c060950 con 0x7fe33c0a1150 2023-07-13T22:09:44.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:44 smithi117 ceph-mon[129074]: osdmap e47: 7 total, 7 up, 7 in 2023-07-13T22:09:44.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:44 smithi117 ceph-mon[129074]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:44.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:44.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:44.448 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:44.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:44 smithi026 ceph-mon[95647]: osdmap e47: 7 total, 7 up, 7 in 2023-07-13T22:09:44.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:44 smithi026 ceph-mon[95647]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:44.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:44.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:44.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:44.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:44 smithi121 ceph-mon[129187]: osdmap e47: 7 total, 7 up, 7 in 2023-07-13T22:09:44.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:44 smithi121 ceph-mon[129187]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:44.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:09:44.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:09:44.499 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:45.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:45 smithi117 ceph-mon[129074]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:45.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:45 smithi026 ceph-mon[95647]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:45.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:45 smithi121 ceph-mon[129187]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:09:46.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:46.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/1972549779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]': finished 2023-07-13T22:09:46.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: osdmap e48: 8 total, 7 up, 8 in 2023-07-13T22:09:46.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:09:46.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:46.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/1972549779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]': finished 2023-07-13T22:09:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: osdmap e48: 8 total, 7 up, 8 in 2023-07-13T22:09:46.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/1972549779' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]: dispatch 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "102fca76-132a-4dfd-86d3-f8adfc6caf04"}]': finished 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: osdmap e48: 8 total, 7 up, 8 in 2023-07-13T22:09:46.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:09:47.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:47 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/4194855557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:47.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:47 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/4194855557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:47.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:47 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/4194855557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:09:48.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:48 smithi117 ceph-mon[129074]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:48.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:48 smithi026 ceph-mon[95647]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:48.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:48 smithi121 ceph-mon[129187]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:50.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:50 smithi117 ceph-mon[129074]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:50.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:50 smithi026 ceph-mon[95647]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:50.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:50 smithi121 ceph-mon[129187]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:52.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:52 smithi117 ceph-mon[129074]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:52.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:52 smithi026 ceph-mon[95647]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:52.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:52 smithi121 ceph-mon[129187]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:53.649 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-13T22:09:53.650 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:53.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-13T22:09:53.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:53.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-13T22:09:53.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:09:54.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:54 smithi117 ceph-mon[129074]: Deploying daemon osd.7 on smithi117 2023-07-13T22:09:54.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:54 smithi117 ceph-mon[129074]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:54.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:54 smithi026 ceph-mon[95647]: Deploying daemon osd.7 on smithi117 2023-07-13T22:09:54.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:54 smithi026 ceph-mon[95647]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:54.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:54 smithi121 ceph-mon[129187]: Deploying daemon osd.7 on smithi117 2023-07-13T22:09:54.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:54 smithi121 ceph-mon[129187]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:56.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:56 smithi117 ceph-mon[129074]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:56.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:56 smithi026 ceph-mon[95647]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:56.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:56 smithi121 ceph-mon[129187]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:58.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:58 smithi117 ceph-mon[129074]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:58.559 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:58 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:58 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:09:58 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:58.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:58 smithi026 ceph-mon[95647]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:58.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:58 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:58 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:09:58 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:09:58.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:58 smithi121 ceph-mon[129187]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:09:58.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:58 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:58 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:09:58.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:09:58 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:00.118 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2023-07-13T22:10:00.119 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.124+0000 7fe349ffb700 1 -- 172.21.15.117:0/998189344 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe34c060950 con 0x7fe33c0a1150 2023-07-13T22:10:00.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 msgr2=0x7fe33c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:00.120 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 0x7fe33c0a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe340009bf0 tx=0x7fe340009260 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 msgr2=0x7fe34c0fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe3380059d0 tx=0x7fe33800f510 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 shutdown_connections 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe33c0a1150 0x7fe33c0a3600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe34c100690 0x7fe34c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe34c1058d0 0x7fe34c0feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 --2- 172.21.15.117:0/998189344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe34c102fb0 0x7fe34c0fe640 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:00.121 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 >> 172.21.15.117:0/998189344 conn(0x7fe34c0fa130 msgr2=0x7fe34c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:00.122 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 shutdown_connections 2023-07-13T22:10:00.122 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:00.127+0000 7fe352d9f700 1 -- 172.21.15.117:0/998189344 wait complete. 2023-07-13T22:10:00.581 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.7.service 2023-07-13T22:10:00.583 INFO:tasks.cephadm:Deploying osd.8 on smithi121 with /dev/vg_nvme/lv_2... 2023-07-13T22:10:00.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-13T22:10:00.616 INFO:journalctl@ceph.osd.7.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:10:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: overall HEALTH_OK 2023-07-13T22:10:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:00.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:00.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:00.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: overall HEALTH_OK 2023-07-13T22:10:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:00.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: overall HEALTH_OK 2023-07-13T22:10:00.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:00.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:01.146 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:10:01.544 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:01 smithi117 ceph-mon[129074]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:01.545 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:01 smithi117 ceph-mon[129074]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:01.545 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:10:01 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:10:01.295+0000 7f1e3d062640 -1 osd.7 0 log_to_monitors true 2023-07-13T22:10:01.626 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:01 smithi121 ceph-mon[129187]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:01.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:01 smithi121 ceph-mon[129187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:01.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:01 smithi026 ceph-mon[95647]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:01.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:01 smithi026 ceph-mon[95647]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-13T22:10:02.588 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-13T22:10:02.589 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-13T22:10:02.589 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2023-07-13T22:10:02.589 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2023-07-13T22:10:02.589 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109364 s, 959 MB/s 2023-07-13T22:10:02.589 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2023-07-13T22:10:02.673 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:10:02.433+0000 7f1e31ed9700 -1 osd.7 0 waiting for initial osdmap 2023-07-13T22:10:02.674 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:10:02.445+0000 7f1e2bcca700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:10:02.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:02.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-13T22:10:02.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: osdmap e49: 8 total, 7 up, 8 in 2023-07-13T22:10:02.674 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:02.675 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.675 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:02 smithi117 ceph-mon[129074]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:02.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-13T22:10:02.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: osdmap e49: 8 total, 7 up, 8 in 2023-07-13T22:10:02.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:02.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:02 smithi026 ceph-mon[95647]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.727 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:02.727 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-13T22:10:02.727 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: osdmap e49: 8 total, 7 up, 8 in 2023-07-13T22:10:02.727 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:02.727 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: from='osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.728 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:02 smithi121 ceph-mon[129187]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:10:02.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi121:vg_nvme/lv_2 2023-07-13T22:10:03.535 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:10:03.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:10:03.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: osdmap e50: 8 total, 7 up, 8 in 2023-07-13T22:10:03.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi117 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:03.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:03.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:10:03.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: osdmap e50: 8 total, 7 up, 8 in 2023-07-13T22:10:03.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi117 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:03.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:03.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.716 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: osdmap e50: 8 total, 7 up, 8 in 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi117 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.717 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:03.718 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:04.568 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi117 to 5722M 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] boot 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: osdmap e51: 8 total, 8 up, 8 in 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:04.569 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:04 smithi121 ceph-mon[129187]: osdmap e52: 8 total, 8 up, 8 in 2023-07-13T22:10:04.697 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- 172.21.15.121:0/4271934439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 msgr2=0x7f9c5c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- 172.21.15.121:0/4271934439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c106f60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400b0a0 tx=0x7f9c5400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- 172.21.15.121:0/4271934439 shutdown_connections 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- 172.21.15.121:0/4271934439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c1000a0 0x7f9c5c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- 172.21.15.121:0/4271934439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c106f60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- 172.21.15.121:0/4271934439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 0x7f9c5c0ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- 172.21.15.121:0/4271934439 >> 172.21.15.121:0/4271934439 conn(0x7f9c5c0f9860 msgr2=0x7f9c5c0fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:04.698 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- 172.21.15.121:0/4271934439 shutdown_connections 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- 172.21.15.121:0/4271934439 wait complete. 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 Processor -- start 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- start start 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 0x7f9c5c195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c1000a0 0x7f9c5c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:04.699 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.703+0000 7f9c641af700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19d5a0 con 0x7f9c5c1000a0 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c641af700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19d6e0 con 0x7f9c5c0fed00 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c641af700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19d870 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40782/0 (socket says 172.21.15.121:40782) 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 -- 172.21.15.121:0/1857562567 learned_addr learned my addr 172.21.15.121:0/1857562567 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c61f4b700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 0x7f9c5c195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 -- 172.21.15.121:0/1857562567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 msgr2=0x7f9c5c195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:04.700 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 0x7f9c5c195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 -- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c1000a0 msgr2=0x7f9c5c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c1000a0 0x7f9c5c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 -- 172.21.15.121:0/1857562567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c54009d00 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c6174a700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c950 tx=0x7f9c5800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58007ac0 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c58004cf0 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.701 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58005550 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c19db50 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.702 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.704+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c19e0d0 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:10:04.702 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi117 to 5722M 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] boot 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: osdmap e51: 8 total, 8 up, 8 in 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:04.703 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:04 smithi117 ceph-mon[129074]: osdmap e52: 8 total, 8 up, 8 in 2023-07-13T22:10:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.705+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c106b60 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.706+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9c58004750 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.707+0000 7f9c52ffd700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 0x7f9c480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:04.704 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.707+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f9c580cb950 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.707+0000 7f9c61f4b700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 0x7f9c480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.708+0000 7f9c61f4b700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 0x7f9c480a3680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006fd0 tx=0x7f9c4c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:04.705 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.711+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9c58095be0 con 0x7f9c5c0ff6b0 2023-07-13T22:10:04.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:10:04.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi117 to 5722M 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: osd.7 [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] boot 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: osdmap e51: 8 total, 8 up, 8 in 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-13T22:10:04.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:04 smithi026 ceph-mon[95647]: osdmap e52: 8 total, 8 up, 8 in 2023-07-13T22:10:04.895 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:04.902+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c5c060950 con 0x7f9c480a11d0 2023-07-13T22:10:05.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:05.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:05.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:05.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:05 smithi117 ceph-mon[129074]: osdmap e53: 8 total, 8 up, 8 in 2023-07-13T22:10:05.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:05.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:05.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:05.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:05 smithi026 ceph-mon[95647]: osdmap e53: 8 total, 8 up, 8 in 2023-07-13T22:10:05.706 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:05.706 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:05.706 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:05.706 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:05 smithi121 ceph-mon[129187]: osdmap e53: 8 total, 8 up, 8 in 2023-07-13T22:10:06.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:06 smithi117 ceph-mon[129074]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:06.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:06 smithi117 ceph-mon[129074]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:06.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:06 smithi026 ceph-mon[95647]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:06.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:06 smithi026 ceph-mon[95647]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:06.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:06 smithi121 ceph-mon[129187]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:06.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:06 smithi121 ceph-mon[129187]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:07.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: osdmap e54: 8 total, 8 up, 8 in 2023-07-13T22:10:07.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/4017941062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]': finished 2023-07-13T22:10:07.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: osdmap e55: 9 total, 8 up, 9 in 2023-07-13T22:10:07.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:07.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: osdmap e54: 8 total, 8 up, 8 in 2023-07-13T22:10:07.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/4017941062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]': finished 2023-07-13T22:10:07.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: osdmap e55: 9 total, 8 up, 9 in 2023-07-13T22:10:07.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:07.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: osdmap e54: 8 total, 8 up, 8 in 2023-07-13T22:10:07.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/4017941062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]: dispatch 2023-07-13T22:10:07.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dac2076e-cc3e-47d2-8d8c-267556ea9a0f"}]': finished 2023-07-13T22:10:07.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: osdmap e55: 9 total, 8 up, 9 in 2023-07-13T22:10:07.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:08.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:08 smithi117 ceph-mon[129074]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:08.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:08 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/1114484176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:08.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:08 smithi026 ceph-mon[95647]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:08.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:08 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/1114484176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:08.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:08 smithi121 ceph-mon[129187]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:08.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:08 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/1114484176' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:10.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:10 smithi117 ceph-mon[129074]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:10.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:10 smithi026 ceph-mon[95647]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:10.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:10 smithi121 ceph-mon[129187]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:12.482 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:12 smithi121 ceph-mon[129187]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:12.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:12 smithi026 ceph-mon[95647]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:12 smithi117 ceph-mon[129074]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:14.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:14 smithi121 ceph-mon[129187]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:14.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-13T22:10:14.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:14.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:14 smithi121 ceph-mon[129187]: Deploying daemon osd.8 on smithi121 2023-07-13T22:10:14.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:14 smithi026 ceph-mon[95647]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:14.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-13T22:10:14.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:14.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:14 smithi026 ceph-mon[95647]: Deploying daemon osd.8 on smithi121 2023-07-13T22:10:14.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:14 smithi117 ceph-mon[129074]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:14.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-13T22:10:14.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:14.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:14 smithi117 ceph-mon[129074]: Deploying daemon osd.8 on smithi121 2023-07-13T22:10:16.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:16 smithi026 ceph-mon[95647]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:16.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:16 smithi121 ceph-mon[129187]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:16.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:16 smithi117 ceph-mon[129074]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:18.707 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:18 smithi121 ceph-mon[129187]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:18.708 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.708 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.708 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:18.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:18 smithi117 ceph-mon[129074]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:18.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:18 smithi026 ceph-mon[95647]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:18.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:20.409 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.415+0000 7f9c52ffd700 1 -- 172.21.15.121:0/1857562567 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9c5c060950 con 0x7f9c480a11d0 2023-07-13T22:10:20.410 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 8 on host 'smithi121' 2023-07-13T22:10:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.417+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 msgr2=0x7f9c480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:20.411 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.417+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 0x7f9c480a3680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006fd0 tx=0x7f9c4c008040 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.417+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 msgr2=0x7f9c5c195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.417+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c950 tx=0x7f9c5800cd10 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 shutdown_connections 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9c480a11d0 0x7f9c480a3680 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c5c1000a0 0x7f9c5c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c5c0fed00 0x7f9c5c195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 --2- 172.21.15.121:0/1857562567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c5c0ff6b0 0x7f9c5c195f50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:20.412 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 >> 172.21.15.121:0/1857562567 conn(0x7f9c5c0f9860 msgr2=0x7f9c5c0fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 shutdown_connections 2023-07-13T22:10:20.413 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:10:20.418+0000 7f9c641af700 1 -- 172.21.15.121:0/1857562567 wait complete. 2023-07-13T22:10:20.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:20.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:20.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:20.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:20.826 DEBUG:teuthology.orchestra.run.smithi121:osd.8> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.8.service 2023-07-13T22:10:20.828 INFO:tasks.cephadm:Deploying osd.9 on smithi026 with /dev/vg_nvme/lv_1... 2023-07-13T22:10:20.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-13T22:10:20.858 INFO:journalctl@ceph.osd.8.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:10:20.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:20.899 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.900 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:20.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:20.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:21.580 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:10:21.997 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:10:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:10:21.670+0000 7f2a67f0d640 -1 osd.8 0 log_to_monitors true 2023-07-13T22:10:22.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:22 smithi121 ceph-mon[129187]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:22.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:22 smithi121 ceph-mon[129187]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:22.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:22 smithi121 ceph-mon[129187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:22.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:22 smithi117 ceph-mon[129074]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:22.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:22 smithi117 ceph-mon[129074]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:22.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:22 smithi117 ceph-mon[129074]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:22.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:22 smithi026 ceph-mon[95647]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:22.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:22 smithi026 ceph-mon[95647]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:22.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:22 smithi026 ceph-mon[95647]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-13T22:10:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-13T22:10:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-13T22:10:23.353 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10+0 records in 2023-07-13T22:10:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:10+0 records out 2023-07-13T22:10:23.353 INFO:teuthology.orchestra.run.smithi026.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0322944 s, 325 MB/s 2023-07-13T22:10:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:--> Zapping successful for: 2023-07-13T22:10:23.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: osdmap e56: 9 total, 8 up, 9 in 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:23.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.747 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:10:23.486+0000 7f2a5e587700 -1 osd.8 0 waiting for initial osdmap 2023-07-13T22:10:23.748 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:10:23.497+0000 7f2a55372700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:10:23.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-13T22:10:23.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: osdmap e56: 9 total, 8 up, 9 in 2023-07-13T22:10:23.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:23.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:23.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:23.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi026:vg_nvme/lv_1 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: osdmap e56: 9 total, 8 up, 9 in 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:23.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:24.498 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:10:24.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi121 2023-07-13T22:10:24.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi121 to 5722M 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: osdmap e57: 9 total, 8 up, 9 in 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] boot 2023-07-13T22:10:24.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: osdmap e58: 9 total, 9 up, 9 in 2023-07-13T22:10:24.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.753 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi121 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi121 to 5722M 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: osdmap e57: 9 total, 8 up, 9 in 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.754 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] boot 2023-07-13T22:10:24.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: osdmap e58: 9 total, 9 up, 9 in 2023-07-13T22:10:24.755 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi121 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi121 to 5722M 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: osdmap e57: 9 total, 8 up, 9 in 2023-07-13T22:10:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:24.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: osd.8 [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] boot 2023-07-13T22:10:24.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: osdmap e58: 9 total, 9 up, 9 in 2023-07-13T22:10:24.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-13T22:10:25.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:25 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:10:25.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:25 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:10:25.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:25 smithi121 ceph-mon[129187]: osdmap e59: 9 total, 9 up, 9 in 2023-07-13T22:10:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 -- 172.21.15.26:0/941087024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c105850 msgr2=0x7f8e6c107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 --2- 172.21.15.26:0/941087024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c105850 0x7f8e6c107cb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400b0a0 tx=0x7f8e6400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 -- 172.21.15.26:0/941087024 shutdown_connections 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 --2- 172.21.15.26:0/941087024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c105850 0x7f8e6c107cb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 --2- 172.21.15.26:0/941087024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 0x7f8e6c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 --2- 172.21.15.26:0/941087024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c102f30 0x7f8e6c105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 -- 172.21.15.26:0/941087024 >> 172.21.15.26:0/941087024 conn(0x7f8e6c0fa0b0 msgr2=0x7f8e6c0fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 -- 172.21.15.26:0/941087024 shutdown_connections 2023-07-13T22:10:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.872+0000 7f8e7465d700 1 -- 172.21.15.26:0/941087024 wait complete. 2023-07-13T22:10:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 Processor -- start 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 -- start start 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 0x7f8e6c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c105850 0x7f8e6c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.874+0000 7f8e7465d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c19f6f0 con 0x7f8e6c102f30 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e7465d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c19f830 con 0x7f8e6c105850 2023-07-13T22:10:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e7465d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c19f9c0 con 0x7f8e6c100610 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52462/0 (socket says 172.21.15.26:52462) 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 -- 172.21.15.26:0/3016378002 learned_addr learned my addr 172.21.15.26:0/3016378002 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e723f9700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 0x7f8e6c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e72bfa700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c105850 0x7f8e6c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 -- 172.21.15.26:0/3016378002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 msgr2=0x7f8e6c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 0x7f8e6c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 -- 172.21.15.26:0/3016378002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c105850 msgr2=0x7f8e6c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c105850 0x7f8e6c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 -- 172.21.15.26:0/3016378002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e64009d00 con 0x7f8e6c102f30 2023-07-13T22:10:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.875+0000 7f8e71bf8700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00d980 tx=0x7f8e5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.876+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c0098d0 con 0x7f8e6c102f30 2023-07-13T22:10:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.876+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c19fca0 con 0x7f8e6c102f30 2023-07-13T22:10:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.876+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e5c010460 con 0x7f8e6c102f30 2023-07-13T22:10:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.876+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c00f620 con 0x7f8e6c102f30 2023-07-13T22:10:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.876+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c1a01a0 con 0x7f8e6c102f30 2023-07-13T22:10:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.877+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c1073c0 con 0x7f8e6c102f30 2023-07-13T22:10:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.879+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8e5c010860 con 0x7f8e6c102f30 2023-07-13T22:10:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.881+0000 7f8e637fe700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 0x7f8e580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.881+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f8e5c0cbd70 con 0x7f8e6c102f30 2023-07-13T22:10:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.881+0000 7f8e723f9700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 0x7f8e580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.882+0000 7f8e723f9700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 0x7f8e580a3600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005950 tx=0x7f8e6800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:25.883+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e5c095db0 con 0x7f8e6c102f30 2023-07-13T22:10:25.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:25 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:10:25.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:25 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:10:25.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:25 smithi117 ceph-mon[129074]: osdmap e59: 9 total, 9 up, 9 in 2023-07-13T22:10:25.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:25 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:10:25.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:25 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:10:25.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:25 smithi026 ceph-mon[95647]: osdmap e59: 9 total, 9 up, 9 in 2023-07-13T22:10:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:26.100+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6c060950 con 0x7f8e580a1150 2023-07-13T22:10:26.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:26 smithi121 ceph-mon[129187]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:26.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:26.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:26.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:26 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:26.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:26 smithi121 ceph-mon[129187]: osdmap e60: 9 total, 9 up, 9 in 2023-07-13T22:10:26.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:26 smithi117 ceph-mon[129074]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:26.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:26.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:26.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:26 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:26.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:26 smithi117 ceph-mon[129074]: osdmap e60: 9 total, 9 up, 9 in 2023-07-13T22:10:26.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:26 smithi026 ceph-mon[95647]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:26.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:26.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:26.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:26 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:26.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:26 smithi026 ceph-mon[95647]: osdmap e60: 9 total, 9 up, 9 in 2023-07-13T22:10:27.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:27 smithi026 ceph-mon[95647]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:27.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:27 smithi121 ceph-mon[129187]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:27.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:27 smithi117 ceph-mon[129074]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi026:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:28.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:28 smithi117 ceph-mon[129074]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:28.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:28 smithi117 ceph-mon[129074]: osdmap e61: 9 total, 9 up, 9 in 2023-07-13T22:10:28.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:28 smithi026 ceph-mon[95647]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:28.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:28 smithi026 ceph-mon[95647]: osdmap e61: 9 total, 9 up, 9 in 2023-07-13T22:10:28.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:28 smithi121 ceph-mon[129187]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:28.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:28 smithi121 ceph-mon[129187]: osdmap e61: 9 total, 9 up, 9 in 2023-07-13T22:10:29.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:29 smithi117 ceph-mon[129074]: osdmap e62: 9 total, 9 up, 9 in 2023-07-13T22:10:29.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:29 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]: dispatch 2023-07-13T22:10:29.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:29 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]': finished 2023-07-13T22:10:29.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:29 smithi117 ceph-mon[129074]: osdmap e63: 10 total, 9 up, 10 in 2023-07-13T22:10:29.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:29.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:29 smithi026 ceph-mon[95647]: osdmap e62: 9 total, 9 up, 9 in 2023-07-13T22:10:29.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:29 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]: dispatch 2023-07-13T22:10:29.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:29 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]': finished 2023-07-13T22:10:29.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:29 smithi026 ceph-mon[95647]: osdmap e63: 10 total, 9 up, 10 in 2023-07-13T22:10:29.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:29.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:29 smithi121 ceph-mon[129187]: osdmap e62: 9 total, 9 up, 9 in 2023-07-13T22:10:29.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:29 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]: dispatch 2023-07-13T22:10:29.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:29 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3703700259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a2b112d-b9df-4867-bbfe-7d3455f384ca"}]': finished 2023-07-13T22:10:29.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:29 smithi121 ceph-mon[129187]: osdmap e63: 10 total, 9 up, 10 in 2023-07-13T22:10:29.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:30.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:30 smithi117 ceph-mon[129074]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:10:30.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:30 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2478722636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:30.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:30 smithi026 ceph-mon[95647]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:10:30.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:30 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2478722636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:30.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:30 smithi121 ceph-mon[129187]: pgmap v192: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:10:30.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:30 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2478722636' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:32.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:32 smithi117 ceph-mon[129074]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-13T22:10:32.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:32 smithi026 ceph-mon[95647]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-13T22:10:32.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:32 smithi121 ceph-mon[129187]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2023-07-13T22:10:34.814 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:34 smithi026 ceph-mon[95647]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:10:34.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:34 smithi117 ceph-mon[129074]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:10:34.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:34 smithi121 ceph-mon[129187]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-13T22:10:36.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:36 smithi117 ceph-mon[129074]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-13T22:10:36.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-13T22:10:36.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:36.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:36 smithi026 ceph-mon[95647]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-13T22:10:36.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-13T22:10:36.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:36.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:36 smithi121 ceph-mon[129187]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-13T22:10:36.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-13T22:10:36.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:37.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:37 smithi026 ceph-mon[95647]: Deploying daemon osd.9 on smithi026 2023-07-13T22:10:37.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:37 smithi117 ceph-mon[129074]: Deploying daemon osd.9 on smithi026 2023-07-13T22:10:37.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:37 smithi121 ceph-mon[129187]: Deploying daemon osd.9 on smithi026 2023-07-13T22:10:38.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:38 smithi026 ceph-mon[95647]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:38.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:38 smithi117 ceph-mon[129074]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:38.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:38 smithi121 ceph-mon[129187]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:40.773 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:40 smithi026 ceph-mon[95647]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:40.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:40 smithi117 ceph-mon[129074]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:40.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:40 smithi121 ceph-mon[129187]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:42.644 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:42 smithi026 ceph-mon[95647]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:42 smithi117 ceph-mon[129074]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:42.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:42 smithi121 ceph-mon[129187]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:43.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:43.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:43.881 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:43 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:43.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:43 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:43.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:43.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:43.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:43 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:44.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:44 smithi026 ceph-mon[95647]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:44.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:44 smithi117 ceph-mon[129074]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:44.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:44 smithi121 ceph-mon[129187]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:46.170 INFO:teuthology.orchestra.run.smithi026.stdout:Created osd(s) 9 on host 'smithi026' 2023-07-13T22:10:46.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.176+0000 7f8e637fe700 1 -- 172.21.15.26:0/3016378002 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8e6c060950 con 0x7f8e580a1150 2023-07-13T22:10:46.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 msgr2=0x7f8e580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 0x7f8e580a3600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e68005950 tx=0x7f8e6800b410 comp rx=0 tx=0).stop 2023-07-13T22:10:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 msgr2=0x7f8e6c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:46.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00d980 tx=0x7f8e5c00dc90 comp rx=0 tx=0).stop 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 shutdown_connections 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e6c105850 0x7f8e6c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e6c100610 0x7f8e6c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8e580a1150 0x7f8e580a3600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 --2- 172.21.15.26:0/3016378002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e6c102f30 0x7f8e6c1980a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 >> 172.21.15.26:0/3016378002 conn(0x7f8e6c0fa0b0 msgr2=0x7f8e6c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:46.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 shutdown_connections 2023-07-13T22:10:46.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:10:46.179+0000 7f8e7465d700 1 -- 172.21.15.26:0/3016378002 wait complete. 2023-07-13T22:10:46.627 DEBUG:teuthology.orchestra.run.smithi026:osd.9> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.9.service 2023-07-13T22:10:46.632 INFO:tasks.cephadm:Deploying osd.10 on smithi117 with /dev/vg_nvme/lv_1... 2023-07-13T22:10:46.632 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-13T22:10:46.721 INFO:journalctl@ceph.osd.9.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:10:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:46.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:46.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:46.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:46.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:46.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:10:47.196 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:10:47.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:47 smithi026 ceph-mon[95647]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:47.549 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:47 smithi026 ceph-mon[95647]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:47.664 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:47 smithi117 ceph-mon[129074]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:47.664 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:47 smithi117 ceph-mon[129074]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:47.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:47 smithi121 ceph-mon[129187]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:47.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:47 smithi121 ceph-mon[129187]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-13T22:10:48.661 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-13T22:10:48.662 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-13T22:10:48.662 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2023-07-13T22:10:48.662 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2023-07-13T22:10:48.662 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111733 s, 938 MB/s 2023-07-13T22:10:48.662 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2023-07-13T22:10:48.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:48.805 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-13T22:10:48.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: osdmap e64: 10 total, 9 up, 10 in 2023-07-13T22:10:48.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:48.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:48.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:48 smithi117 ceph-mon[129074]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:48.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:48.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-13T22:10:48.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: osdmap e64: 10 total, 9 up, 10 in 2023-07-13T22:10:48.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:48.917 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:48.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-mon[95647]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:48.918 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:10:48.557+0000 7feae8b56700 -1 osd.9 0 waiting for initial osdmap 2023-07-13T22:10:48.918 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:10:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:10:48.576+0000 7feae2146700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:10:48.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:48.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-13T22:10:48.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: osdmap e64: 10 total, 9 up, 10 in 2023-07-13T22:10:48.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:48.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: from='osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:48.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:48 smithi121 ceph-mon[129187]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]: dispatch 2023-07-13T22:10:49.041 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2023-07-13T22:10:49.593 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:10:49.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: osdmap e65: 10 total, 9 up, 10 in 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:49.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: osdmap e65: 10 total, 9 up, 10 in 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.810 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:49.811 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi026", "root=default"]}]': finished 2023-07-13T22:10:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: osdmap e65: 10 total, 9 up, 10 in 2023-07-13T22:10:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:10:49.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:49.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:49.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:10:49.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:10:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.740+0000 7f53b6b8e700 1 -- 172.21.15.117:0/4154940171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0100610 msgr2=0x7f53b01029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.740+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/4154940171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0100610 0x7f53b01029f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009990 tx=0x7f53a0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- 172.21.15.117:0/4154940171 shutdown_connections 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/4154940171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0105850 0x7f53b0107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/4154940171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0100610 0x7f53b01029f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/4154940171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0102f30 0x7f53b0105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- 172.21.15.117:0/4154940171 >> 172.21.15.117:0/4154940171 conn(0x7f53b00fa0b0 msgr2=0x7f53b00fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- 172.21.15.117:0/4154940171 shutdown_connections 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- 172.21.15.117:0/4154940171 wait complete. 2023-07-13T22:10:50.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 Processor -- start 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- start start 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0100610 0x7f53b0195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0105850 0x7f53b019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019d460 con 0x7f53b0100610 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019d5a0 con 0x7f53b0102f30 2023-07-13T22:10:50.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.741+0000 7f53b6b8e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b019d730 con 0x7f53b0105850 2023-07-13T22:10:50.737 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:50.737 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59392/0 (socket says 172.21.15.117:59392) 2023-07-13T22:10:50.737 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 -- 172.21.15.117:0/1690381711 learned_addr learned my addr 172.21.15.117:0/1690381711 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-07-13T22:10:50.737 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53b512b700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0105850 0x7f53b019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:50.737 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53b492a700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0100610 0x7f53b0195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:50.738 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 -- 172.21.15.117:0/1690381711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0105850 msgr2=0x7f53b019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:50.738 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0105850 0x7f53b019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 -- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0100610 msgr2=0x7f53b0195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0100610 0x7f53b0195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 -- 172.21.15.117:0/1690381711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a00095f0 con 0x7f53b0102f30 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53affff700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f539800e9a0 tx=0x7f539800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398009790 con 0x7f53b0102f30 2023-07-13T22:10:50.739 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b019da10 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f539800cbd0 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398018680 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.742+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b019df90 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.743+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b00fbc20 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.744+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f539800cd40 con 0x7f53b0102f30 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.745+0000 7f53adffb700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 0x7f539c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:10:50.740 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.745+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5398014070 con 0x7f53b0102f30 2023-07-13T22:10:50.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.746+0000 7f53b492a700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 0x7f539c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:10:50.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.746+0000 7f53b492a700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 0x7f539c0a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53a0006010 tx=0x7f53a000b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:10:50.741 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.749+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5398096590 con 0x7f53b0102f30 2023-07-13T22:10:50.935 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:10:50.942+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f53b0060950 con 0x7f539c0a1150 2023-07-13T22:10:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:10:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:10:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi026 2023-07-13T22:10:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi026 to 4318M 2023-07-13T22:10:50.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:50.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] boot 2023-07-13T22:10:50.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: osdmap e66: 10 total, 10 up, 10 in 2023-07-13T22:10:50.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:50.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:10:50.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi026 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi026 to 4318M 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] boot 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: osdmap e66: 10 total, 10 up, 10 in 2023-07-13T22:10:50.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:50.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:10:50.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:10:50.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi026 2023-07-13T22:10:50.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi026 to 4318M 2023-07-13T22:10:50.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-07-13T22:10:50.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: osd.9 [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] boot 2023-07-13T22:10:50.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: osdmap e66: 10 total, 10 up, 10 in 2023-07-13T22:10:50.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-13T22:10:51.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:51 smithi117 ceph-mon[129074]: osdmap e67: 10 total, 10 up, 10 in 2023-07-13T22:10:51.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:51.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:51.924 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:51.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:51 smithi026 ceph-mon[95647]: osdmap e67: 10 total, 10 up, 10 in 2023-07-13T22:10:51.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:51.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:51.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:51.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:51 smithi121 ceph-mon[129187]: osdmap e67: 10 total, 10 up, 10 in 2023-07-13T22:10:51.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:10:51.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:10:51.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:10:52.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:52 smithi117 ceph-mon[129074]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:52 smithi117 ceph-mon[129074]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:52 smithi026 ceph-mon[95647]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:52 smithi026 ceph-mon[95647]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:52 smithi121 ceph-mon[129187]: from='client.24374 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:10:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:52 smithi121 ceph-mon[129187]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:53.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:53 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/3555110808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:53 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:53 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]': finished 2023-07-13T22:10:53.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:53 smithi117 ceph-mon[129074]: osdmap e68: 11 total, 10 up, 11 in 2023-07-13T22:10:53.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:10:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:53 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/3555110808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:53 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:53 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]': finished 2023-07-13T22:10:53.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:53 smithi026 ceph-mon[95647]: osdmap e68: 11 total, 10 up, 11 in 2023-07-13T22:10:53.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:10:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:53 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/3555110808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:53 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]: dispatch 2023-07-13T22:10:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:53 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a52c43f1-5b23-401c-bea3-05c5bacc3ca7"}]': finished 2023-07-13T22:10:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:53 smithi121 ceph-mon[129187]: osdmap e68: 11 total, 10 up, 11 in 2023-07-13T22:10:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:10:54.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:54 smithi117 ceph-mon[129074]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:54.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:54 smithi117 ceph-mon[129074]: from='client.? 172.21.15.117:0/1459464707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:54.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:54 smithi026 ceph-mon[95647]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:54.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:54 smithi026 ceph-mon[95647]: from='client.? 172.21.15.117:0/1459464707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:54 smithi121 ceph-mon[129187]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:54 smithi121 ceph-mon[129187]: from='client.? 172.21.15.117:0/1459464707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:10:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:56 smithi117 ceph-mon[129074]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:56.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:56 smithi026 ceph-mon[95647]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:56.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:56 smithi121 ceph-mon[129187]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:58.849 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:10:58 smithi117 ceph-mon[129074]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:58.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:10:58 smithi026 ceph-mon[95647]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:10:58.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:10:58 smithi121 ceph-mon[129187]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:00.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:00 smithi117 ceph-mon[129074]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-13T22:11:00.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:00 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:00.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:00 smithi026 ceph-mon[95647]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:00.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-13T22:11:00.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:00 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:00.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:00 smithi121 ceph-mon[129187]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:00.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-13T22:11:00.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:00 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:01.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:01 smithi117 ceph-mon[129074]: Deploying daemon osd.10 on smithi117 2023-07-13T22:11:01.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:01 smithi026 ceph-mon[95647]: Deploying daemon osd.10 on smithi117 2023-07-13T22:11:01.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:01 smithi121 ceph-mon[129187]: Deploying daemon osd.10 on smithi117 2023-07-13T22:11:02.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:02 smithi117 ceph-mon[129074]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:02.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:02 smithi026 ceph-mon[95647]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:02.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:02 smithi121 ceph-mon[129187]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:04.850 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:04 smithi117 ceph-mon[129074]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:04.850 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.850 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.850 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:04 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:04.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:04 smithi026 ceph-mon[95647]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:04.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:04 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:04.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:04 smithi121 ceph-mon[129187]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:04.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:04.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:04 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:06.748 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.756+0000 7f53adffb700 1 -- 172.21.15.117:0/1690381711 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f53b0060950 con 0x7f539c0a1150 2023-07-13T22:11:06.749 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 10 on host 'smithi117' 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 msgr2=0x7f539c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 0x7f539c0a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53a0006010 tx=0x7f53a000b260 comp rx=0 tx=0).stop 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 msgr2=0x7f53b0195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f539800e9a0 tx=0x7f539800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 shutdown_connections 2023-07-13T22:11:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f539c0a1150 0x7f539c0a3600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b0100610 0x7f53b0195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53b0105850 0x7f53b019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 --2- 172.21.15.117:0/1690381711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f53b0102f30 0x7f53b0195e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 >> 172.21.15.117:0/1690381711 conn(0x7f53b00fa0b0 msgr2=0x7f53b0101310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 shutdown_connections 2023-07-13T22:11:06.752 INFO:teuthology.orchestra.run.smithi117.stderr:2023-07-13T22:11:06.758+0000 7f53b6b8e700 1 -- 172.21.15.117:0/1690381711 wait complete. 2023-07-13T22:11:06.876 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:06 smithi117 ceph-mon[129074]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:06.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:06 smithi026 ceph-mon[95647]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:06.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:06 smithi121 ceph-mon[129187]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:07.070 DEBUG:teuthology.orchestra.run.smithi117:osd.10> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.10.service 2023-07-13T22:11:07.072 INFO:tasks.cephadm:Deploying osd.11 on smithi121 with /dev/vg_nvme/lv_1... 2023-07-13T22:11:07.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-13T22:11:07.104 INFO:journalctl@ceph.osd.10.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:11:07.640 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:11:07.886 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.886 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.887 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.897 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.898 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:07.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:07.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:08.196 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:11:08 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:11:08.005+0000 7f6c052ac640 -1 osd.10 0 log_to_monitors true 2023-07-13T22:11:08.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:08 smithi117 ceph-mon[129074]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:08.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:08 smithi117 ceph-mon[129074]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:08.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:08 smithi117 ceph-mon[129074]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:08.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:08 smithi026 ceph-mon[95647]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:08.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:08 smithi026 ceph-mon[95647]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:08.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:08 smithi026 ceph-mon[95647]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:08.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:08 smithi121 ceph-mon[129187]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:08.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:08 smithi121 ceph-mon[129187]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:08.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:08 smithi121 ceph-mon[129187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-13T22:11:09.124 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-13T22:11:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-13T22:11:09.125 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10+0 records in 2023-07-13T22:11:09.125 INFO:teuthology.orchestra.run.smithi121.stderr:10+0 records out 2023-07-13T22:11:09.126 INFO:teuthology.orchestra.run.smithi121.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010884 s, 963 MB/s 2023-07-13T22:11:09.126 INFO:teuthology.orchestra.run.smithi121.stderr:--> Zapping successful for: 2023-07-13T22:11:09.597 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch daemon add osd smithi121:vg_nvme/lv_1 2023-07-13T22:11:09.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-13T22:11:09.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: osdmap e69: 11 total, 10 up, 11 in 2023-07-13T22:11:09.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:09.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:09.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi117 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:09.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:09.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi117 to 4292M 2023-07-13T22:11:09.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:09.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:09.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:09.999 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.000 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:09 smithi121 ceph-mon[129187]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:10.149 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:11:10.196 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:11:09.733+0000 7f6bfa123700 -1 osd.10 0 waiting for initial osdmap 2023-07-13T22:11:10.197 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:11:09.745+0000 7f6bf4715700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:11:10.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-13T22:11:10.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: osdmap e69: 11 total, 10 up, 11 in 2023-07-13T22:11:10.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:10.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:10.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:10.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi117 2023-07-13T22:11:10.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.199 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.200 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi117 to 4292M 2023-07-13T22:11:10.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:10.201 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:10.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.202 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:09 smithi117 ceph-mon[129074]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:10.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-13T22:11:10.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: osdmap e69: 11 total, 10 up, 11 in 2023-07-13T22:11:10.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:10.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:10.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2023-07-13T22:11:10.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi117 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi117 to 4292M 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:10.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:09 smithi026 ceph-mon[95647]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:10.968 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:10 smithi121 ceph-mon[129187]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:11:10.968 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:10 smithi121 ceph-mon[129187]: osdmap e70: 11 total, 10 up, 11 in 2023-07-13T22:11:10.969 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:10.969 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:10 smithi117 ceph-mon[129074]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:11:11.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:10 smithi117 ceph-mon[129074]: osdmap e70: 11 total, 10 up, 11 in 2023-07-13T22:11:11.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:10 smithi026 ceph-mon[95647]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2023-07-13T22:11:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:10 smithi026 ceph-mon[95647]: osdmap e70: 11 total, 10 up, 11 in 2023-07-13T22:11:11.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 -- 172.21.15.121:0/4118023033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368100610 msgr2=0x7fb368106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:11.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 --2- 172.21.15.121:0/4118023033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368100610 0x7fb368106ed0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb36000b0a0 tx=0x7fb36000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 -- 172.21.15.121:0/4118023033 shutdown_connections 2023-07-13T22:11:11.337 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 --2- 172.21.15.121:0/4118023033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb368100610 0x7fb368106ed0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 --2- 172.21.15.121:0/4118023033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3680fea80 0x7fb3680fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 --2- 172.21.15.121:0/4118023033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680ffc80 0x7fb3681000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 -- 172.21.15.121:0/4118023033 >> 172.21.15.121:0/4118023033 conn(0x7fb3680fa0b0 msgr2=0x7fb3680fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 -- 172.21.15.121:0/4118023033 shutdown_connections 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.343+0000 7fb36d406700 1 -- 172.21.15.121:0/4118023033 wait complete. 2023-07-13T22:11:11.338 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 Processor -- start 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 -- start start 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680fea80 0x7fb368195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb368100610 0x7fb36819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36819d620 con 0x7fb3680fea80 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36819d760 con 0x7fb368100610 2023-07-13T22:11:11.339 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb36d406700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36819d8f0 con 0x7fb3680ffc80 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:40928/0 (socket says 172.21.15.121:40928) 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 -- 172.21.15.121:0/1217296371 learned_addr learned my addr 172.21.15.121:0/1217296371 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3677fe700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb368100610 0x7fb36819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb366ffd700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680fea80 0x7fb368195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 -- 172.21.15.121:0/1217296371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb368100610 msgr2=0x7fb36819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:11.340 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb368100610 0x7fb36819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 -- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680fea80 msgr2=0x7fb368195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.344+0000 7fb3667fc700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680fea80 0x7fb368195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb3667fc700 1 -- 172.21.15.121:0/1217296371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb360009d00 con 0x7fb3680ffc80 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb3667fc700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb35800b560 tx=0x7fb35800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358011830 con 0x7fb3680ffc80 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36819dbd0 con 0x7fb3680ffc80 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb358011e50 con 0x7fb3680ffc80 2023-07-13T22:11:11.341 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358010e50 con 0x7fb3680ffc80 2023-07-13T22:11:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.345+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36819e120 con 0x7fb3680ffc80 2023-07-13T22:11:11.342 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.346+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36810a360 con 0x7fb3680ffc80 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.347+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3580104f0 con 0x7fb3680ffc80 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.348+0000 7fb34ffff700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 0x7fb3500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.348+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fb3580cc290 con 0x7fb3680ffc80 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.348+0000 7fb366ffd700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 0x7fb3500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.349+0000 7fb366ffd700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 0x7fb3500a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ffae0 tx=0x7fb35c009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:11.343 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.351+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb358095f70 con 0x7fb3680ffc80 2023-07-13T22:11:11.535 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:11.543+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb368068720 con 0x7fb3500a1150 2023-07-13T22:11:11.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:11:11.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:11:11.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] boot 2023-07-13T22:11:11.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: osdmap e71: 11 total, 11 up, 11 in 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:11:11.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:11 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:12.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:11:12.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:11:12.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] boot 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: osdmap e71: 11 total, 11 up, 11 in 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:11:12.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:11:12.198 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:11 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:12.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: osd.10 [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] boot 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: osdmap e71: 11 total, 11 up, 11 in 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-07-13T22:11:12.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:12.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-13T22:11:12.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-13T22:11:12.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:11 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:12.931 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:12 smithi121 ceph-mon[129187]: osdmap e72: 11 total, 11 up, 11 in 2023-07-13T22:11:13.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:12 smithi117 ceph-mon[129074]: osdmap e72: 11 total, 11 up, 11 in 2023-07-13T22:11:13.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:12 smithi026 ceph-mon[95647]: osdmap e72: 11 total, 11 up, 11 in 2023-07-13T22:11:14.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:13 smithi117 ceph-mon[129074]: osdmap e73: 11 total, 11 up, 11 in 2023-07-13T22:11:14.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:13 smithi117 ceph-mon[129074]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:14.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:13 smithi026 ceph-mon[95647]: osdmap e73: 11 total, 11 up, 11 in 2023-07-13T22:11:14.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:13 smithi026 ceph-mon[95647]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:14.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:13 smithi121 ceph-mon[129187]: osdmap e73: 11 total, 11 up, 11 in 2023-07-13T22:11:14.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:13 smithi121 ceph-mon[129187]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:15.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: osdmap e74: 11 total, 11 up, 11 in 2023-07-13T22:11:15.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/3290677083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]': finished 2023-07-13T22:11:15.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: osdmap e75: 12 total, 11 up, 12 in 2023-07-13T22:11:15.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:15.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:14 smithi117 ceph-mon[129074]: from='client.? 172.21.15.121:0/3960351670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:11:15.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: osdmap e74: 11 total, 11 up, 11 in 2023-07-13T22:11:15.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/3290677083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]': finished 2023-07-13T22:11:15.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: osdmap e75: 12 total, 11 up, 12 in 2023-07-13T22:11:15.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:15.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:14 smithi026 ceph-mon[95647]: from='client.? 172.21.15.121:0/3960351670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: osdmap e74: 11 total, 11 up, 11 in 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/3290677083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]: dispatch 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43cbee7d-bb99-429b-a99a-102d7497f251"}]': finished 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: osdmap e75: 12 total, 11 up, 12 in 2023-07-13T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:15.248 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:14 smithi121 ceph-mon[129187]: from='client.? 172.21.15.121:0/3960351670' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-13T22:11:16.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:15 smithi117 ceph-mon[129074]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:11:16.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:15 smithi026 ceph-mon[95647]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:11:16.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:15 smithi121 ceph-mon[129187]: pgmap v227: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-13T22:11:18.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:18 smithi026 ceph-mon[95647]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-07-13T22:11:18.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:18 smithi121 ceph-mon[129187]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-07-13T22:11:18.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:18 smithi117 ceph-mon[129074]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2023-07-13T22:11:20.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:20 smithi026 ceph-mon[95647]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-07-13T22:11:20.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:20 smithi121 ceph-mon[129187]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-07-13T22:11:20.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:20 smithi117 ceph-mon[129074]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-07-13T22:11:21.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-13T22:11:21.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:21 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:21.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:21 smithi121 ceph-mon[129187]: Deploying daemon osd.11 on smithi121 2023-07-13T22:11:21.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-13T22:11:21.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:21 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:21.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:21 smithi117 ceph-mon[129074]: Deploying daemon osd.11 on smithi121 2023-07-13T22:11:21.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-13T22:11:21.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:21 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:21.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:21 smithi026 ceph-mon[95647]: Deploying daemon osd.11 on smithi121 2023-07-13T22:11:22.645 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:22 smithi121 ceph-mon[129187]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:11:22.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:22 smithi117 ceph-mon[129074]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:11:22.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:22 smithi026 ceph-mon[95647]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-13T22:11:24.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:24 smithi121 ceph-mon[129187]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:24 smithi117 ceph-mon[129074]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:24 smithi026 ceph-mon[95647]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:25.974 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:25 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:25.974 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:25 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:25.974 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:25 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:25.974 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:25 smithi121 ceph-mon[129187]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:26.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:25 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:26.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:25 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:26.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:25 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:26.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:25 smithi117 ceph-mon[129074]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:25 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:25 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:25 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:25 smithi026 ceph-mon[95647]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:27.452 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.459+0000 7fb34ffff700 1 -- 172.21.15.121:0/1217296371 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb368068720 con 0x7fb3500a1150 2023-07-13T22:11:27.453 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 11 on host 'smithi121' 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 msgr2=0x7fb3500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 0x7fb3500a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3680ffae0 tx=0x7fb35c009190 comp rx=0 tx=0).stop 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 msgr2=0x7fb368195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb35800b560 tx=0x7fb35800b870 comp rx=0 tx=0).stop 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 shutdown_connections 2023-07-13T22:11:27.455 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3500a1150 0x7fb3500a3600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:27.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3680fea80 0x7fb368195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:27.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb368100610 0x7fb36819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:27.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 --2- 172.21.15.121:0/1217296371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3680ffc80 0x7fb368195fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:27.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 >> 172.21.15.121:0/1217296371 conn(0x7fb3680fa0b0 msgr2=0x7fb368068010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:27.456 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 shutdown_connections 2023-07-13T22:11:27.457 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:27.462+0000 7fb36d406700 1 -- 172.21.15.121:0/1217296371 wait complete. 2023-07-13T22:11:27.832 DEBUG:teuthology.orchestra.run.smithi121:osd.11> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.11.service 2023-07-13T22:11:27.834 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-13T22:11:27.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd stat -f json 2023-07-13T22:11:27.865 INFO:journalctl@ceph.osd.11.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:11:28.579 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:11:28.656 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-mon[129187]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:28.656 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.657 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.657 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.657 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:28 smithi117 ceph-mon[129074]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:28.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:28 smithi026 ceph-mon[95647]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:28.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:28.997 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:11:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:11:28.741+0000 7f8c68a92640 -1 osd.11 0 log_to_monitors true 2023-07-13T22:11:29.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:29 smithi117 ceph-mon[129074]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:29 smithi117 ceph-mon[129074]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:29 smithi026 ceph-mon[95647]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:29 smithi026 ceph-mon[95647]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:29 smithi121 ceph-mon[129187]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:29 smithi121 ceph-mon[129187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-13T22:11:29.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 -- 172.21.15.26:0/2630420635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39580ffb90 msgr2=0x7f39580fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 --2- 172.21.15.26:0/2630420635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39580ffb90 0x7f39580fffa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3948009990 tx=0x7f3948009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 -- 172.21.15.26:0/2630420635 shutdown_connections 2023-07-13T22:11:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 --2- 172.21.15.26:0/2630420635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3958101720 0x7f3958107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 --2- 172.21.15.26:0/2630420635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39580ffb90 0x7f39580fffa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 --2- 172.21.15.26:0/2630420635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958100d90 0x7f39581011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 -- 172.21.15.26:0/2630420635 >> 172.21.15.26:0/2630420635 conn(0x7f39580fb1a0 msgr2=0x7f39580fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 -- 172.21.15.26:0/2630420635 shutdown_connections 2023-07-13T22:11:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.959+0000 7f395e63e700 1 -- 172.21.15.26:0/2630420635 wait complete. 2023-07-13T22:11:29.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 Processor -- start 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 -- start start 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3958100d90 0x7f3958195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958101720 0x7f395819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395819d480 con 0x7f3958101720 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395819d5c0 con 0x7f39580ffb90 2023-07-13T22:11:29.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.961+0000 7f395e63e700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395819d750 con 0x7f3958100d90 2023-07-13T22:11:29.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:29.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:33186/0 (socket says 172.21.15.26:33186) 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 -- 172.21.15.26:0/179441726 learned_addr learned my addr 172.21.15.26:0/179441726 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f39577fe700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3958100d90 0x7f3958195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f395cbdb700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958101720 0x7f395819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 -- 172.21.15.26:0/179441726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3958100d90 msgr2=0x7f3958195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3958100d90 0x7f3958195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 -- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958101720 msgr2=0x7f395819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:29.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958101720 0x7f395819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 -- 172.21.15.26:0/179441726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480095f0 con 0x7f39580ffb90 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.962+0000 7f3957fff700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3948004710 tx=0x7f3948009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.963+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394801d070 con 0x7f39580ffb90 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.963+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3948021410 con 0x7f39580ffb90 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.963+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395819d9d0 con 0x7f39580ffb90 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.963+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948017650 con 0x7f39580ffb90 2023-07-13T22:11:29.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.963+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395819dde0 con 0x7f39580ffb90 2023-07-13T22:11:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.964+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3958107170 con 0x7f39580ffb90 2023-07-13T22:11:29.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.966+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3948021580 con 0x7f39580ffb90 2023-07-13T22:11:29.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.968+0000 7f39557fa700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 0x7f39400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:29.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.968+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f39480cde40 con 0x7f39580ffb90 2023-07-13T22:11:29.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.968+0000 7f39577fe700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 0x7f39400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:29.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.969+0000 7f39577fe700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 0x7f39400a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f394c009c60 tx=0x7f394c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:29.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:29.969+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3948097b50 con 0x7f39580ffb90 2023-07-13T22:11:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.178+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f395802ca20 con 0x7f39580ffb90 2023-07-13T22:11:30.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.178+0000 7f39557fa700 1 -- 172.21.15.26:0/179441726 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f394809b170 con 0x7f39580ffb90 2023-07-13T22:11:30.171 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 msgr2=0x7f39400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 0x7f39400a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f394c009c60 tx=0x7f394c009260 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 msgr2=0x7f39581958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3948004710 tx=0x7f3948009580 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 shutdown_connections 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f39400a1150 0x7f39400a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3958101720 0x7f395819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39580ffb90 0x7f39581958f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 --2- 172.21.15.26:0/179441726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3958100d90 0x7f3958195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 >> 172.21.15.26:0/179441726 conn(0x7f39580fb1a0 msgr2=0x7f39581042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 shutdown_connections 2023-07-13T22:11:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:30.181+0000 7f395e63e700 1 -- 172.21.15.26:0/179441726 wait complete. 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: osdmap e76: 12 total, 11 up, 12 in 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/179441726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:30 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.660 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1689286270,"num_in_osds":12,"osd_in_since":1689286273,"num_remapped_pgs":0} 2023-07-13T22:11:30.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:30.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-13T22:11:30.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: osdmap e76: 12 total, 11 up, 12 in 2023-07-13T22:11:30.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:30.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/179441726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.698 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.699 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:30 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.746 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:11:30.440+0000 7f8c5d909700 -1 osd.11 0 waiting for initial osdmap 2023-07-13T22:11:30.746 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:11:30.455+0000 7f8c57efb700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-13T22:11:30.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:30.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-13T22:11:30.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: osdmap e76: 12 total, 11 up, 12 in 2023-07-13T22:11:30.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:30.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:11:30.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/179441726' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:30.749 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:30.750 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:30 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:31.661 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd stat -f json 2023-07-13T22:11:31.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: Detected new or changed devices on smithi121 2023-07-13T22:11:31.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi121 to 4292M 2023-07-13T22:11:31.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:11:31.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: osdmap e77: 12 total, 11 up, 12 in 2023-07-13T22:11:31.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:31.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:31 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:31.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: Detected new or changed devices on smithi121 2023-07-13T22:11:31.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi121 to 4292M 2023-07-13T22:11:31.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:11:31.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: osdmap e77: 12 total, 11 up, 12 in 2023-07-13T22:11:31.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:31.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:31.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: Detected new or changed devices on smithi121 2023-07-13T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi121 to 4292M 2023-07-13T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2023-07-13T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: osdmap e77: 12 total, 11 up, 12 in 2023-07-13T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:31 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:32.405 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:11:32.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: purged_snaps scrub starts 2023-07-13T22:11:32.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: purged_snaps scrub ok 2023-07-13T22:11:32.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:32.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] boot 2023-07-13T22:11:32.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: osdmap e78: 12 total, 12 up, 12 in 2023-07-13T22:11:32.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:32 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:32.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: purged_snaps scrub starts 2023-07-13T22:11:32.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: purged_snaps scrub ok 2023-07-13T22:11:32.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:32.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] boot 2023-07-13T22:11:32.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: osdmap e78: 12 total, 12 up, 12 in 2023-07-13T22:11:32.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:32 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:32.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: purged_snaps scrub starts 2023-07-13T22:11:32.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: purged_snaps scrub ok 2023-07-13T22:11:32.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-07-13T22:11:32.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: osd.11 [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] boot 2023-07-13T22:11:32.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: osdmap e78: 12 total, 12 up, 12 in 2023-07-13T22:11:32.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:32 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-13T22:11:33.529 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:33 smithi026 ceph-mon[95647]: osdmap e79: 12 total, 12 up, 12 in 2023-07-13T22:11:33.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:33 smithi121 ceph-mon[129187]: osdmap e79: 12 total, 12 up, 12 in 2023-07-13T22:11:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.791+0000 7ffac6e7a700 1 -- 172.21.15.26:0/3090707218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 msgr2=0x7ffac01011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.791+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/3090707218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac01011e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009990 tx=0x7ffaac009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 -- 172.21.15.26:0/3090707218 shutdown_connections 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/3090707218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 0x7ffac0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/3090707218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 0x7ffac00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/3090707218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac01011e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 -- 172.21.15.26:0/3090707218 >> 172.21.15.26:0/3090707218 conn(0x7ffac00fb1a0 msgr2=0x7ffac00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 -- 172.21.15.26:0/3090707218 shutdown_connections 2023-07-13T22:11:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.792+0000 7ffac6e7a700 1 -- 172.21.15.26:0/3090707218 wait complete. 2023-07-13T22:11:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 Processor -- start 2023-07-13T22:11:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 -- start start 2023-07-13T22:11:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 0x7ffac0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 0x7ffac019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac019f680 con 0x7ffac0100d90 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac019f7c0 con 0x7ffac0101720 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac6e7a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac019f950 con 0x7ffac00ffb90 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffabffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.793+0000 7ffac4c16700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 0x7ffac0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60688/0 (socket says 172.21.15.26:60688) 2023-07-13T22:11:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 -- 172.21.15.26:0/1054905166 learned_addr learned my addr 172.21.15.26:0/1054905166 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffac5417700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 0x7ffac019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 -- 172.21.15.26:0/1054905166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 msgr2=0x7ffac0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 0x7ffac0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 -- 172.21.15.26:0/1054905166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 msgr2=0x7ffac019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 0x7ffac019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 -- 172.21.15.26:0/1054905166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0095f0 con 0x7ffac0100d90 2023-07-13T22:11:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabffff700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005910 tx=0x7ffaac00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac01d070 con 0x7ffac0100d90 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac021410 con 0x7ffac0100d90 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac017650 con 0x7ffac0100d90 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.794+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac019fbd0 con 0x7ffac0100d90 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.795+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac01a0020 con 0x7ffac0100d90 2023-07-13T22:11:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.796+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac0192210 con 0x7ffac0100d90 2023-07-13T22:11:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.798+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffaac021580 con 0x7ffac0100d90 2023-07-13T22:11:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.799+0000 7ffabdffb700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 0x7ffab00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:33.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.799+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ffaac0cd890 con 0x7ffac0100d90 2023-07-13T22:11:33.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.799+0000 7ffac4c16700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 0x7ffab00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:33.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.800+0000 7ffac4c16700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 0x7ffab00a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffac0100bf0 tx=0x7ffab400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:33.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:33.803+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffaac097470 con 0x7ffac0100d90 2023-07-13T22:11:33.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:33 smithi117 ceph-mon[129074]: osdmap e79: 12 total, 12 up, 12 in 2023-07-13T22:11:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.012+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ffac002c850 con 0x7ffac0100d90 2023-07-13T22:11:34.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.013+0000 7ffabdffb700 1 -- 172.21.15.26:0/1054905166 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7ffaac09aa90 con 0x7ffac0100d90 2023-07-13T22:11:34.005 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:34.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 msgr2=0x7ffab00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 0x7ffab00a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffac0100bf0 tx=0x7ffab400b410 comp rx=0 tx=0).stop 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 msgr2=0x7ffac0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffaac005910 tx=0x7ffaac00f460 comp rx=0 tx=0).stop 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 shutdown_connections 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffac0101720 0x7ffac019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:34.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffac00ffb90 0x7ffac0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffab00a11d0 0x7ffab00a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 --2- 172.21.15.26:0/1054905166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffac0100d90 0x7ffac0198030 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 >> 172.21.15.26:0/1054905166 conn(0x7ffac00fb1a0 msgr2=0x7ffac01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 shutdown_connections 2023-07-13T22:11:34.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:34.015+0000 7ffac6e7a700 1 -- 172.21.15.26:0/1054905166 wait complete. 2023-07-13T22:11:34.487 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1689286291,"num_in_osds":12,"osd_in_since":1689286273,"num_remapped_pgs":0} 2023-07-13T22:11:34.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-13T22:11:34.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:34 smithi026 ceph-mon[95647]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:34.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:34 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1054905166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:34.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:34 smithi121 ceph-mon[129187]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:34.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:34 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1054905166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:34.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:34 smithi117 ceph-mon[129074]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:34.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:34 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1054905166' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-13T22:11:35.257 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:11:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.641+0000 7fe5a80f9700 1 -- 172.21.15.26:0/2538404950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 msgr2=0x7fe5a0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.641+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/2538404950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a0104770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe590009920 tx=0x7fe590009c30 comp rx=0 tx=0).stop 2023-07-13T22:11:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 -- 172.21.15.26:0/2538404950 shutdown_connections 2023-07-13T22:11:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/2538404950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 0x7fe5a010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/2538404950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 0x7fe5a01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/2538404950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a0104770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 -- 172.21.15.26:0/2538404950 >> 172.21.15.26:0/2538404950 conn(0x7fe5a00fb660 msgr2=0x7fe5a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 -- 172.21.15.26:0/2538404950 shutdown_connections 2023-07-13T22:11:36.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.642+0000 7fe5a80f9700 1 -- 172.21.15.26:0/2538404950 wait complete. 2023-07-13T22:11:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 Processor -- start 2023-07-13T22:11:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 -- start start 2023-07-13T22:11:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 0x7fe5a0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 0x7fe5a019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a019cf40 con 0x7fe5a01007a0 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a019d0b0 con 0x7fe5a00ffe00 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.643+0000 7fe5a80f9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a019d270 con 0x7fe5a0104ea0 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60710/0 (socket says 172.21.15.26:60710) 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 -- 172.21.15.26:0/1754451102 learned_addr learned my addr 172.21.15.26:0/1754451102 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5e95700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 0x7fe5a0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a6696700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 0x7fe5a019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 -- 172.21.15.26:0/1754451102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 msgr2=0x7fe5a019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 0x7fe5a019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 -- 172.21.15.26:0/1754451102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 msgr2=0x7fe5a0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 0x7fe5a0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 -- 172.21.15.26:0/1754451102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0095f0 con 0x7fe5a01007a0 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.644+0000 7fe5a5694700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe590009230 tx=0x7fe59000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.645+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59001c070 con 0x7fe5a01007a0 2023-07-13T22:11:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.645+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe59000baa0 con 0x7fe5a01007a0 2023-07-13T22:11:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.645+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900038e0 con 0x7fe5a01007a0 2023-07-13T22:11:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.645+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590009520 con 0x7fe5a01007a0 2023-07-13T22:11:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.645+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a019d720 con 0x7fe5a01007a0 2023-07-13T22:11:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.646+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a004e210 con 0x7fe5a01007a0 2023-07-13T22:11:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.649+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe590020030 con 0x7fe5a01007a0 2023-07-13T22:11:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.651+0000 7fe596ffd700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 0x7fe58c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.651+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe5900cd590 con 0x7fe5a01007a0 2023-07-13T22:11:36.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.651+0000 7fe5a5e95700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 0x7fe58c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:36.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.652+0000 7fe5a5e95700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 0x7fe58c0a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005fd0 tx=0x7fe59c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.653+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe590097170 con 0x7fe5a01007a0 2023-07-13T22:11:36.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:36 smithi026 ceph-mon[95647]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:36.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:36 smithi121 ceph-mon[129187]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.861+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5a01992e0 con 0x7fe5a01007a0 2023-07-13T22:11:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.863+0000 7fe596ffd700 1 -- 172.21.15.26:0/1754451102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18399 (secure 0 0 0) 0x7fe59009a790 con 0x7fe5a01007a0 2023-07-13T22:11:36.855 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:36.855 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:11:32.444721+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:11:36.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 msgr2=0x7fe58c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 0x7fe58c0a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe59c005fd0 tx=0x7fe59c009310 comp rx=0 tx=0).stop 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 msgr2=0x7fe5a01984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe590009230 tx=0x7fe59000f9e0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 shutdown_connections 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0104ea0 0x7fe5a019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a00ffe00 0x7fe5a0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe58c0a1150 0x7fe58c0a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 --2- 172.21.15.26:0/1754451102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a01007a0 0x7fe5a01984c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 >> 172.21.15.26:0/1754451102 conn(0x7fe5a00fb660 msgr2=0x7fe5a00fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 shutdown_connections 2023-07-13T22:11:36.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:36.865+0000 7fe5a80f9700 1 -- 172.21.15.26:0/1754451102 wait complete. 2023-07-13T22:11:36.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:36 smithi117 ceph-mon[129074]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:37.243 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:11:37.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-13T22:11:37.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:37 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1754451102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:37.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:37 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1754451102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:37.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:37 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1754451102' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:11:38.478 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:38 smithi026 ceph-mon[95647]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:38.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:38 smithi121 ceph-mon[129187]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:38.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:38 smithi117 ceph-mon[129074]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 -- 172.21.15.26:0/2925627960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0ffe00 msgr2=0x7ff44c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 --2- 172.21.15.26:0/2925627960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0ffe00 0x7ff44c1001d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0099c0 tx=0x7ff43c009cd0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 -- 172.21.15.26:0/2925627960 shutdown_connections 2023-07-13T22:11:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 --2- 172.21.15.26:0/2925627960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c0ffe00 0x7ff44c1001d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 --2- 172.21.15.26:0/2925627960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c104ea0 0x7ff44c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 --2- 172.21.15.26:0/2925627960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c1007a0 0x7ff44c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 -- 172.21.15.26:0/2925627960 >> 172.21.15.26:0/2925627960 conn(0x7ff44c0fb660 msgr2=0x7ff44c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 -- 172.21.15.26:0/2925627960 shutdown_connections 2023-07-13T22:11:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.270+0000 7ff4514c9700 1 -- 172.21.15.26:0/2925627960 wait complete. 2023-07-13T22:11:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.271+0000 7ff4514c9700 1 Processor -- start 2023-07-13T22:11:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.271+0000 7ff4514c9700 1 -- start start 2023-07-13T22:11:39.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 0x7ff44c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 0x7ff44c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 0x7ff44c0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1a4140 con 0x7ff44c104ea0 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1a42b0 con 0x7ff44c0ffe00 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff4514c9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1a4470 con 0x7ff44c1007a0 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 0x7ff44c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff44affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 0x7ff44c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff44b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 0x7ff44c0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff44a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 0x7ff44c072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35218/0 (socket says 172.21.15.26:35218) 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.272+0000 7ff44a7fc700 1 -- 172.21.15.26:0/1728651000 learned_addr learned my addr 172.21.15.26:0/1728651000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 -- 172.21.15.26:0/1728651000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 msgr2=0x7ff44c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 0x7ff44c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 -- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 msgr2=0x7ff44c0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 0x7ff44c0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 -- 172.21.15.26:0/1728651000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0095f0 con 0x7ff44c0ffe00 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44b7fe700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 0x7ff44c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff44affd700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 0x7ff44c072810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00b580 tx=0x7ff43c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c01d070 con 0x7ff44c0ffe00 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c1a46f0 con 0x7ff44c0ffe00 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff43c005020 con 0x7ff44c0ffe00 2023-07-13T22:11:39.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c004a90 con 0x7ff44c0ffe00 2023-07-13T22:11:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.273+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c1a4ab0 con 0x7ff44c0ffe00 2023-07-13T22:11:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.274+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c04e210 con 0x7ff44c0ffe00 2023-07-13T22:11:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.276+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff43c005400 con 0x7ff44c0ffe00 2023-07-13T22:11:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.278+0000 7ff433fff700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 0x7ff4340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.278+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff43c0cd830 con 0x7ff44c0ffe00 2023-07-13T22:11:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.278+0000 7ff44a7fc700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 0x7ff4340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.278+0000 7ff44a7fc700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 0x7ff4340a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff44c074000 tx=0x7ff44000b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.280+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff43c097410 con 0x7ff44c0ffe00 2023-07-13T22:11:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.490+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff44c1a4e30 con 0x7ff44c0ffe00 2023-07-13T22:11:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.491+0000 7ff433fff700 1 -- 172.21.15.26:0/1728651000 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff43c005190 con 0x7ff44c0ffe00 2023-07-13T22:11:39.484 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:11:39.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 msgr2=0x7ff4340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:39.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 0x7ff4340a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff44c074000 tx=0x7ff44000b480 comp rx=0 tx=0).stop 2023-07-13T22:11:39.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 msgr2=0x7ff44c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:39.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 0x7ff44c072810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff43c00b580 tx=0x7ff43c00f690 comp rx=0 tx=0).stop 2023-07-13T22:11:39.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 shutdown_connections 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff4340a1150 0x7ff4340a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff44c104ea0 0x7ff44c0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff44c0ffe00 0x7ff44c072810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 --2- 172.21.15.26:0/1728651000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff44c1007a0 0x7ff44c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 >> 172.21.15.26:0/1728651000 conn(0x7ff44c0fb660 msgr2=0x7ff44c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 shutdown_connections 2023-07-13T22:11:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:39.494+0000 7ff4514c9700 1 -- 172.21.15.26:0/1728651000 wait complete. 2023-07-13T22:11:39.958 INFO:tasks.cephadm:Adding mds.a on smithi026 2023-07-13T22:11:39.958 INFO:tasks.cephadm:Adding mds.d on smithi026 2023-07-13T22:11:39.958 INFO:tasks.cephadm:Adding mds.g on smithi026 2023-07-13T22:11:39.958 INFO:tasks.cephadm:Adding mds.j on smithi026 2023-07-13T22:11:39.959 INFO:tasks.cephadm:Adding mds.b on smithi117 2023-07-13T22:11:39.959 INFO:tasks.cephadm:Adding mds.e on smithi117 2023-07-13T22:11:39.959 INFO:tasks.cephadm:Adding mds.h on smithi117 2023-07-13T22:11:39.959 INFO:tasks.cephadm:Adding mds.k on smithi117 2023-07-13T22:11:39.960 INFO:tasks.cephadm:Adding mds.c on smithi121 2023-07-13T22:11:39.960 INFO:tasks.cephadm:Adding mds.f on smithi121 2023-07-13T22:11:39.960 INFO:tasks.cephadm:Adding mds.i on smithi121 2023-07-13T22:11:39.960 INFO:tasks.cephadm:Adding mds.l on smithi121 2023-07-13T22:11:39.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph orch apply mds all '12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l' 2023-07-13T22:11:40.526 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:11:40.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:40 smithi121 ceph-mon[129187]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:40.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:40 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1728651000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:40.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:40 smithi117 ceph-mon[129074]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:40.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:40 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1728651000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:40.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:40 smithi026 ceph-mon[95647]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:40.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:40 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1728651000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:41.793 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 -- 172.21.15.121:0/1518569299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e80ffc80 msgr2=0x7f43e81000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 --2- 172.21.15.121:0/1518569299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e80ffc80 0x7f43e81000d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009990 tx=0x7f43dc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 -- 172.21.15.121:0/1518569299 shutdown_connections 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 --2- 172.21.15.121:0/1518569299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e8100610 0x7f43e8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 --2- 172.21.15.121:0/1518569299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 0x7f43e80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 --2- 172.21.15.121:0/1518569299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e80ffc80 0x7f43e81000d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.794 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 -- 172.21.15.121:0/1518569299 >> 172.21.15.121:0/1518569299 conn(0x7f43e80fa0b0 msgr2=0x7f43e80fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:41.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 -- 172.21.15.121:0/1518569299 shutdown_connections 2023-07-13T22:11:41.795 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.801+0000 7f43eff6d700 1 -- 172.21.15.121:0/1518569299 wait complete. 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 Processor -- start 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 -- start start 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 0x7f43e81958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 0x7f43e8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:41.797 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e819d440 con 0x7f43e80ffc80 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e819d580 con 0x7f43e80fea80 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43eff6d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e819d710 con 0x7f43e8100610 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:41060/0 (socket says 172.21.15.121:41060) 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 -- 172.21.15.121:0/4011298116 learned_addr learned my addr 172.21.15.121:0/4011298116 (peer_addr_for_me v2:172.21.15.121:0/0) 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43edd09700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 0x7f43e81958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:41.798 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ed508700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 0x7f43e8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 -- 172.21.15.121:0/4011298116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 msgr2=0x7f43e81958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 0x7f43e81958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 -- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 msgr2=0x7f43e8195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 0x7f43e8195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ee50a700 1 -- 172.21.15.121:0/4011298116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0095f0 con 0x7f43e8100610 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.802+0000 7f43ed508700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 0x7f43e8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:11:41.799 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43ee50a700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b9a0 tx=0x7f43d800bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800c930 con 0x7f43e8100610 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43d8014440 con 0x7f43e8100610 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e819d9f0 con 0x7f43e8100610 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80075d0 con 0x7f43e8100610 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.803+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e819dea0 con 0x7f43e8100610 2023-07-13T22:11:41.800 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.804+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8105990 con 0x7f43e8100610 2023-07-13T22:11:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.805+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43d80145b0 con 0x7f43e8100610 2023-07-13T22:11:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.805+0000 7f43e6ffd700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 0x7f43d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:41.801 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.806+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43d8006e40 con 0x7f43e8100610 2023-07-13T22:11:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.806+0000 7f43edd09700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 0x7f43d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.806+0000 7f43edd09700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 0x7f43d00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ffae0 tx=0x7f43e000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:41.802 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:41.809+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f43d8095b50 con 0x7f43e8100610 2023-07-13T22:11:41.995 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.002+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f43e8060a50 con 0x7f43d00a1150 2023-07-13T22:11:42.004 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.012+0000 7f43e6ffd700 1 -- 172.21.15.121:0/4011298116 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f43e8060a50 con 0x7f43d00a1150 2023-07-13T22:11:42.004 INFO:teuthology.orchestra.run.smithi121.stdout:Scheduled mds.all update... 2023-07-13T22:11:42.006 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 msgr2=0x7f43d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 0x7f43d00a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ffae0 tx=0x7f43e000a380 comp rx=0 tx=0).stop 2023-07-13T22:11:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 msgr2=0x7f43e819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b9a0 tx=0x7f43d800bcb0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 shutdown_connections 2023-07-13T22:11:42.007 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43e8100610 0x7f43e819ce70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43e80fea80 0x7f43e81958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d00a1150 0x7f43d00a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 --2- 172.21.15.121:0/4011298116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43e80ffc80 0x7f43e8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 >> 172.21.15.121:0/4011298116 conn(0x7f43e80fa0b0 msgr2=0x7f43e80fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 shutdown_connections 2023-07-13T22:11:42.008 INFO:teuthology.orchestra.run.smithi121.stderr:2023-07-13T22:11:42.014+0000 7f43eff6d700 1 -- 172.21.15.121:0/4011298116 wait complete. 2023-07-13T22:11:42.369 DEBUG:teuthology.orchestra.run.smithi026:mds.a> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.a.service 2023-07-13T22:11:42.374 DEBUG:teuthology.orchestra.run.smithi026:mds.d> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.d.service 2023-07-13T22:11:42.421 DEBUG:teuthology.orchestra.run.smithi026:mds.g> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.g.service 2023-07-13T22:11:42.460 INFO:journalctl@ceph.mds.a.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:11:42.464 DEBUG:teuthology.orchestra.run.smithi026:mds.j> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.j.service 2023-07-13T22:11:42.498 INFO:journalctl@ceph.mds.d.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:11:42.501 DEBUG:teuthology.orchestra.run.smithi117:mds.b> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.b.service 2023-07-13T22:11:42.504 DEBUG:teuthology.orchestra.run.smithi117:mds.e> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.e.service 2023-07-13T22:11:42.540 INFO:journalctl@ceph.mds.g.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:11:42.540 INFO:journalctl@ceph.mds.b.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:11:42.542 DEBUG:teuthology.orchestra.run.smithi117:mds.h> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.h.service 2023-07-13T22:11:42.571 INFO:journalctl@ceph.mds.e.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:11:42.572 DEBUG:teuthology.orchestra.run.smithi117:mds.k> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.k.service 2023-07-13T22:11:42.573 INFO:journalctl@ceph.mds.j.smithi026.stdout:-- Logs begin at Thu 2023-07-13 21:47:13 UTC. -- 2023-07-13T22:11:42.599 INFO:journalctl@ceph.mds.h.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:11:42.600 DEBUG:teuthology.orchestra.run.smithi121:mds.c> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.c.service 2023-07-13T22:11:42.603 DEBUG:teuthology.orchestra.run.smithi121:mds.f> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.f.service 2023-07-13T22:11:42.626 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:42.626 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.626 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:42.626 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:42.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:42.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:42.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:42.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:42.628 INFO:journalctl@ceph.mds.k.smithi117.stdout:-- Logs begin at Thu 2023-07-13 21:40:17 UTC. -- 2023-07-13T22:11:42.628 DEBUG:teuthology.orchestra.run.smithi121:mds.i> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.i.service 2023-07-13T22:11:42.632 INFO:journalctl@ceph.mds.c.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:11:42.633 DEBUG:teuthology.orchestra.run.smithi121:mds.l> sudo journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.l.service 2023-07-13T22:11:42.655 INFO:journalctl@ceph.mds.f.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:11:42.656 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-13T22:11:42.656 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:11:42.660 INFO:journalctl@ceph.mds.i.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:11:42.683 INFO:journalctl@ceph.mds.l.smithi121.stdout:-- Logs begin at Thu 2023-07-13 21:43:57 UTC. -- 2023-07-13T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 -- 172.21.15.26:0/1938455374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec101680 msgr2=0x7febec107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 --2- 172.21.15.26:0/1938455374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec101680 0x7febec107f40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7febe400b0a0 tx=0x7febe400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 -- 172.21.15.26:0/1938455374 shutdown_connections 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 --2- 172.21.15.26:0/1938455374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec101680 0x7febec107f40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 --2- 172.21.15.26:0/1938455374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec0ffb00 0x7febec0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 --2- 172.21.15.26:0/1938455374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec100cf0 0x7febec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 -- 172.21.15.26:0/1938455374 >> 172.21.15.26:0/1938455374 conn(0x7febec0fb0e0 msgr2=0x7febec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 -- 172.21.15.26:0/1938455374 shutdown_connections 2023-07-13T22:11:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.877+0000 7febf1fff700 1 -- 172.21.15.26:0/1938455374 wait complete. 2023-07-13T22:11:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 Processor -- start 2023-07-13T22:11:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 -- start start 2023-07-13T22:11:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0ffb00 0x7febec197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 0x7febec198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec19f690 con 0x7febec100cf0 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec19f7d0 con 0x7febec0ffb00 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febf1fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec19f960 con 0x7febec101680 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.878+0000 7febebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febeb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0ffb00 0x7febec197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febeaffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 0x7febec198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35224/0 (socket says 172.21.15.26:35224) 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 -- 172.21.15.26:0/258353229 learned_addr learned my addr 172.21.15.26:0/258353229 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 -- 172.21.15.26:0/258353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0ffb00 msgr2=0x7febec197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0ffb00 0x7febec197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 -- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 msgr2=0x7febec198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 0x7febec198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febebfff700 1 -- 172.21.15.26:0/258353229 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe4009d00 con 0x7febec101680 2023-07-13T22:11:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.879+0000 7febeaffd700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 0x7febec198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.880+0000 7febebfff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febe400d010 tx=0x7febe400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.880+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe400e040 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.880+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe400bdb0 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.880+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4005260 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.880+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec19fbe0 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.881+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec19ffb0 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.881+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febec10b330 con 0x7febec101680 2023-07-13T22:11:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.884+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7febe401d480 con 0x7febec101680 2023-07-13T22:11:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.885+0000 7febe8ff9700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 0x7febd40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.885+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7febe40cd520 con 0x7febec101680 2023-07-13T22:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.885+0000 7febeb7fe700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 0x7febd40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.886+0000 7febeb7fe700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 0x7febd40a36d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7febec100b50 tx=0x7febdc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:42.887+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7febe4097100 con 0x7febec101680 2023-07-13T22:11:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:11:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:42.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:11:42.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:42.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:42.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:42.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:43.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.099+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febec1a05c0 con 0x7febec101680 2023-07-13T22:11:43.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.101+0000 7febe8ff9700 1 -- 172.21.15.26:0/258353229 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18399 (secure 0 0 0) 0x7febe409a720 con 0x7febec101680 2023-07-13T22:11:43.094 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:43.094 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:11:32.444721+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 msgr2=0x7febd40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 0x7febd40a36d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7febec100b50 tx=0x7febdc00b410 comp rx=0 tx=0).stop 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 msgr2=0x7febec19f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7febe400d010 tx=0x7febe400bc80 comp rx=0 tx=0).stop 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 shutdown_connections 2023-07-13T22:11:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febec101680 0x7febec19f0c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febec0ffb00 0x7febec197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd40a1220 0x7febd40a36d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 --2- 172.21.15.26:0/258353229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febec100cf0 0x7febec198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 >> 172.21.15.26:0/258353229 conn(0x7febec0fb0e0 msgr2=0x7febec106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.103+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 shutdown_connections 2023-07-13T22:11:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.104+0000 7febf1fff700 1 -- 172.21.15.26:0/258353229 wait complete. 2023-07-13T22:11:43.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:11:43.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:11:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.331+0000 7f067a5ff700 1 -- 172.21.15.26:0/2419641190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674074550 msgr2=0x7f06740729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.331+0000 7f067a5ff700 1 --2- 172.21.15.26:0/2419641190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674074550 0x7f06740729a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0664009990 tx=0x7f0664009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 -- 172.21.15.26:0/2419641190 shutdown_connections 2023-07-13T22:11:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 --2- 172.21.15.26:0/2419641190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06741079a0 0x7f0674109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 --2- 172.21.15.26:0/2419641190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674074550 0x7f06740729a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 --2- 172.21.15.26:0/2419641190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0674072f70 0x7f06740733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 -- 172.21.15.26:0/2419641190 >> 172.21.15.26:0/2419641190 conn(0x7f06740fb0e0 msgr2=0x7f06740fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 -- 172.21.15.26:0/2419641190 shutdown_connections 2023-07-13T22:11:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.332+0000 7f067a5ff700 1 -- 172.21.15.26:0/2419641190 wait complete. 2023-07-13T22:11:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 Processor -- start 2023-07-13T22:11:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 -- start start 2023-07-13T22:11:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674072f70 0x7f0674197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06741079a0 0x7f067419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f5d0 con 0x7f0674074550 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f710 con 0x7f06741079a0 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f067a5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067419f8a0 con 0x7f0674072f70 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f066bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f0673fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674072f70 0x7f0674197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f066bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57340/0 (socket says 172.21.15.26:57340) 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.333+0000 7f066bfff700 1 -- 172.21.15.26:0/3076681953 learned_addr learned my addr 172.21.15.26:0/3076681953 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f0678b9c700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06741079a0 0x7f067419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 -- 172.21.15.26:0/3076681953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674072f70 msgr2=0x7f0674197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674072f70 0x7f0674197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 -- 172.21.15.26:0/3076681953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06741079a0 msgr2=0x7f067419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06741079a0 0x7f067419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 -- 172.21.15.26:0/3076681953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06640095f0 con 0x7f0674074550 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f066bfff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e970 tx=0x7f065c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c009790 con 0x7f0674074550 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f065c00c7e0 con 0x7f0674074550 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067419fb80 con 0x7f0674074550 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c003c70 con 0x7f0674074550 2023-07-13T22:11:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.334+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06741a00e0 con 0x7f0674074550 2023-07-13T22:11:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.335+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0674106050 con 0x7f0674074550 2023-07-13T22:11:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.337+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f065c00cc50 con 0x7f0674074550 2023-07-13T22:11:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.339+0000 7f0671ffb700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 0x7f06600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.339+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f065c014070 con 0x7f0674074550 2023-07-13T22:11:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.339+0000 7f0673fff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 0x7f06600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.340+0000 7f0673fff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 0x7f06600a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f066400b680 tx=0x7f0664005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.343+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f065c096140 con 0x7f0674074550 2023-07-13T22:11:43.478 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:43 smithi121 systemd[1]: Starting Ceph mds.l for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.553+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06740611c0 con 0x7f0674074550 2023-07-13T22:11:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.553+0000 7f0671ffb700 1 -- 172.21.15.26:0/3076681953 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f065c003dd0 con 0x7f0674074550 2023-07-13T22:11:43.546 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:11:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 msgr2=0x7f06600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 0x7f06600a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f066400b680 tx=0x7f0664005fd0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 msgr2=0x7f0674197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f065c00e970 tx=0x7f065c00ec80 comp rx=0 tx=0).stop 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 shutdown_connections 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06741079a0 0x7f067419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0674072f70 0x7f0674197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06600a11d0 0x7f06600a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 --2- 172.21.15.26:0/3076681953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0674074550 0x7f0674197f90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 >> 172.21.15.26:0/3076681953 conn(0x7f06740fb0e0 msgr2=0x7f06740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 shutdown_connections 2023-07-13T22:11:43.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.556+0000 7f067a5ff700 1 -- 172.21.15.26:0/3076681953 wait complete. 2023-07-13T22:11:43.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:43.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-mon[129187]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:43.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-mon[129187]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2023-07-13T22:11:43.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-mon[129187]: Deploying daemon mds.l on smithi121 2023-07-13T22:11:43.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/258353229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 -- 172.21.15.26:0/1532660779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80ffb00 msgr2=0x7f0dc80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/1532660779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80ffb00 0x7f0dc80fff10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009990 tx=0x7f0dc4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 -- 172.21.15.26:0/1532660779 shutdown_connections 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/1532660779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc8101680 0x7f0dc8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/1532660779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc80ffb00 0x7f0dc80fff10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/1532660779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 -- 172.21.15.26:0/1532660779 >> 172.21.15.26:0/1532660779 conn(0x7f0dc80fb0e0 msgr2=0x7f0dc80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 -- 172.21.15.26:0/1532660779 shutdown_connections 2023-07-13T22:11:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.778+0000 7f0dd07ff700 1 -- 172.21.15.26:0/1532660779 wait complete. 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 Processor -- start 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 -- start start 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 0x7f0dc8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819f5d0 con 0x7f0dc8100cf0 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819f710 con 0x7f0dc80ffb00 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.779+0000 7f0dd07ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc819f8a0 con 0x7f0dc8101680 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dced9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dce59b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 0x7f0dc8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dced9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:45942/0 (socket says 172.21.15.26:45942) 2023-07-13T22:11:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dced9c700 1 -- 172.21.15.26:0/3146427873 learned_addr learned my addr 172.21.15.26:0/3146427873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 -- 172.21.15.26:0/3146427873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 msgr2=0x7f0dc819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 -- 172.21.15.26:0/3146427873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 msgr2=0x7f0dc8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 0x7f0dc8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 -- 172.21.15.26:0/3146427873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc40095f0 con 0x7f0dc8100cf0 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dce59b700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 0x7f0dc8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dced9c700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.780+0000 7f0dcdd9a700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8197f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0db800e9f0 tx=0x7f0db800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.781+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800c9c0 con 0x7f0dc8100cf0 2023-07-13T22:11:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.781+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc819fb80 con 0x7f0dc8100cf0 2023-07-13T22:11:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.781+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db8010430 con 0x7f0dc8100cf0 2023-07-13T22:11:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.781+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8018680 con 0x7f0dc8100cf0 2023-07-13T22:11:43.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.781+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc81a00e0 con 0x7f0dc8100cf0 2023-07-13T22:11:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.782+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc810b2c0 con 0x7f0dc8100cf0 2023-07-13T22:11:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.784+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0db8010860 con 0x7f0dc8100cf0 2023-07-13T22:11:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.785+0000 7f0dbf7fe700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 0x7f0db40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.785+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0db8014070 con 0x7f0dc8100cf0 2023-07-13T22:11:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.785+0000 7f0dce59b700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 0x7f0db40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.786+0000 7f0dce59b700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 0x7f0db40a3650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b2e0 tx=0x7f0dc4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:43.789+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0db8051790 con 0x7f0dc8100cf0 2023-07-13T22:11:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:43 smithi117 ceph-mon[129074]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:43 smithi117 ceph-mon[129074]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2023-07-13T22:11:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:43 smithi117 ceph-mon[129074]: Deploying daemon mds.l on smithi121 2023-07-13T22:11:43.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:43 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/258353229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:43.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:43 smithi026 ceph-mon[95647]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:11:43.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:43 smithi026 ceph-mon[95647]: Saving service mds.all spec with placement smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2023-07-13T22:11:43.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:43 smithi026 ceph-mon[95647]: Deploying daemon mds.l on smithi121 2023-07-13T22:11:43.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:43 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/258353229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:11:43.996 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:43 smithi121 podman[154220]: 2023-07-13T22:11:43.997 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-mds[154260]: -- 172.21.15.121:0/1172148226 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5629b5eb11e0 con 0x5629b5e87000 2023-07-13T22:11:43.997 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: starting mds.l at 2023-07-13T22:11:43.997 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:43 smithi121 bash[154220]: bc972ac8dbd48694894098dbab740bb795d601df598d0f9ea7804fe64216d06f 2023-07-13T22:11:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.048+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dc810b480 con 0x7f0dc8100cf0 2023-07-13T22:11:44.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.048+0000 7f0dbf7fe700 1 -- 172.21.15.26:0/3146427873 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f0db8096160 con 0x7f0dc8100cf0 2023-07-13T22:11:44.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:44.041 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-13T22:11:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 msgr2=0x7f0db40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 0x7f0db40a3650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400b2e0 tx=0x7f0dc4005fb0 comp rx=0 tx=0).stop 2023-07-13T22:11:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 msgr2=0x7f0dc8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8197f90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0db800e9f0 tx=0x7f0db800ed00 comp rx=0 tx=0).stop 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 shutdown_connections 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0dc8101680 0x7f0dc819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dc80ffb00 0x7f0dc8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0db40a11a0 0x7f0db40a3650 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 --2- 172.21.15.26:0/3146427873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0dc8100cf0 0x7f0dc8197f90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 >> 172.21.15.26:0/3146427873 conn(0x7f0dc80fb0e0 msgr2=0x7f0dc8106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 shutdown_connections 2023-07-13T22:11:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:44.051+0000 7f0dd07ff700 1 -- 172.21.15.26:0/3146427873 wait complete. 2023-07-13T22:11:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2023-07-13T22:11:44.479 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:11:44 smithi121 systemd[1]: Started Ceph mds.l for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:44.731 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:44.732 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3076681953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:44.732 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6832/930793918,v1:172.21.15.121:6833/930793918] up:boot 2023-07-13T22:11:44.732 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: fsmap 1 up:standby 2023-07-13T22:11:44.732 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:11:44.732 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3146427873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:44.733 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:44.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:44.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3076681953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:44.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6832/930793918,v1:172.21.15.121:6833/930793918] up:boot 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: fsmap 1 up:standby 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3146427873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:44.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:44.748 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:44.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:44.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3076681953' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:11:44.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6832/930793918,v1:172.21.15.121:6833/930793918] up:boot 2023-07-13T22:11:44.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: fsmap 1 up:standby 2023-07-13T22:11:44.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3146427873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:44.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:45.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:45 smithi117 ceph-mon[129074]: Deploying daemon mds.k on smithi117 2023-07-13T22:11:45.696 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:45 smithi117 systemd[1]: Starting Ceph mds.k for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:45.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:45 smithi121 ceph-mon[129187]: Deploying daemon mds.k on smithi117 2023-07-13T22:11:45.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:45 smithi026 ceph-mon[95647]: Deploying daemon mds.k on smithi117 2023-07-13T22:11:46.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:46.122 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:45 smithi117 podman[153257]: 2023-07-13T22:11:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3086615706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc102f10 msgr2=0x7f4acc103360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3086615706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc102f10 0x7f4acc103360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009990 tx=0x7f4abc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3086615706 shutdown_connections 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3086615706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc1038a0 0x7f4acc10a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3086615706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc101d20 0x7f4acc102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3086615706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc102f10 0x7f4acc103360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3086615706 >> 172.21.15.26:0/3086615706 conn(0x7f4acc0fd360 msgr2=0x7f4acc0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3086615706 shutdown_connections 2023-07-13T22:11:46.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.289+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3086615706 wait complete. 2023-07-13T22:11:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.290+0000 7f4ad45ff700 1 Processor -- start 2023-07-13T22:11:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 -- start start 2023-07-13T22:11:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 0x7f4acc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc1038a0 0x7f4acc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19f580 con 0x7f4acc1038a0 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19f6c0 con 0x7f4acc102f10 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad45ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19f850 con 0x7f4acc101d20 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad239b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 0x7f4acc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:47544/0 (socket says 172.21.15.26:47544) 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad1b9a700 1 -- 172.21.15.26:0/3774339925 learned_addr learned my addr 172.21.15.26:0/3774339925 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad239b700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 0x7f4acc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:45960/0 (socket says 172.21.15.26:45960) 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.291+0000 7f4ad2b9c700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc1038a0 0x7f4acc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 -- 172.21.15.26:0/3774339925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 msgr2=0x7f4acc197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 0x7f4acc197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 -- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc1038a0 msgr2=0x7f4acc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc1038a0 0x7f4acc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 -- 172.21.15.26:0/3774339925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc0095f0 con 0x7f4acc102f10 2023-07-13T22:11:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad1b9a700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc0047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc01d070 con 0x7f4acc102f10 2023-07-13T22:11:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4abc022470 con 0x7f4acc102f10 2023-07-13T22:11:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc017650 con 0x7f4acc102f10 2023-07-13T22:11:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc19fad0 con 0x7f4acc102f10 2023-07-13T22:11:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.292+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc19fea0 con 0x7f4acc102f10 2023-07-13T22:11:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.293+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc10d500 con 0x7f4acc102f10 2023-07-13T22:11:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.295+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4abc0225e0 con 0x7f4acc102f10 2023-07-13T22:11:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.298+0000 7f4ac37fe700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 0x7f4ab80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.298+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4abc0cda00 con 0x7f4acc102f10 2023-07-13T22:11:46.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.298+0000 7f4ad239b700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 0x7f4ab80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.299+0000 7f4ad239b700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 0x7f4ab80a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4acc102d70 tx=0x7f4ac8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.300+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4abc0975e0 con 0x7f4acc102f10 2023-07-13T22:11:46.445 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mds[153297]: -- 172.21.15.117:0/3058960834 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560313bd91e0 con 0x560313baf000 2023-07-13T22:11:46.446 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:46 smithi117 bash[153257]: f6f5dfedac8ca0eae4b06b564e1ebb8f9fc5334aadd131bb4c21380b14ccba38 2023-07-13T22:11:46.446 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: starting mds.k at 2023-07-13T22:11:46.446 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:11:46 smithi117 systemd[1]: Started Ceph mds.k for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:46.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.565+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4acc108ba0 con 0x7f4acc102f10 2023-07-13T22:11:46.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.565+0000 7f4ac37fe700 1 -- 172.21.15.26:0/3774339925 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f4abc09ac00 con 0x7f4acc102f10 2023-07-13T22:11:46.558 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:46.558 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-13T22:11:46.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 msgr2=0x7f4ab80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:46.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 0x7f4ab80a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4acc102d70 tx=0x7f4ac8009260 comp rx=0 tx=0).stop 2023-07-13T22:11:46.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 msgr2=0x7f4acc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc0047a0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 shutdown_connections 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ab80a1220 0x7f4ab80a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4acc1038a0 0x7f4acc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4acc101d20 0x7f4acc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 --2- 172.21.15.26:0/3774339925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4acc102f10 0x7f4acc197f40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 >> 172.21.15.26:0/3774339925 conn(0x7f4acc0fd360 msgr2=0x7f4acc0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:46.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 shutdown_connections 2023-07-13T22:11:46.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:46.568+0000 7f4ad45ff700 1 -- 172.21.15.26:0/3774339925 wait complete. 2023-07-13T22:11:46.562 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2023-07-13T22:11:46.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:46.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:46 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:46.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:46.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:46 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:46.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:46.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:46 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:47.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:47 smithi026 ceph-mon[95647]: Deploying daemon mds.j on smithi026 2023-07-13T22:11:47.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:47 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:boot 2023-07-13T22:11:47.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:47 smithi026 ceph-mon[95647]: fsmap 2 up:standby 2023-07-13T22:11:47.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:11:47.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:47 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3774339925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:47.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:47 smithi117 ceph-mon[129074]: Deploying daemon mds.j on smithi026 2023-07-13T22:11:47.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:47 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:boot 2023-07-13T22:11:47.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:47 smithi117 ceph-mon[129074]: fsmap 2 up:standby 2023-07-13T22:11:47.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:11:47.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:47 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3774339925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:47.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:47 smithi121 ceph-mon[129187]: Deploying daemon mds.j on smithi026 2023-07-13T22:11:47.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:47 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:boot 2023-07-13T22:11:47.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:47 smithi121 ceph-mon[129187]: fsmap 2 up:standby 2023-07-13T22:11:47.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:11:47.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:47 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3774339925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:48.388 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:48 smithi026 systemd[1]: Starting Ceph mds.j for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:48.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:48.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 -- 172.21.15.26:0/4136783793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8101680 msgr2=0x7f7eb8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:48.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/4136783793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8101680 0x7f7eb8107f40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000b0a0 tx=0x7f7eb000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 -- 172.21.15.26:0/4136783793 shutdown_connections 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/4136783793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8101680 0x7f7eb8107f40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/4136783793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/4136783793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 -- 172.21.15.26:0/4136783793 >> 172.21.15.26:0/4136783793 conn(0x7f7eb80fb0e0 msgr2=0x7f7eb80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 -- 172.21.15.26:0/4136783793 shutdown_connections 2023-07-13T22:11:48.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.815+0000 7f7ec03ff700 1 -- 172.21.15.26:0/4136783793 wait complete. 2023-07-13T22:11:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 Processor -- start 2023-07-13T22:11:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 -- start start 2023-07-13T22:11:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:48.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8101680 0x7f7eb819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819f6a0 con 0x7f7eb8100cf0 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819f7e0 con 0x7f7eb80ffb00 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.817+0000 7f7ec03ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819f970 con 0x7f7eb8101680 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebe19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57388/0 (socket says 172.21.15.26:57388) 2023-07-13T22:11:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 -- 172.21.15.26:0/1061159017 learned_addr learned my addr 172.21.15.26:0/1061159017 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebe99c700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8101680 0x7f7eb819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 -- 172.21.15.26:0/1061159017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8101680 msgr2=0x7f7eb819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8101680 0x7f7eb819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 -- 172.21.15.26:0/1061159017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 msgr2=0x7f7eb8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 -- 172.21.15.26:0/1061159017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4009520 con 0x7f7eb8100cf0 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebe19b700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7ebd99a700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d950 tx=0x7f7ea800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:48.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.818+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8009940 con 0x7f7eb8100cf0 2023-07-13T22:11:48.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.819+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ea8010460 con 0x7f7eb8100cf0 2023-07-13T22:11:48.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.819+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb0009d00 con 0x7f7eb8100cf0 2023-07-13T22:11:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.819+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800f620 con 0x7f7eb8100cf0 2023-07-13T22:11:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.819+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb81a0020 con 0x7f7eb8100cf0 2023-07-13T22:11:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.820+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb810b330 con 0x7f7eb8100cf0 2023-07-13T22:11:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.822+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ea80108d0 con 0x7f7eb8100cf0 2023-07-13T22:11:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.823+0000 7f7eaf7fe700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 0x7f7ea40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.824+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ea80cc2c0 con 0x7f7eb8100cf0 2023-07-13T22:11:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.824+0000 7f7ebe19b700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 0x7f7ea40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.824+0000 7f7ebe19b700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 0x7f7ea40a36d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009fd0 tx=0x7f7eb4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:48.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:48.826+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ea8095e10 con 0x7f7eb8100cf0 2023-07-13T22:11:48.873 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:48 smithi026 ceph-mon[95647]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:48.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:48 smithi117 ceph-mon[129074]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:48.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:48 smithi121 ceph-mon[129187]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:49.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.097+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7eb810b5c0 con 0x7f7eb8100cf0 2023-07-13T22:11:49.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.098+0000 7f7eaf7fe700 1 -- 172.21.15.26:0/1061159017 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2017 (secure 0 0 0) 0x7f7ea8016800 con 0x7f7eb8100cf0 2023-07-13T22:11:49.090 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:49.090 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 msgr2=0x7f7ea40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 0x7f7ea40a36d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009fd0 tx=0x7f7eb4009260 comp rx=0 tx=0).stop 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 msgr2=0x7f7eb8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d950 tx=0x7f7ea800dc60 comp rx=0 tx=0).stop 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 shutdown_connections 2023-07-13T22:11:49.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7eb8101680 0x7f7eb819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7eb80ffb00 0x7f7eb8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ea40a1220 0x7f7ea40a36d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 --2- 172.21.15.26:0/1061159017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7eb8100cf0 0x7f7eb8198060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 >> 172.21.15.26:0/1061159017 conn(0x7f7eb80fb0e0 msgr2=0x7f7eb80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 shutdown_connections 2023-07-13T22:11:49.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:49.101+0000 7f7ec03ff700 1 -- 172.21.15.26:0/1061159017 wait complete. 2023-07-13T22:11:49.095 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2023-07-13T22:11:49.170 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:48 smithi026 podman[129229]: 2023-07-13T22:11:49.170 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:49 smithi026 bash[129229]: bd91c73332787232f8d3992ccc420a3c86b55e2a0b77f5ca0d9c7519ab803ec1 2023-07-13T22:11:49.171 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mds[129289]: -- 172.21.15.26:0/1843556742 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563337ce51e0 con 0x563337cbb000 2023-07-13T22:11:49.171 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-j[129265]: starting mds.j at 2023-07-13T22:11:49.453 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:11:49 smithi026 systemd[1]: Started Ceph mds.j for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:49.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1061159017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:49.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:49 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:49.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1061159017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:49.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:49.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:49.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:49 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:49.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1061159017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:49.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:49.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:49 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:50.788 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:50 smithi026 ceph-mon[95647]: Deploying daemon mds.d on smithi026 2023-07-13T22:11:50.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:50 smithi026 ceph-mon[95647]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:50.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:50 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6834/956191234,v1:172.21.15.26:6835/956191234] up:boot 2023-07-13T22:11:50.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:50 smithi026 ceph-mon[95647]: fsmap 3 up:standby 2023-07-13T22:11:50.789 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-13T22:11:50.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:50 smithi117 ceph-mon[129074]: Deploying daemon mds.d on smithi026 2023-07-13T22:11:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:50 smithi117 ceph-mon[129074]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:50 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6834/956191234,v1:172.21.15.26:6835/956191234] up:boot 2023-07-13T22:11:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:50 smithi117 ceph-mon[129074]: fsmap 3 up:standby 2023-07-13T22:11:50.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-13T22:11:50.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:50 smithi121 ceph-mon[129187]: Deploying daemon mds.d on smithi026 2023-07-13T22:11:50.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:50 smithi121 ceph-mon[129187]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:50.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:50 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6834/956191234,v1:172.21.15.26:6835/956191234] up:boot 2023-07-13T22:11:50.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:50 smithi121 ceph-mon[129187]: fsmap 3 up:standby 2023-07-13T22:11:50.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-13T22:11:51.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 -- 172.21.15.26:0/4190663744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 msgr2=0x7fd104107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:51.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 --2- 172.21.15.26:0/4190663744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104107f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b0a0 tx=0x7fd0fc00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 -- 172.21.15.26:0/4190663744 shutdown_connections 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 --2- 172.21.15.26:0/4190663744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104107f40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 --2- 172.21.15.26:0/4190663744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1040ffb00 0x7fd1040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 --2- 172.21.15.26:0/4190663744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd104100cf0 0x7fd104101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 -- 172.21.15.26:0/4190663744 >> 172.21.15.26:0/4190663744 conn(0x7fd1040fb0e0 msgr2=0x7fd1040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 -- 172.21.15.26:0/4190663744 shutdown_connections 2023-07-13T22:11:51.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.545+0000 7fd109fff700 1 -- 172.21.15.26:0/4190663744 wait complete. 2023-07-13T22:11:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.546+0000 7fd109fff700 1 Processor -- start 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.546+0000 7fd109fff700 1 -- start start 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd104100cf0 0x7fd104072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040735e0 con 0x7fd104101680 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd104073750 con 0x7fd1040ffb00 2023-07-13T22:11:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd109fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10419f9a0 con 0x7fd104100cf0 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd103fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:47594/0 (socket says 172.21.15.26:47594) 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 -- 172.21.15.26:0/1766754940 learned_addr learned my addr 172.21.15.26:0/1766754940 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd102ffd700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd104100cf0 0x7fd104072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 -- 172.21.15.26:0/1766754940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd104100cf0 msgr2=0x7fd104072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd104100cf0 0x7fd104072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 -- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 msgr2=0x7fd104072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.547+0000 7fd1037fe700 1 -- 172.21.15.26:0/1766754940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009520 con 0x7fd1040ffb00 2023-07-13T22:11:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd1037fe700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400ea50 tx=0x7fd0f400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:51.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400cbd0 con 0x7fd1040ffb00 2023-07-13T22:11:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f400cd30 con 0x7fd1040ffb00 2023-07-13T22:11:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4018680 con 0x7fd1040ffb00 2023-07-13T22:11:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc009d00 con 0x7fd1040ffb00 2023-07-13T22:11:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.548+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10419ff80 con 0x7fd1040ffb00 2023-07-13T22:11:51.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.550+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd104106f90 con 0x7fd1040ffb00 2023-07-13T22:11:51.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.552+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd0f4010430 con 0x7fd1040ffb00 2023-07-13T22:11:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.553+0000 7fd100ff9700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 0x7fd0ec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.554+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd0f4014070 con 0x7fd1040ffb00 2023-07-13T22:11:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.554+0000 7fd102ffd700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 0x7fd0ec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:51.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.554+0000 7fd102ffd700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 0x7fd0ec0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009c60 tx=0x7fd0f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:51.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.555+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd0f40968a0 con 0x7fd1040ffb00 2023-07-13T22:11:51.703 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:51 smithi026 systemd[1]: Starting Ceph mds.d for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.832+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd104067300 con 0x7fd1040ffb00 2023-07-13T22:11:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.833+0000 7fd100ff9700 1 -- 172.21.15.26:0/1766754940 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7fd0f4096030 con 0x7fd1040ffb00 2023-07-13T22:11:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:51.825 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-13T22:11:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 msgr2=0x7fd0ec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 0x7fd0ec0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009c60 tx=0x7fd0f8009260 comp rx=0 tx=0).stop 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 msgr2=0x7fd1040742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400ea50 tx=0x7fd0f400ed60 comp rx=0 tx=0).stop 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 shutdown_connections 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0ec0a11a0 0x7fd0ec0a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd104101680 0x7fd104072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1040ffb00 0x7fd1040742c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 --2- 172.21.15.26:0/1766754940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd104100cf0 0x7fd104072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.835+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 >> 172.21.15.26:0/1766754940 conn(0x7fd1040fb0e0 msgr2=0x7fd1041040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.836+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 shutdown_connections 2023-07-13T22:11:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:51.836+0000 7fd109fff700 1 -- 172.21.15.26:0/1766754940 wait complete. 2023-07-13T22:11:51.829 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 4 2023-07-13T22:11:52.156 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:51 smithi026 podman[129634]: 2023-07-13T22:11:52.453 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:52 smithi026 bash[129634]: d007cff9f7c520906b8dfb6aae298476952bba0d769244d1202f4ba8b5e7a1c1 2023-07-13T22:11:52.454 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mds[129676]: -- 172.21.15.26:0/2408751975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558458edd1e0 con 0x558458eb3000 2023-07-13T22:11:52.454 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-d[129652]: starting mds.d at 2023-07-13T22:11:52.454 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:11:52 smithi026 systemd[1]: Started Ceph mds.d for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:52.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1766754940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:52.922 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:52.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1766754940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:52.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1766754940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:52.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:53.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:53 smithi117 ceph-mon[129074]: Deploying daemon mds.e on smithi117 2023-07-13T22:11:53.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:53 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6836/3391570568,v1:172.21.15.26:6837/3391570568] up:boot 2023-07-13T22:11:53.779 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:53 smithi117 ceph-mon[129074]: fsmap 4 up:standby 2023-07-13T22:11:53.780 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-13T22:11:53.780 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:53 smithi117 systemd[1]: Starting Ceph mds.e for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:53.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:53.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:53 smithi026 ceph-mon[95647]: Deploying daemon mds.e on smithi117 2023-07-13T22:11:53.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:53 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6836/3391570568,v1:172.21.15.26:6837/3391570568] up:boot 2023-07-13T22:11:53.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:53 smithi026 ceph-mon[95647]: fsmap 4 up:standby 2023-07-13T22:11:53.918 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-13T22:11:53.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:53 smithi121 ceph-mon[129187]: Deploying daemon mds.e on smithi117 2023-07-13T22:11:53.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:53 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6836/3391570568,v1:172.21.15.26:6837/3391570568] up:boot 2023-07-13T22:11:53.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:53 smithi121 ceph-mon[129187]: fsmap 4 up:standby 2023-07-13T22:11:53.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-13T22:11:54.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/2341963261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f4072f70 msgr2=0x7fa2f40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/2341963261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f4072f70 0x7fa2f40733e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009990 tx=0x7fa2e8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/2341963261 shutdown_connections 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/2341963261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f41079c0 0x7fa2f4109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/2341963261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/2341963261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f4072f70 0x7fa2f40733e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/2341963261 >> 172.21.15.26:0/2341963261 conn(0x7fa2f40fb0e0 msgr2=0x7fa2f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:54.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/2341963261 shutdown_connections 2023-07-13T22:11:54.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.074+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/2341963261 wait complete. 2023-07-13T22:11:54.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.075+0000 7fa2fa5ff700 1 Processor -- start 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 -- start start 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f41079c0 0x7fa2f419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419f5e0 con 0x7fa2f41079c0 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419f720 con 0x7fa2f4072f70 2023-07-13T22:11:54.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2fa5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f419f8b0 con 0x7fa2f4074550 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:41682/0 (socket says 172.21.15.26:41682) 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f8b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f41079c0 0x7fa2f419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f3fff700 1 -- 172.21.15.26:0/182916544 learned_addr learned my addr 172.21.15.26:0/182916544 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.076+0000 7fa2f37fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35398/0 (socket says 172.21.15.26:35398) 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 -- 172.21.15.26:0/182916544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 msgr2=0x7fa2f4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:54.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 -- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f41079c0 msgr2=0x7fa2f419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f41079c0 0x7fa2f419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 -- 172.21.15.26:0/182916544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4009520 con 0x7fa2f4072f70 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f3fff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400ea30 tx=0x7fa2e400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e400cbd0 con 0x7fa2f4072f70 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e400eec0 con 0x7fa2f4072f70 2023-07-13T22:11:54.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4003d20 con 0x7fa2f4072f70 2023-07-13T22:11:54.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec009c40 con 0x7fa2f4072f70 2023-07-13T22:11:54.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.077+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e80095f0 con 0x7fa2f4072f70 2023-07-13T22:11:54.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.078+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f4106070 con 0x7fa2f4072f70 2023-07-13T22:11:54.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.080+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa2e4003ee0 con 0x7fa2f4072f70 2023-07-13T22:11:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.082+0000 7fa2f17fa700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 0x7fa2dc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.082+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa2e4014070 con 0x7fa2f4072f70 2023-07-13T22:11:54.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.082+0000 7fa2f37fe700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 0x7fa2dc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.083+0000 7fa2f37fe700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 0x7fa2dc0a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b2e0 tx=0x7fa2e8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:54.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.083+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa2e4096900 con 0x7fa2f4072f70 2023-07-13T22:11:54.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.339+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2f402c850 con 0x7fa2f4072f70 2023-07-13T22:11:54.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.340+0000 7fa2f17fa700 1 -- 172.21.15.26:0/182916544 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7fa2e4099f20 con 0x7fa2f4072f70 2023-07-13T22:11:54.332 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:54.332 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-13T22:11:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 msgr2=0x7fa2dc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 0x7fa2dc0a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800b2e0 tx=0x7fa2e8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 msgr2=0x7fa2f4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2e400ea30 tx=0x7fa2e400ed40 comp rx=0 tx=0).stop 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 shutdown_connections 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2dc0a1220 0x7fa2dc0a36d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2f41079c0 0x7fa2f419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2f4072f70 0x7fa2f4197a60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 --2- 172.21.15.26:0/182916544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2f4074550 0x7fa2f4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 >> 172.21.15.26:0/182916544 conn(0x7fa2f40fb0e0 msgr2=0x7fa2f40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 shutdown_connections 2023-07-13T22:11:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:54.342+0000 7fa2fa5ff700 1 -- 172.21.15.26:0/182916544 wait complete. 2023-07-13T22:11:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 5 2023-07-13T22:11:54.365 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:54 smithi117 podman[153597]: 2023-07-13T22:11:54.365 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mds[153637]: -- 172.21.15.117:0/2856661548 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fee240d1e0 con 0x55fee23e2800 2023-07-13T22:11:54.365 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:54 smithi117 bash[153597]: 8d0b67855ae880a331efd96be44c6d87c0df0a6c21b3965fa173778cd9dfec9f 2023-07-13T22:11:54.366 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: starting mds.e at 2023-07-13T22:11:54.695 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:11:54 smithi117 systemd[1]: Started Ceph mds.e for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:54.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:54.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/182916544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:54.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:54.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:54.697 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:54.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/182916544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:54.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:54.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:54.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/182916544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:54.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:54.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:55.579 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:55 smithi121 ceph-mon[129187]: Deploying daemon mds.f on smithi121 2023-07-13T22:11:55.579 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:55 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6834/1862924316,v1:172.21.15.117:6835/1862924316] up:boot 2023-07-13T22:11:55.579 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:55 smithi121 ceph-mon[129187]: fsmap 5 up:standby 2023-07-13T22:11:55.579 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:55 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:11:55.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:55 smithi117 ceph-mon[129074]: Deploying daemon mds.f on smithi121 2023-07-13T22:11:55.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:55 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6834/1862924316,v1:172.21.15.117:6835/1862924316] up:boot 2023-07-13T22:11:55.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:55 smithi117 ceph-mon[129074]: fsmap 5 up:standby 2023-07-13T22:11:55.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:55 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:11:55.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:55 smithi026 ceph-mon[95647]: Deploying daemon mds.f on smithi121 2023-07-13T22:11:55.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:55 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6834/1862924316,v1:172.21.15.117:6835/1862924316] up:boot 2023-07-13T22:11:55.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:55 smithi026 ceph-mon[95647]: fsmap 5 up:standby 2023-07-13T22:11:55.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:55 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:11:55.996 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:55 smithi121 systemd[1]: Starting Ceph mds.f for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:56.354 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:56.398 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:56 smithi121 podman[154558]: 2023-07-13T22:11:56.398 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:56 smithi121 bash[154558]: 56a5e85e47c3983b5b6b9285c573eb709cd9cb32273c684e6403b353790324eb 2023-07-13T22:11:56.398 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mds[154598]: -- 172.21.15.121:0/807560722 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558960aab1e0 con 0x558960a80800 2023-07-13T22:11:56.398 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: starting mds.f at 2023-07-13T22:11:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 -- 172.21.15.26:0/2594295909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28100cf0 msgr2=0x7faf28101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 --2- 172.21.15.26:0/2594295909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28100cf0 0x7faf28101140 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf18009990 tx=0x7faf18009ca0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 -- 172.21.15.26:0/2594295909 shutdown_connections 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 --2- 172.21.15.26:0/2594295909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 --2- 172.21.15.26:0/2594295909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28101680 0x7faf28107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.584+0000 7faf307ff700 1 --2- 172.21.15.26:0/2594295909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28100cf0 0x7faf28101140 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.585+0000 7faf307ff700 1 -- 172.21.15.26:0/2594295909 >> 172.21.15.26:0/2594295909 conn(0x7faf280fb0e0 msgr2=0x7faf280fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:56.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.585+0000 7faf307ff700 1 -- 172.21.15.26:0/2594295909 shutdown_connections 2023-07-13T22:11:56.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.585+0000 7faf307ff700 1 -- 172.21.15.26:0/2594295909 wait complete. 2023-07-13T22:11:56.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 Processor -- start 2023-07-13T22:11:56.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 -- start start 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28100cf0 0x7faf28072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 0x7faf28072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf280735e0 con 0x7faf280ffb00 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf28073750 con 0x7faf28100cf0 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf307ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819f7f0 con 0x7faf28101680 2023-07-13T22:11:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf2e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf2e59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44652/0 (socket says 172.21.15.26:44652) 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.586+0000 7faf2e59b700 1 -- 172.21.15.26:0/358157996 learned_addr learned my addr 172.21.15.26:0/358157996 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 0x7faf28072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2dd9a700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28100cf0 0x7faf28072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 -- 172.21.15.26:0/358157996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28100cf0 msgr2=0x7faf28072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28100cf0 0x7faf28072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 -- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 msgr2=0x7faf280742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 -- 172.21.15.26:0/358157996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24009520 con 0x7faf28101680 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2e59b700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf2ed9c700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 0x7faf28072e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf2000efc0 tx=0x7faf2000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20018070 con 0x7faf28101680 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf2000f040 con 0x7faf28101680 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf180095f0 con 0x7faf28101680 2023-07-13T22:11:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.587+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000cd40 con 0x7faf28101680 2023-07-13T22:11:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.588+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2819fe40 con 0x7faf28101680 2023-07-13T22:11:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.588+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf28106fe0 con 0x7faf28101680 2023-07-13T22:11:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.590+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faf20003a00 con 0x7faf28101680 2023-07-13T22:11:56.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.592+0000 7faf1f7fe700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 0x7faf140a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:56.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.592+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf200cc120 con 0x7faf28101680 2023-07-13T22:11:56.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.592+0000 7faf2e59b700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 0x7faf140a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.593+0000 7faf2e59b700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 0x7faf140a36d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf24009fd0 tx=0x7faf24009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.594+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf20095d00 con 0x7faf28101680 2023-07-13T22:11:56.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:56.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:56.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:56.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:56.746 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:11:56 smithi121 systemd[1]: Started Ceph mds.f for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:56.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:56.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:56.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:56.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.849+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf2802ca60 con 0x7faf28101680 2023-07-13T22:11:56.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.850+0000 7faf1f7fe700 1 -- 172.21.15.26:0/358157996 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5042 (secure 0 0 0) 0x7faf20099320 con 0x7faf28101680 2023-07-13T22:11:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:56.842 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-13T22:11:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 msgr2=0x7faf140a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 0x7faf140a36d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf24009fd0 tx=0x7faf24009260 comp rx=0 tx=0).stop 2023-07-13T22:11:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 msgr2=0x7faf28072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 0x7faf28072e40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf2000efc0 tx=0x7faf2000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 shutdown_connections 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faf140a1220 0x7faf140a36d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf280ffb00 0x7faf280742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf28101680 0x7faf28072e40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 --2- 172.21.15.26:0/358157996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf28100cf0 0x7faf28072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 >> 172.21.15.26:0/358157996 conn(0x7faf280fb0e0 msgr2=0x7faf28104120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 shutdown_connections 2023-07-13T22:11:56.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:56.852+0000 7faf307ff700 1 -- 172.21.15.26:0/358157996 wait complete. 2023-07-13T22:11:56.846 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2023-07-13T22:11:56.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:56.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:57.702 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:57 smithi121 ceph-mon[129187]: Deploying daemon mds.c on smithi121 2023-07-13T22:11:57.703 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:57 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:boot 2023-07-13T22:11:57.703 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:57 smithi121 ceph-mon[129187]: fsmap 6 up:standby 2023-07-13T22:11:57.703 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:11:57.703 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:57 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/358157996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:57.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:57 smithi117 ceph-mon[129074]: Deploying daemon mds.c on smithi121 2023-07-13T22:11:57.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:57 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:boot 2023-07-13T22:11:57.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:57 smithi117 ceph-mon[129074]: fsmap 6 up:standby 2023-07-13T22:11:57.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:11:57.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:57 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/358157996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:57.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:57 smithi026 ceph-mon[95647]: Deploying daemon mds.c on smithi121 2023-07-13T22:11:57.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:57 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:boot 2023-07-13T22:11:57.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:57 smithi026 ceph-mon[95647]: fsmap 6 up:standby 2023-07-13T22:11:57.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:11:57.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:57 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/358157996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:57.996 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:57 smithi121 systemd[1]: Starting Ceph mds.c for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:11:58.496 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:58 smithi121 podman[154897]: 2023-07-13T22:11:58.497 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:58 smithi121 bash[154897]: cad4444215a047599b8c6600bc0c09dc54a6bf299b53727d850f74e006c7c54e 2023-07-13T22:11:58.497 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:58 smithi121 ceph-mds[154937]: -- 172.21.15.121:0/3115289739 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3a26cf1e0 con 0x55c3a26a4800 2023-07-13T22:11:58.497 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: starting mds.c at 2023-07-13T22:11:58.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:11:58.931 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:58 smithi026 ceph-mon[95647]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:58.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:58 smithi117 ceph-mon[129074]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:58.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:58 smithi121 ceph-mon[129187]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:11:58.996 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:11:58 smithi121 systemd[1]: Started Ceph mds.c for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:11:59.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 -- 172.21.15.26:0/419421008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 msgr2=0x7fab6c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 --2- 172.21.15.26:0/419421008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c107f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b0a0 tx=0x7fab6400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 -- 172.21.15.26:0/419421008 shutdown_connections 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 --2- 172.21.15.26:0/419421008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 --2- 172.21.15.26:0/419421008 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c107f40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 --2- 172.21.15.26:0/419421008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c100cf0 0x7fab6c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 -- 172.21.15.26:0/419421008 >> 172.21.15.26:0/419421008 conn(0x7fab6c0fb0e0 msgr2=0x7fab6c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:59.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 -- 172.21.15.26:0/419421008 shutdown_connections 2023-07-13T22:11:59.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.089+0000 7fab737ff700 1 -- 172.21.15.26:0/419421008 wait complete. 2023-07-13T22:11:59.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.090+0000 7fab737ff700 1 Processor -- start 2023-07-13T22:11:59.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 -- start start 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c100cf0 0x7fab6c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c0735e0 con 0x7fab6c100cf0 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c073750 con 0x7fab6c101680 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab737ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c19f9a0 con 0x7fab6c0ffb00 2023-07-13T22:11:59.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab7159b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab7159b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35436/0 (socket says 172.21.15.26:35436) 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab7159b700 1 -- 172.21.15.26:0/1177215970 learned_addr learned my addr 172.21.15.26:0/1177215970 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab70d9a700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c100cf0 0x7fab6c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab71d9c700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab7159b700 1 -- 172.21.15.26:0/1177215970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 msgr2=0x7fab6c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.091+0000 7fab7159b700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab7159b700 1 -- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c100cf0 msgr2=0x7fab6c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:59.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab7159b700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c100cf0 0x7fab6c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab7159b700 1 -- 172.21.15.26:0/1177215970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68009520 con 0x7fab6c0ffb00 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab7159b700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab6800ea50 tx=0x7fab6800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800cbd0 con 0x7fab6c0ffb00 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab6800cd30 con 0x7fab6c0ffb00 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab64009d00 con 0x7fab6c0ffb00 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68018680 con 0x7fab6c0ffb00 2023-07-13T22:11:59.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.092+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c19ffd0 con 0x7fab6c0ffb00 2023-07-13T22:11:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.093+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c10b330 con 0x7fab6c0ffb00 2023-07-13T22:11:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.095+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab68010430 con 0x7fab6c0ffb00 2023-07-13T22:11:59.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.096+0000 7fab627fc700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 0x7fab580a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:11:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.096+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fab68014070 con 0x7fab6c0ffb00 2023-07-13T22:11:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.097+0000 7fab70d9a700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 0x7fab580a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:11:59.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.097+0000 7fab70d9a700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 0x7fab580a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009c60 tx=0x7fab5c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:11:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.098+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fab680968e0 con 0x7fab6c0ffb00 2023-07-13T22:11:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.355+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6c106910 con 0x7fab6c0ffb00 2023-07-13T22:11:59.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.356+0000 7fab627fc700 1 -- 172.21.15.26:0/1177215970 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5802 (secure 0 0 0) 0x7fab68099f00 con 0x7fab6c0ffb00 2023-07-13T22:11:59.349 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:11:59.349 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-13T22:11:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 msgr2=0x7fab580a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 0x7fab580a3650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009c60 tx=0x7fab5c009260 comp rx=0 tx=0).stop 2023-07-13T22:11:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 msgr2=0x7fab6c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:11:59.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fab6800ea50 tx=0x7fab6800ed60 comp rx=0 tx=0).stop 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 shutdown_connections 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab6c101680 0x7fab6c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab6c0ffb00 0x7fab6c0742c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fab580a11a0 0x7fab580a3650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 --2- 172.21.15.26:0/1177215970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab6c100cf0 0x7fab6c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 >> 172.21.15.26:0/1177215970 conn(0x7fab6c0fb0e0 msgr2=0x7fab6c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:11:59.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 shutdown_connections 2023-07-13T22:11:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:11:59.359+0000 7fab737ff700 1 -- 172.21.15.26:0/1177215970 wait complete. 2023-07-13T22:11:59.353 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-07-13T22:11:59.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6836/1505729647,v1:172.21.15.121:6837/1505729647] up:boot 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: fsmap 7 up:standby 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:59.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:59.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:59.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: Deploying daemon mds.h on smithi117 2023-07-13T22:11:59.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:11:59 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1177215970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:59.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6836/1505729647,v1:172.21.15.121:6837/1505729647] up:boot 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: fsmap 7 up:standby 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:59.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:59.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:59.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: Deploying daemon mds.h on smithi117 2023-07-13T22:11:59.795 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:11:59 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1177215970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:11:59.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6836/1505729647,v1:172.21.15.121:6837/1505729647] up:boot 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: fsmap 7 up:standby 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:11:59.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:11:59.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:11:59.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: Deploying daemon mds.h on smithi117 2023-07-13T22:11:59.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:11:59 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1177215970' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:00.195 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 systemd[1]: Starting Ceph mds.h for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:12:00.614 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 podman[153938]: 2023-07-13T22:12:00.615 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 ceph-mds[153978]: -- 172.21.15.117:0/4226441982 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557be12af1e0 con 0x557be1285000 2023-07-13T22:12:00.615 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 bash[153938]: 1fdcbd30178efb4b98bbb344a5d8f8dc738646577b9a8253d021b929f3d31f89 2023-07-13T22:12:00.615 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: starting mds.h at 2023-07-13T22:12:00.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:00 smithi117 ceph-mon[129074]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:00.946 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:12:00 smithi117 systemd[1]: Started Ceph mds.h for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:12:00.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:00 smithi026 ceph-mon[95647]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:00.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:00 smithi121 ceph-mon[129187]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:01.371 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:01.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 -- 172.21.15.26:0/3203114078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 msgr2=0x7f14700fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 --2- 172.21.15.26:0/3203114078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700fff10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f146c009990 tx=0x7f146c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 -- 172.21.15.26:0/3203114078 shutdown_connections 2023-07-13T22:12:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 --2- 172.21.15.26:0/3203114078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700fff10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 --2- 172.21.15.26:0/3203114078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470101680 0x7f1470107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 --2- 172.21.15.26:0/3203114078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470100cf0 0x7f1470101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 -- 172.21.15.26:0/3203114078 >> 172.21.15.26:0/3203114078 conn(0x7f14700fb0e0 msgr2=0x7f14700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 -- 172.21.15.26:0/3203114078 shutdown_connections 2023-07-13T22:12:01.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.594+0000 7f14775ff700 1 -- 172.21.15.26:0/3203114078 wait complete. 2023-07-13T22:12:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.595+0000 7f14775ff700 1 Processor -- start 2023-07-13T22:12:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.595+0000 7f14775ff700 1 -- start start 2023-07-13T22:12:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:01.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 0x7f1470072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470101680 0x7f1470072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14701996a0 con 0x7f14700ffb00 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1470073380 con 0x7f1470101680 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f14775ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1470073540 con 0x7f1470100cf0 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44696/0 (socket says 172.21.15.26:44696) 2023-07-13T22:12:01.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 -- 172.21.15.26:0/3437169907 learned_addr learned my addr 172.21.15.26:0/3437169907 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f1474b9a700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 0x7f1470072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f1475b9c700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470101680 0x7f1470072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 -- 172.21.15.26:0/3437169907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 msgr2=0x7f1470072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 0x7f1470072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 -- 172.21.15.26:0/3437169907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470101680 msgr2=0x7f1470072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470101680 0x7f1470072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f147539b700 1 -- 172.21.15.26:0/3437169907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0095f0 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.596+0000 7f1474b9a700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 0x7f1470072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f147539b700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f146c005740 tx=0x7f146c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c01d070 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146c021410 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14701a4050 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c017650 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.597+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14701a43c0 con 0x7f14700ffb00 2023-07-13T22:12:01.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.598+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147010b320 con 0x7f14700ffb00 2023-07-13T22:12:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.600+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f146c021580 con 0x7f14700ffb00 2023-07-13T22:12:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.601+0000 7f14667fc700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 0x7f145c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:01.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.601+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f146c0cd5a0 con 0x7f14700ffb00 2023-07-13T22:12:01.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.602+0000 7f1474b9a700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 0x7f145c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:01.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.602+0000 7f1474b9a700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 0x7f145c0a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1460011490 tx=0x7f1460009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:01.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.605+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f146c097180 con 0x7f14700ffb00 2023-07-13T22:12:01.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:boot 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: fsmap 8 up:standby 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:01.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:01.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:01 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.863+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f147010b4e0 con 0x7f14700ffb00 2023-07-13T22:12:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.863+0000 7f14667fc700 1 -- 172.21.15.26:0/3437169907 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6562 (secure 0 0 0) 0x7f146c09a7a0 con 0x7f14700ffb00 2023-07-13T22:12:01.856 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:01.856 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-13T22:12:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 msgr2=0x7f145c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 0x7f145c0a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1460011490 tx=0x7f1460009260 comp rx=0 tx=0).stop 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 msgr2=0x7f14700742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f146c005740 tx=0x7f146c005770 comp rx=0 tx=0).stop 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 shutdown_connections 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f145c0a1220 0x7f145c0a36d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14700ffb00 0x7f14700742c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1470101680 0x7f1470072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 --2- 172.21.15.26:0/3437169907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1470100cf0 0x7f1470072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 >> 172.21.15.26:0/3437169907 conn(0x7f14700fb0e0 msgr2=0x7f14700fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 shutdown_connections 2023-07-13T22:12:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:01.866+0000 7f14775ff700 1 -- 172.21.15.26:0/3437169907 wait complete. 2023-07-13T22:12:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 9 2023-07-13T22:12:01.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:boot 2023-07-13T22:12:01.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: fsmap 8 up:standby 2023-07-13T22:12:01.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:01.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:01 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:boot 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: fsmap 8 up:standby 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:01.951 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:01.952 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:01.952 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:01 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:02.246 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 systemd[1]: Starting Ceph mds.i for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:12:02.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:02 smithi121 ceph-mon[129187]: Deploying daemon mds.i on smithi121 2023-07-13T22:12:02.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:02 smithi121 ceph-mon[129187]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:02.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:02 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3437169907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:02.747 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 podman[155238]: 2023-07-13T22:12:02.747 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 ceph-mds[155278]: -- 172.21.15.121:0/29808152 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650478071e0 con 0x5650477dcc00 2023-07-13T22:12:02.747 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: starting mds.i at 2023-07-13T22:12:02.747 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 bash[155238]: b9db45c554d447ef5ab465c0f1b1a4215fcc78ae347fac74e075e0514e5143b1 2023-07-13T22:12:02.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:02 smithi117 ceph-mon[129074]: Deploying daemon mds.i on smithi121 2023-07-13T22:12:02.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:02 smithi117 ceph-mon[129074]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:02.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:02 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3437169907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:02.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:02 smithi026 ceph-mon[95647]: Deploying daemon mds.i on smithi121 2023-07-13T22:12:02.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:02 smithi026 ceph-mon[95647]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:02.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:02 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3437169907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:03.246 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:12:02 smithi121 systemd[1]: Started Ceph mds.i for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:12:03.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:04.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:04.091 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: Deploying daemon mds.g on smithi026 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6838/3259819874,v1:172.21.15.121:6839/3259819874] up:boot 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: fsmap 9 up:standby 2023-07-13T22:12:04.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:03 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:12:04.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 -- 172.21.15.26:0/1712488873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec100460 msgr2=0x7f7bec102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/1712488873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec100460 0x7f7bec102840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc009990 tx=0x7f7bdc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 -- 172.21.15.26:0/1712488873 shutdown_connections 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/1712488873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/1712488873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec100460 0x7f7bec102840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/1712488873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec102d80 0x7f7bec105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.115+0000 7f7bf09fe700 1 -- 172.21.15.26:0/1712488873 >> 172.21.15.26:0/1712488873 conn(0x7f7bec0f9ee0 msgr2=0x7f7bec0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:04.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.116+0000 7f7bf09fe700 1 -- 172.21.15.26:0/1712488873 shutdown_connections 2023-07-13T22:12:04.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.116+0000 7f7bf09fe700 1 -- 172.21.15.26:0/1712488873 wait complete. 2023-07-13T22:12:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 Processor -- start 2023-07-13T22:12:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 -- start start 2023-07-13T22:12:04.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec100460 0x7f7bec197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec102d80 0x7f7bec197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19f2b0 con 0x7f7bec1056a0 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19f3f0 con 0x7f7bec102d80 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bf09fe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec19f580 con 0x7f7bec100460 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7beb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7beb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56598/0 (socket says 172.21.15.26:56598) 2023-07-13T22:12:04.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7beb59e700 1 -- 172.21.15.26:0/177212748 learned_addr learned my addr 172.21.15.26:0/177212748 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bead9d700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec100460 0x7f7bec197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.117+0000 7f7bea59c700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec102d80 0x7f7bec197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 -- 172.21.15.26:0/177212748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec100460 msgr2=0x7f7bec197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec100460 0x7f7bec197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 -- 172.21.15.26:0/177212748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec102d80 msgr2=0x7f7bec197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec102d80 0x7f7bec197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 -- 172.21.15.26:0/177212748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0095f0 con 0x7f7bec1056a0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7beb59e700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ef30 tx=0x7f7be400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4018070 con 0x7f7bec1056a0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be400f040 con 0x7f7bec1056a0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec19f800 con 0x7f7bec1056a0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be400cd00 con 0x7f7bec1056a0 2023-07-13T22:12:04.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.118+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec19fd10 con 0x7f7bec1056a0 2023-07-13T22:12:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.120+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec107260 con 0x7f7bec1056a0 2023-07-13T22:12:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.122+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7be4003a00 con 0x7f7bec1056a0 2023-07-13T22:12:04.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.123+0000 7f7bd3fff700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 0x7f7bd40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.123+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7be40cbff0 con 0x7f7bec1056a0 2023-07-13T22:12:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.123+0000 7f7bead9d700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 0x7f7bd40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.124+0000 7f7bead9d700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 0x7f7bd40a36f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00b2e0 tx=0x7f7bdc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:04.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.127+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7be4095c10 con 0x7f7bec1056a0 2023-07-13T22:12:04.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: Deploying daemon mds.g on smithi026 2023-07-13T22:12:04.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:04.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6838/3259819874,v1:172.21.15.121:6839/3259819874] up:boot 2023-07-13T22:12:04.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: fsmap 9 up:standby 2023-07-13T22:12:04.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:03 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:12:04.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: Deploying daemon mds.g on smithi026 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6838/3259819874,v1:172.21.15.121:6839/3259819874] up:boot 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: fsmap 9 up:standby 2023-07-13T22:12:04.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:03 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:12:04.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.384+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bec02c850 con 0x7f7bec1056a0 2023-07-13T22:12:04.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.384+0000 7f7bd3fff700 1 -- 172.21.15.26:0/177212748 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7324 (secure 0 0 0) 0x7f7be4099230 con 0x7f7bec1056a0 2023-07-13T22:12:04.376 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:04.376 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-13T22:12:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 msgr2=0x7f7bd40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 0x7f7bd40a36f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00b2e0 tx=0x7f7bdc005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 msgr2=0x7f7bec19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7be400ef30 tx=0x7f7be400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 shutdown_connections 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bd40a1240 0x7f7bd40a36f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bec1056a0 0x7f7bec19ece0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bec100460 0x7f7bec197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 --2- 172.21.15.26:0/177212748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bec102d80 0x7f7bec197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 >> 172.21.15.26:0/177212748 conn(0x7f7bec0f9ee0 msgr2=0x7f7bec0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 shutdown_connections 2023-07-13T22:12:04.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:04.387+0000 7f7bf09fe700 1 -- 172.21.15.26:0/177212748 wait complete. 2023-07-13T22:12:04.381 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 10 2023-07-13T22:12:05.088 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/177212748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:05.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:04 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/177212748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:05.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:04 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/177212748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:05.453 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 systemd[1]: Starting Ceph mds.g for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:12:05.855 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 podman[130207]: 2023-07-13T22:12:05.855 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 ceph-mds[130247]: -- 172.21.15.26:0/424291539 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d369f671e0 con 0x55d369f3cc00 2023-07-13T22:12:05.855 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: starting mds.g at 2023-07-13T22:12:05.855 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 bash[130207]: c7293f35388a51d91f226e00e76a750a33b8070b02d827ef56c8f04f9b6f4fe6 2023-07-13T22:12:06.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:05 smithi117 ceph-mon[129074]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:06.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:05 smithi026 ceph-mon[95647]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:06.203 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:12:05 smithi026 systemd[1]: Started Ceph mds.g for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:12:06.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:05 smithi121 ceph-mon[129187]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:06.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 -- 172.21.15.26:0/1857993130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e40ffb00 msgr2=0x7f70e40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/1857993130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e40ffb00 0x7f70e40fff10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009990 tx=0x7f70e0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 -- 172.21.15.26:0/1857993130 shutdown_connections 2023-07-13T22:12:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/1857993130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e40ffb00 0x7f70e40fff10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/1857993130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4101680 0x7f70e4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/1857993130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e4100cf0 0x7f70e4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 -- 172.21.15.26:0/1857993130 >> 172.21.15.26:0/1857993130 conn(0x7f70e40fb0e0 msgr2=0x7f70e40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 -- 172.21.15.26:0/1857993130 shutdown_connections 2023-07-13T22:12:06.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.628+0000 7f70eb5ff700 1 -- 172.21.15.26:0/1857993130 wait complete. 2023-07-13T22:12:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.629+0000 7f70eb5ff700 1 Processor -- start 2023-07-13T22:12:06.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.629+0000 7f70eb5ff700 1 -- start start 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.629+0000 7f70eb5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70eb5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4100cf0 0x7f70e419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70eb5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70eb5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41a39b0 con 0x7f70e4101680 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70eb5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41a3af0 con 0x7f70e40ffb00 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70eb5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41a3c80 con 0x7f70e4100cf0 2023-07-13T22:12:06.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e9b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46432/0 (socket says 172.21.15.26:46432) 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 -- 172.21.15.26:0/683221310 learned_addr learned my addr 172.21.15.26:0/683221310 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e9b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56642/0 (socket says 172.21.15.26:56642) 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e8b9a700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4100cf0 0x7f70e419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 -- 172.21.15.26:0/683221310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4100cf0 msgr2=0x7f70e419c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:06.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4100cf0 0x7f70e419c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 -- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 msgr2=0x7f70e41a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e939b700 1 -- 172.21.15.26:0/683221310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00095f0 con 0x7f70e40ffb00 2023-07-13T22:12:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.630+0000 7f70e9b9c700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:06.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70e939b700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70e00098e0 tx=0x7f70e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e001d070 con 0x7f70e40ffb00 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e0005030 con 0x7f70e40ffb00 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e41a3f00 con 0x7f70e40ffb00 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0004b50 con 0x7f70e40ffb00 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.631+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e41a4270 con 0x7f70e40ffb00 2023-07-13T22:12:06.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.632+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e41049d0 con 0x7f70e40ffb00 2023-07-13T22:12:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.634+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f70e0004cd0 con 0x7f70e40ffb00 2023-07-13T22:12:06.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.636+0000 7f70da7fc700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 0x7f70d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.636+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70e00cdc00 con 0x7f70e40ffb00 2023-07-13T22:12:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.636+0000 7f70e8b9a700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 0x7f70d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:06.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.637+0000 7f70e8b9a700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 0x7f70d00a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009c60 tx=0x7f70d4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:06.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.638+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f70e00977e0 con 0x7f70e40ffb00 2023-07-13T22:12:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.910+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70e402cc70 con 0x7f70e40ffb00 2023-07-13T22:12:06.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.911+0000 7f70da7fc700 1 -- 172.21.15.26:0/683221310 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8082 (secure 0 0 0) 0x7f70e009ae00 con 0x7f70e40ffb00 2023-07-13T22:12:06.903 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:06.903 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-13T22:12:06.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 msgr2=0x7f70d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 0x7f70d00a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009c60 tx=0x7f70d4009260 comp rx=0 tx=0).stop 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 msgr2=0x7f70e419be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70e00098e0 tx=0x7f70e0004730 comp rx=0 tx=0).stop 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 shutdown_connections 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d00a11a0 0x7f70d00a3650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e4101680 0x7f70e41a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70e40ffb00 0x7f70e419be30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 --2- 172.21.15.26:0/683221310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70e4100cf0 0x7f70e419c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 >> 172.21.15.26:0/683221310 conn(0x7f70e40fb0e0 msgr2=0x7f70e40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 shutdown_connections 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:06.914+0000 7f70eb5ff700 1 -- 172.21.15.26:0/683221310 wait complete. 2023-07-13T22:12:06.907 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 11 2023-07-13T22:12:07.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6838/1019342380,v1:172.21.15.26:6839/1019342380] up:boot 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: fsmap 10 up:standby 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:07.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:07.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:07.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:06 smithi117 ceph-mon[129074]: Deploying daemon mds.a on smithi026 2023-07-13T22:12:07.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6838/1019342380,v1:172.21.15.26:6839/1019342380] up:boot 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: fsmap 10 up:standby 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:07.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:07.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:07.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:06 smithi026 ceph-mon[95647]: Deploying daemon mds.a on smithi026 2023-07-13T22:12:07.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6838/1019342380,v1:172.21.15.26:6839/1019342380] up:boot 2023-07-13T22:12:07.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: fsmap 10 up:standby 2023-07-13T22:12:07.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:12:07.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:06 smithi121 ceph-mon[129187]: Deploying daemon mds.a on smithi026 2023-07-13T22:12:07.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/683221310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:07.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:07 smithi026 ceph-mon[95647]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:08.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:07 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/683221310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:08.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:07 smithi117 ceph-mon[129074]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:08.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:07 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/683221310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:08.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:07 smithi121 ceph-mon[129187]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:08.703 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:08 smithi026 systemd[1]: Starting Ceph mds.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:12:08.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:09.124 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:08 smithi026 podman[130598]: 2023-07-13T22:12:09.124 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:09 smithi026 bash[130598]: 93f18eff857ec62f1c8d027640cd8124bc620e53904c6ba5c49c302aae78789a 2023-07-13T22:12:09.124 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:09 smithi026 ceph-mds[130661]: -- 172.21.15.26:0/3129951390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558ab2d0b1e0 con 0x558ab2ce1000 2023-07-13T22:12:09.124 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: starting mds.a at 2023-07-13T22:12:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 -- 172.21.15.26:0/2398099415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760072f70 msgr2=0x7fc7600733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 --2- 172.21.15.26:0/2398099415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760072f70 0x7fc7600733e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc748009990 tx=0x7fc748009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 -- 172.21.15.26:0/2398099415 shutdown_connections 2023-07-13T22:12:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 --2- 172.21.15.26:0/2398099415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc76010c0d0 0x7fc76010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 --2- 172.21.15.26:0/2398099415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc760074550 0x7fc7600729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 --2- 172.21.15.26:0/2398099415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760072f70 0x7fc7600733e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 -- 172.21.15.26:0/2398099415 >> 172.21.15.26:0/2398099415 conn(0x7fc7600fb0e0 msgr2=0x7fc7600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 -- 172.21.15.26:0/2398099415 shutdown_connections 2023-07-13T22:12:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.158+0000 7fc765bff700 1 -- 172.21.15.26:0/2398099415 wait complete. 2023-07-13T22:12:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 Processor -- start 2023-07-13T22:12:09.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 -- start start 2023-07-13T22:12:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 0x7fc76019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 0x7fc7601a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601a39d0 con 0x7fc760072f70 2023-07-13T22:12:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601a3b10 con 0x7fc760074550 2023-07-13T22:12:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.160+0000 7fc765bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7601a3ca0 con 0x7fc76010c0d0 2023-07-13T22:12:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc75f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 0x7fc76019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:46478/0 (socket says 172.21.15.26:46478) 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 -- 172.21.15.26:0/3994276068 learned_addr learned my addr 172.21.15.26:0/3994276068 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc75f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 0x7fc76019be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56672/0 (socket says 172.21.15.26:56672) 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc75ffff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 0x7fc7601a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 -- 172.21.15.26:0/3994276068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 msgr2=0x7fc7601a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 0x7fc7601a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 -- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 msgr2=0x7fc76019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 0x7fc76019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 -- 172.21.15.26:0/3994276068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480095f0 con 0x7fc760074550 2023-07-13T22:12:09.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc75ffff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 0x7fc7601a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:09.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.161+0000 7fc757fff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc748005720 tx=0x7fc748009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.162+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74801d070 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.162+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc748021410 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.162+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748017650 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.162+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7601a3f20 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.162+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7601a4290 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.163+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc760101c20 con 0x7fc760074550 2023-07-13T22:12:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.165+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc748021580 con 0x7fc760074550 2023-07-13T22:12:09.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.167+0000 7fc75d7fa700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:09.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.167+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc7480cd680 con 0x7fc760074550 2023-07-13T22:12:09.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.167+0000 7fc75f7fe700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:09.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.167+0000 7fc75f7fe700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 0x7fc74c0a36d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc760074920 tx=0x7fc750009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:09.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.168+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc748097260 con 0x7fc760074550 2023-07-13T22:12:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.450+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc76002cc70 con 0x7fc760074550 2023-07-13T22:12:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.451+0000 7fc75d7fa700 1 -- 172.21.15.26:0/3994276068 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7fc74809a880 con 0x7fc760074550 2023-07-13T22:12:09.443 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:09.443 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-07-13T22:12:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 msgr2=0x7fc74c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 0x7fc74c0a36d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc760074920 tx=0x7fc750009260 comp rx=0 tx=0).stop 2023-07-13T22:12:09.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 msgr2=0x7fc76019c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc748005720 tx=0x7fc748009580 comp rx=0 tx=0).stop 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 shutdown_connections 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc74c0a1220 0x7fc74c0a36d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc760072f70 0x7fc76019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc76010c0d0 0x7fc7601a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 --2- 172.21.15.26:0/3994276068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc760074550 0x7fc76019c390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.453+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 >> 172.21.15.26:0/3994276068 conn(0x7fc7600fb0e0 msgr2=0x7fc7600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.454+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 shutdown_connections 2023-07-13T22:12:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:09.454+0000 7fc765bff700 1 -- 172.21.15.26:0/3994276068 wait complete. 2023-07-13T22:12:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2023-07-13T22:12:09.453 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:12:09 smithi026 systemd[1]: Started Ceph mds.a for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:12:10.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:boot 2023-07-13T22:12:10.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: fsmap 11 up:standby 2023-07-13T22:12:10.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:12:10.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: Deploying daemon mds.b on smithi117 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:10.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:10 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3994276068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:10.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:boot 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: fsmap 11 up:standby 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:10.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:10.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:10.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: Deploying daemon mds.b on smithi117 2023-07-13T22:12:10.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:10.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:10 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3994276068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:10.496 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:boot 2023-07-13T22:12:10.496 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: fsmap 11 up:standby 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-13T22:12:10.497 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:10.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: Deploying daemon mds.b on smithi117 2023-07-13T22:12:10.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:10.498 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:10 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3994276068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:11.196 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 systemd[1]: Starting Ceph mds.b for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:12:11.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:11.608 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 podman[154279]: 2023-07-13T22:12:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 -- 172.21.15.26:0/2240396298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0ffbc0 msgr2=0x7f438c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 --2- 172.21.15.26:0/2240396298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0ffbc0 0x7f438c100010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4380009990 tx=0x7f4380009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 -- 172.21.15.26:0/2240396298 shutdown_connections 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 --2- 172.21.15.26:0/2240396298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 0x7f438c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 --2- 172.21.15.26:0/2240396298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0fe9d0 0x7f438c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 --2- 172.21.15.26:0/2240396298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0ffbc0 0x7f438c100010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.691+0000 7f43923ff700 1 -- 172.21.15.26:0/2240396298 >> 172.21.15.26:0/2240396298 conn(0x7f438c0f9fd0 msgr2=0x7f438c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:11.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.692+0000 7f43923ff700 1 -- 172.21.15.26:0/2240396298 shutdown_connections 2023-07-13T22:12:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.692+0000 7f43923ff700 1 -- 172.21.15.26:0/2240396298 wait complete. 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 Processor -- start 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 -- start start 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 0x7f438c0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 0x7f438c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 0x7f438c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d340 con 0x7f438c0fe9d0 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d4b0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f43923ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c19d670 con 0x7f438c100550 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f438bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 0x7f438c0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f438b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 0x7f438c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f438bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 0x7f438c0681e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56690/0 (socket says 172.21.15.26:56690) 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f439099c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 0x7f438c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.693+0000 7f438bfff700 1 -- 172.21.15.26:0/2707482632 learned_addr learned my addr 172.21.15.26:0/2707482632 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 -- 172.21.15.26:0/2707482632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 msgr2=0x7f438c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 0x7f438c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 -- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 msgr2=0x7f438c0681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 0x7f438c0681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 -- 172.21.15.26:0/2707482632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43800095f0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.694+0000 7f438b7fe700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 0x7f438c19a560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43800098e0 tx=0x7f4380005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.697+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438001d070 con 0x7f438c0ffbc0 2023-07-13T22:12:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.697+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4380021410 con 0x7f438c0ffbc0 2023-07-13T22:12:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.697+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380017650 con 0x7f438c0ffbc0 2023-07-13T22:12:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.697+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c19d8f0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.697+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c19dcc0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.699+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c105f60 con 0x7f438c0ffbc0 2023-07-13T22:12:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.700+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f438000fb10 con 0x7f438c0ffbc0 2023-07-13T22:12:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.702+0000 7f43897fa700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 0x7f43740a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:11.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.702+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43800cd7e0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.702+0000 7f438bfff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 0x7f43740a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.703+0000 7f438bfff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 0x7f43740a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f437c005950 tx=0x7f437c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:11.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.707+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f43800973c0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.945 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 ceph-mds[154319]: -- 172.21.15.117:0/191116283 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d5847d71e0 con 0x55d5847ac800 2023-07-13T22:12:11.945 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 bash[154279]: f2c0f1b34b6c02915b27f73b6984a10ba044b747a0f73397472d8e78a244b778 2023-07-13T22:12:11.945 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: starting mds.b at 2023-07-13T22:12:11.946 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:11 smithi117 systemd[1]: Started Ceph mds.b for 23b26990-21c9-11ee-9b32-001a4aab830c. 2023-07-13T22:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.971+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f438c061120 con 0x7f438c0ffbc0 2023-07-13T22:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.972+0000 7f43897fa700 1 -- 172.21.15.26:0/2707482632 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8840 (secure 0 0 0) 0x7f438009a9e0 con 0x7f438c0ffbc0 2023-07-13T22:12:11.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:11.964 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-07-13T22:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 msgr2=0x7f43740a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 0x7f43740a36d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f437c005950 tx=0x7f437c009260 comp rx=0 tx=0).stop 2023-07-13T22:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 msgr2=0x7f438c19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 0x7f438c19a560 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43800098e0 tx=0x7f4380005740 comp rx=0 tx=0).stop 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 shutdown_connections 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43740a1220 0x7f43740a36d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f438c0fe9d0 0x7f438c0681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f438c100550 0x7f438c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 --2- 172.21.15.26:0/2707482632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f438c0ffbc0 0x7f438c19a560 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 >> 172.21.15.26:0/2707482632 conn(0x7f438c0f9fd0 msgr2=0x7f438c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 shutdown_connections 2023-07-13T22:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:11.975+0000 7f43923ff700 1 -- 172.21.15.26:0/2707482632 wait complete. 2023-07-13T22:12:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 12 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2707482632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:boot 2023-07-13T22:12:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: fsmap 12 up:standby 2023-07-13T22:12:12.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:12 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:12:12.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:12.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2707482632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:boot 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: fsmap 12 up:standby 2023-07-13T22:12:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:12 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:12:12.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:12.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2707482632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:boot 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: fsmap 12 up:standby 2023-07-13T22:12:12.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:12 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:12:13.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 -- 172.21.15.26:0/1619289098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0410c0d0 msgr2=0x7fdf0410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/1619289098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0410c0d0 0x7fdf0410e4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b0a0 tx=0x7fdefc00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 -- 172.21.15.26:0/1619289098 shutdown_connections 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/1619289098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf0410c0d0 0x7fdf0410e4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/1619289098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 0x7fdf040729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/1619289098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf04072f70 0x7fdf040733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 -- 172.21.15.26:0/1619289098 >> 172.21.15.26:0/1619289098 conn(0x7fdf040fb0e0 msgr2=0x7fdf040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 -- 172.21.15.26:0/1619289098 shutdown_connections 2023-07-13T22:12:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.216+0000 7fdf0bfff700 1 -- 172.21.15.26:0/1619289098 wait complete. 2023-07-13T22:12:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 Processor -- start 2023-07-13T22:12:14.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 -- start start 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 0x7fdf0419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0410c0d0 0x7fdf041a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a3ac0 con 0x7fdf0410c0d0 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a3c00 con 0x7fdf04074550 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.218+0000 7fdf0bfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a3d90 con 0x7fdf04072f70 2023-07-13T22:12:14.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41066/0 (socket says 172.21.15.26:41066) 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 -- 172.21.15.26:0/2383672697 learned_addr learned my addr 172.21.15.26:0/2383672697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf0959a700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 0x7fdf0419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf0a59c700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0410c0d0 0x7fdf041a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 -- 172.21.15.26:0/2383672697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 msgr2=0x7fdf0419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 0x7fdf0419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 -- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0410c0d0 msgr2=0x7fdf041a34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0410c0d0 0x7fdf041a34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 -- 172.21.15.26:0/2383672697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009d00 con 0x7fdf04072f70 2023-07-13T22:12:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdf09d9b700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e970 tx=0x7fdf0000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:14.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00009790 con 0x7fdf04072f70 2023-07-13T22:12:14.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.219+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0000c7e0 con 0x7fdf04072f70 2023-07-13T22:12:14.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.220+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf041a4070 con 0x7fdf04072f70 2023-07-13T22:12:14.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.220+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000054e0 con 0x7fdf04072f70 2023-07-13T22:12:14.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.220+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf041a45d0 con 0x7fdf04072f70 2023-07-13T22:12:14.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.221+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04101c20 con 0x7fdf04072f70 2023-07-13T22:12:14.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.223+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdf0000c950 con 0x7fdf04072f70 2023-07-13T22:12:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.224+0000 7fdefaffd700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 0x7fdef00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.224+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf00014070 con 0x7fdf04072f70 2023-07-13T22:12:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.224+0000 7fdf0959a700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 0x7fdef00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.225+0000 7fdf0959a700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 0x7fdef00a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf04074270 tx=0x7fdef4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:14.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.227+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdf000968a0 con 0x7fdf04072f70 2023-07-13T22:12:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.493+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf0402c850 con 0x7fdf04072f70 2023-07-13T22:12:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.493+0000 7fdefaffd700 1 -- 172.21.15.26:0/2383672697 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9601 (secure 0 0 0) 0x7fdf00096020 con 0x7fdf04072f70 2023-07-13T22:12:14.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:14.486 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14520,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24467,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24563,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24575,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-13T22:12:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 msgr2=0x7fdef00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 0x7fdef00a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdf04074270 tx=0x7fdef4009260 comp rx=0 tx=0).stop 2023-07-13T22:12:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 msgr2=0x7fdf0419bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000e970 tx=0x7fdf0000ec80 comp rx=0 tx=0).stop 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 shutdown_connections 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdef00a1220 0x7fdef00a36d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0410c0d0 0x7fdf041a34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdf04072f70 0x7fdf0419bf40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 --2- 172.21.15.26:0/2383672697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf04074550 0x7fdf0419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 >> 172.21.15.26:0/2383672697 conn(0x7fdf040fb0e0 msgr2=0x7fdf040fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 shutdown_connections 2023-07-13T22:12:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.496+0000 7fdf0bfff700 1 -- 172.21.15.26:0/2383672697 wait complete. 2023-07-13T22:12:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 13 2023-07-13T22:12:14.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:12:14.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 -- 172.21.15.26:0/4184261658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70900ffb00 msgr2=0x7f70900fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 --2- 172.21.15.26:0/4184261658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70900ffb00 0x7f70900fff10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7080009990 tx=0x7f7080009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 -- 172.21.15.26:0/4184261658 shutdown_connections 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 --2- 172.21.15.26:0/4184261658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090101680 0x7f7090107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 --2- 172.21.15.26:0/4184261658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70900ffb00 0x7f70900fff10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 --2- 172.21.15.26:0/4184261658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7090100cf0 0x7f7090101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.733+0000 7f7094fff700 1 -- 172.21.15.26:0/4184261658 >> 172.21.15.26:0/4184261658 conn(0x7f70900fb0e0 msgr2=0x7f70900fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.734+0000 7f7094fff700 1 -- 172.21.15.26:0/4184261658 shutdown_connections 2023-07-13T22:12:14.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.734+0000 7f7094fff700 1 -- 172.21.15.26:0/4184261658 wait complete. 2023-07-13T22:12:14.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 Processor -- start 2023-07-13T22:12:14.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 -- start start 2023-07-13T22:12:14.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 0x7f7090197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090101680 0x7f709019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709019f5c0 con 0x7f7090100cf0 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709019f700 con 0x7f7090101680 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f7094fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709019f890 con 0x7f70900ffb00 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f708e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f708dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 0x7f7090197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f708e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41074/0 (socket says 172.21.15.26:41074) 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f708e59c700 1 -- 172.21.15.26:0/1948485151 learned_addr learned my addr 172.21.15.26:0/1948485151 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.735+0000 7f708ed9d700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090101680 0x7f709019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 -- 172.21.15.26:0/1948485151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090101680 msgr2=0x7f709019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090101680 0x7f709019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 -- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 msgr2=0x7f7090197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 0x7f7090197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 -- 172.21.15.26:0/1948485151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084009520 con 0x7f70900ffb00 2023-07-13T22:12:14.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708dd9b700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 0x7f7090197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f708e59c700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7080005740 tx=0x7f7080005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708001d070 con 0x7f70900ffb00 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7080021410 con 0x7f70900ffb00 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70800095f0 con 0x7f70900ffb00 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080017650 con 0x7f70900ffb00 2023-07-13T22:12:14.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.736+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709019fe00 con 0x7f70900ffb00 2023-07-13T22:12:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.737+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709010b320 con 0x7f70900ffb00 2023-07-13T22:12:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.739+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7080021580 con 0x7f70900ffb00 2023-07-13T22:12:14.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.740+0000 7f707f7fe700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 0x7f70780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.741+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f70800cd580 con 0x7f70900ffb00 2023-07-13T22:12:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.741+0000 7f708dd9b700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 0x7f70780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:14.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.741+0000 7f708dd9b700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 0x7f70780a36d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7084009fd0 tx=0x7f7084009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:14.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.742+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7080097160 con 0x7f70900ffb00 2023-07-13T22:12:14.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:14.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2383672697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:14.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:14 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:14.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2383672697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:14.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:14 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.969+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70901a03e0 con 0x7f70900ffb00 2023-07-13T22:12:14.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.970+0000 7f707f7fe700 1 -- 172.21.15.26:0/1948485151 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18399 (secure 0 0 0) 0x7f708009a780 con 0x7f70900ffb00 2023-07-13T22:12:14.962 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:14.962 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:11:32.444721+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:14.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.972+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 msgr2=0x7f70780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.972+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 0x7f70780a36d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7084009fd0 tx=0x7f7084009260 comp rx=0 tx=0).stop 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.972+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 msgr2=0x7f7090197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.972+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7080005740 tx=0x7f7080005770 comp rx=0 tx=0).stop 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 shutdown_connections 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7090101680 0x7f709019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70900ffb00 0x7f7090197a40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70780a1220 0x7f70780a36d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 --2- 172.21.15.26:0/1948485151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7090100cf0 0x7f7090197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:14.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 >> 172.21.15.26:0/1948485151 conn(0x7f70900fb0e0 msgr2=0x7f70900fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:14.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 shutdown_connections 2023-07-13T22:12:14.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:14.973+0000 7f7094fff700 1 -- 172.21.15.26:0/1948485151 wait complete. 2023-07-13T22:12:14.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:12:14.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2383672697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:14.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:14 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.225+0000 7f2acd7ff700 1 -- 172.21.15.26:0/1251528838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac80ffae0 msgr2=0x7f2ac80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.225+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/1251528838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac80ffae0 0x7f2ac80ffef0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009990 tx=0x7f2ab8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 -- 172.21.15.26:0/1251528838 shutdown_connections 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/1251528838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac80ffae0 0x7f2ac80ffef0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/1251528838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac8101660 0x7f2ac8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/1251528838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 -- 172.21.15.26:0/1251528838 >> 172.21.15.26:0/1251528838 conn(0x7f2ac80fb0e0 msgr2=0x7f2ac80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 -- 172.21.15.26:0/1251528838 shutdown_connections 2023-07-13T22:12:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.226+0000 7f2acd7ff700 1 -- 172.21.15.26:0/1251528838 wait complete. 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 Processor -- start 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 -- start start 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac80ffae0 0x7f2ac8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 0x7f2ac819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac819f610 con 0x7f2ac8100cd0 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac819f750 con 0x7f2ac8101660 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.227+0000 7f2acd7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac819f8e0 con 0x7f2ac80ffae0 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 0x7f2ac819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56874/0 (socket says 172.21.15.26:56874) 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 -- 172.21.15.26:0/2400371804 learned_addr learned my addr 172.21.15.26:0/2400371804 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac6ffd700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac80ffae0 0x7f2ac8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 -- 172.21.15.26:0/2400371804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac80ffae0 msgr2=0x7f2ac8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac80ffae0 0x7f2ac8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 -- 172.21.15.26:0/2400371804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 msgr2=0x7f2ac819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 0x7f2ac819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 -- 172.21.15.26:0/2400371804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80095f0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac77fe700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 0x7f2ac819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2abffff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9a0 tx=0x7f2ab000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0009790 con 0x7f2ac8100cd0 2023-07-13T22:12:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ab0010430 con 0x7f2ac8100cd0 2023-07-13T22:12:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0018680 con 0x7f2ac8100cd0 2023-07-13T22:12:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.228+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac819fbc0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.229+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac81a00d0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.230+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac81921c0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.232+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2ab00105a0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.233+0000 7f2ac4ff9700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 0x7f2ab40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.233+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2ab0014070 con 0x7f2ac8100cd0 2023-07-13T22:12:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.233+0000 7f2ac6ffd700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 0x7f2ab40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.234+0000 7f2ac6ffd700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 0x7f2ab40a3680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b2e0 tx=0x7f2ab8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.237+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2ab0051790 con 0x7f2ac8100cd0 2023-07-13T22:12:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.456+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ac80611c0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.456+0000 7f2ac4ff9700 1 -- 172.21.15.26:0/2400371804 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ab00961b0 con 0x7f2ac8100cd0 2023-07-13T22:12:15.448 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 msgr2=0x7f2ab40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 0x7f2ab40a3680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800b2e0 tx=0x7f2ab8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 msgr2=0x7f2ac8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e9a0 tx=0x7f2ab000ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 shutdown_connections 2023-07-13T22:12:15.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ac8101660 0x7f2ac819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ac80ffae0 0x7f2ac8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ab40a11d0 0x7f2ab40a3680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 --2- 172.21.15.26:0/2400371804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ac8100cd0 0x7f2ac8197fd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 >> 172.21.15.26:0/2400371804 conn(0x7f2ac80fb0e0 msgr2=0x7f2ac8106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 shutdown_connections 2023-07-13T22:12:15.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.459+0000 7f2acd7ff700 1 -- 172.21.15.26:0/2400371804 wait complete. 2023-07-13T22:12:15.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-13T22:12:15.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 -- 172.21.15.26:0/1826982121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4105790 msgr2=0x7f2ba4107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 --2- 172.21.15.26:0/1826982121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4105790 0x7f2ba4107bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009f90 tx=0x7f2b9c00b330 comp rx=0 tx=0).stop 2023-07-13T22:12:15.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 -- 172.21.15.26:0/1826982121 shutdown_connections 2023-07-13T22:12:15.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 --2- 172.21.15.26:0/1826982121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4105790 0x7f2ba4107bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 --2- 172.21.15.26:0/1826982121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4100550 0x7f2ba4102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 --2- 172.21.15.26:0/1826982121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4102e70 0x7f2ba4105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 -- 172.21.15.26:0/1826982121 >> 172.21.15.26:0/1826982121 conn(0x7f2ba40f9fd0 msgr2=0x7f2ba40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:15.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 -- 172.21.15.26:0/1826982121 shutdown_connections 2023-07-13T22:12:15.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.691+0000 7f2baafff700 1 -- 172.21.15.26:0/1826982121 wait complete. 2023-07-13T22:12:15.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 Processor -- start 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 -- start start 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4100550 0x7f2ba4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4105790 0x7f2ba419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba419f650 con 0x7f2ba4102e70 2023-07-13T22:12:15.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba419f790 con 0x7f2ba4105790 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.692+0000 7f2baafff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba419f920 con 0x7f2ba4100550 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56898/0 (socket says 172.21.15.26:56898) 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 -- 172.21.15.26:0/3445198725 learned_addr learned my addr 172.21.15.26:0/3445198725 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba959c700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4105790 0x7f2ba419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba8d9b700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4100550 0x7f2ba4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 -- 172.21.15.26:0/3445198725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4100550 msgr2=0x7f2ba4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4100550 0x7f2ba4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 -- 172.21.15.26:0/3445198725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4105790 msgr2=0x7f2ba419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4105790 0x7f2ba419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 -- 172.21.15.26:0/3445198725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c009c40 con 0x7f2ba4102e70 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba3fff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000e9a0 tx=0x7f2b9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90009790 con 0x7f2ba4102e70 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba419fc00 con 0x7f2ba4102e70 2023-07-13T22:12:15.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.693+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9000ce80 con 0x7f2ba4102e70 2023-07-13T22:12:15.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.694+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90018680 con 0x7f2ba4102e70 2023-07-13T22:12:15.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.694+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba41a0110 con 0x7f2ba4102e70 2023-07-13T22:12:15.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.695+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba4107300 con 0x7f2ba4102e70 2023-07-13T22:12:15.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.698+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2b90010430 con 0x7f2ba4102e70 2023-07-13T22:12:15.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.698+0000 7f2ba1ffb700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 0x7f2b940a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:15.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.698+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2b90014070 con 0x7f2ba4102e70 2023-07-13T22:12:15.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.698+0000 7f2ba8d9b700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 0x7f2b940a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:15.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.699+0000 7f2ba8d9b700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 0x7f2b940a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005fd0 tx=0x7f2b98009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:15.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.702+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2b90051790 con 0x7f2ba4102e70 2023-07-13T22:12:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.920+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ba402c850 con 0x7f2ba4102e70 2023-07-13T22:12:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.921+0000 7f2ba1ffb700 1 -- 172.21.15.26:0/3445198725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2b90099cb0 con 0x7f2ba4102e70 2023-07-13T22:12:15.913 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:15.913 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-07-13T22:12:15.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:12:15.914 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-07-13T22:12:15.914 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-07-13T22:12:15.914 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:12:15.914 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-07-13T22:12:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 msgr2=0x7f2b940a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 0x7f2b940a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005fd0 tx=0x7f2b98009310 comp rx=0 tx=0).stop 2023-07-13T22:12:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 msgr2=0x7f2ba4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000e9a0 tx=0x7f2b9000ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 shutdown_connections 2023-07-13T22:12:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ba4105790 0x7f2ba419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ba4100550 0x7f2ba4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2b940a11a0 0x7f2b940a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 --2- 172.21.15.26:0/3445198725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba4102e70 0x7f2ba4198010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.923+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 >> 172.21.15.26:0/3445198725 conn(0x7f2ba40f9fd0 msgr2=0x7f2ba40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.924+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 shutdown_connections 2023-07-13T22:12:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:15.924+0000 7f2baafff700 1 -- 172.21.15.26:0/3445198725 wait complete. 2023-07-13T22:12:15.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1948485151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:15.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2400371804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:12:15.932 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:15.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:15.933 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:15 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.934 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-13T22:12:15.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-13T22:12:15.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1948485151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:15.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2400371804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:12:15.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:15.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:15.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:15 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1948485151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:15.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2400371804' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:12:15.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:15.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:15.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:15.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:15 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 -- 172.21.15.26:0/1654415713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc100ce0 msgr2=0x7fe2fc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 --2- 172.21.15.26:0/1654415713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc101130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009990 tx=0x7fe2f0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 -- 172.21.15.26:0/1654415713 shutdown_connections 2023-07-13T22:12:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 --2- 172.21.15.26:0/1654415713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 --2- 172.21.15.26:0/1654415713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 --2- 172.21.15.26:0/1654415713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc101130 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 -- 172.21.15.26:0/1654415713 >> 172.21.15.26:0/1654415713 conn(0x7fe2fc0fb130 msgr2=0x7fe2fc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 -- 172.21.15.26:0/1654415713 shutdown_connections 2023-07-13T22:12:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.154+0000 7fe300dff700 1 -- 172.21.15.26:0/1654415713 wait complete. 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 Processor -- start 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 -- start start 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19d3d0 con 0x7fe2fc0ffaf0 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19d510 con 0x7fe2fc100ce0 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe300dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc19d6a0 con 0x7fe2fc101670 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fa59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56916/0 (socket says 172.21.15.26:56916) 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fa59c700 1 -- 172.21.15.26:0/2764705056 learned_addr learned my addr 172.21.15.26:0/2764705056 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 -- 172.21.15.26:0/2764705056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 msgr2=0x7fe2fc19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 -- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 msgr2=0x7fe2fc195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2f9d9b700 1 -- 172.21.15.26:0/2764705056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00095f0 con 0x7fe2fc100ce0 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fa59c700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.156+0000 7fe2fad9d700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe2f9d9b700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc195d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00098e0 tx=0x7fe2f0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f001d070 con 0x7fe2fc100ce0 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc19d920 con 0x7fe2fc100ce0 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2f0021410 con 0x7fe2fc100ce0 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017650 con 0x7fe2fc100ce0 2023-07-13T22:12:16.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.157+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc19dd40 con 0x7fe2fc100ce0 2023-07-13T22:12:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.158+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc106a20 con 0x7fe2fc100ce0 2023-07-13T22:12:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.161+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe2f0021580 con 0x7fe2fc100ce0 2023-07-13T22:12:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.162+0000 7fe2eb7fe700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 0x7fe2e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.162+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2f00cd9b0 con 0x7fe2fc100ce0 2023-07-13T22:12:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.162+0000 7fe2fa59c700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 0x7fe2e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.164+0000 7fe2fa59c700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 0x7fe2e40a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec005950 tx=0x7fe2ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.164+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe2f0097590 con 0x7fe2fc100ce0 2023-07-13T22:12:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.382+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe2fc02c850 con 0x7fe2fc100ce0 2023-07-13T22:12:16.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.968+0000 7fe2eb7fe700 1 -- 172.21.15.26:0/2764705056 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe2f0095e60 con 0x7fe2fc100ce0 2023-07-13T22:12:16.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 msgr2=0x7fe2e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 0x7fe2e40a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec005950 tx=0x7fe2ec009260 comp rx=0 tx=0).stop 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 msgr2=0x7fe2fc195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc195d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00098e0 tx=0x7fe2f0005740 comp rx=0 tx=0).stop 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 shutdown_connections 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2e40a1150 0x7fe2e40a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc0ffaf0 0x7fe2fc195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2fc101670 0x7fe2fc19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 --2- 172.21.15.26:0/2764705056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2fc100ce0 0x7fe2fc195d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:16.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 >> 172.21.15.26:0/2764705056 conn(0x7fe2fc0fb130 msgr2=0x7fe2fc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:16.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 shutdown_connections 2023-07-13T22:12:16.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:16.973+0000 7fe300dff700 1 -- 172.21.15.26:0/2764705056 wait complete. 2023-07-13T22:12:16.965 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-07-13T22:12:16.982 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-13T22:12:17.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:16 smithi117 ceph-mon[129074]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:17.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:16 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:17.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:16 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3445198725' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-13T22:12:17.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:16 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2764705056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:17.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 -- 172.21.15.26:0/2886853108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 msgr2=0x7fbcd4101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/2886853108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4101130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009990 tx=0x7fbcc8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 -- 172.21.15.26:0/2886853108 shutdown_connections 2023-07-13T22:12:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/2886853108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 0x7fbcd4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/2886853108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 0x7fbcd40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/2886853108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4101130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 -- 172.21.15.26:0/2886853108 >> 172.21.15.26:0/2886853108 conn(0x7fbcd40fb130 msgr2=0x7fbcd40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 -- 172.21.15.26:0/2886853108 shutdown_connections 2023-07-13T22:12:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.209+0000 7fbcda5ff700 1 -- 172.21.15.26:0/2886853108 wait complete. 2023-07-13T22:12:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 Processor -- start 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 -- start start 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 0x7fbcd4197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 0x7fbcd419f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.210+0000 7fbcda5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd419f620 con 0x7fbcd4101670 2023-07-13T22:12:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcda5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd419f760 con 0x7fbcd4100ce0 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcda5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd419f8f0 con 0x7fbcd40ffaf0 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58178/0 (socket says 172.21.15.26:58178) 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 -- 172.21.15.26:0/3292644529 learned_addr learned my addr 172.21.15.26:0/3292644529 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd8b9c700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 0x7fbcd419f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd3fff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 0x7fbcd4197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 -- 172.21.15.26:0/3292644529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 msgr2=0x7fbcd4197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 0x7fbcd4197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 -- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 msgr2=0x7fbcd419f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 0x7fbcd419f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 -- 172.21.15.26:0/3292644529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc80095f0 con 0x7fbcd4100ce0 2023-07-13T22:12:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.211+0000 7fbcd37fe700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005740 tx=0x7fbcc8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:17.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:16 smithi026 ceph-mon[95647]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:17.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:16 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:17.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:16 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3445198725' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-13T22:12:17.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:16 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2764705056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.213+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc801d070 con 0x7fbcd4100ce0 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.213+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc8021410 con 0x7fbcd4100ce0 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.213+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8017650 con 0x7fbcd4100ce0 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.213+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd419fb70 con 0x7fbcd4100ce0 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.213+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd419ffd0 con 0x7fbcd4100ce0 2023-07-13T22:12:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.214+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd4192180 con 0x7fbcd4100ce0 2023-07-13T22:12:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.216+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbcc8021580 con 0x7fbcd4100ce0 2023-07-13T22:12:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.217+0000 7fbcd17fa700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 0x7fbcbc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.217+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fbcc80cd8a0 con 0x7fbcd4100ce0 2023-07-13T22:12:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.217+0000 7fbcd3fff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 0x7fbcbc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.218+0000 7fbcd3fff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 0x7fbcbc0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4100b40 tx=0x7fbcc4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:17.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.221+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbcc8097310 con 0x7fbcd4100ce0 2023-07-13T22:12:17.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:16 smithi121 ceph-mon[129187]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:17.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:16 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:12:17.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:16 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3445198725' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-13T22:12:17.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:16 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2764705056' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:17.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.440+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbcd4061120 con 0x7fbcd4100ce0 2023-07-13T22:12:17.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.986+0000 7fbcd17fa700 1 -- 172.21.15.26:0/3292644529 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbcc809a930 con 0x7fbcd4100ce0 2023-07-13T22:12:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.994+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 msgr2=0x7fbcbc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.994+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 0x7fbcbc0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4100b40 tx=0x7fbcc4009260 comp rx=0 tx=0).stop 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.994+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 msgr2=0x7fbcd4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.994+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005740 tx=0x7fbcc8005770 comp rx=0 tx=0).stop 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 shutdown_connections 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbcbc0a11d0 0x7fbcbc0a3680 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4100b40 tx=0x7fbcc4009260 comp rx=0 tx=0).stop 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd4101670 0x7fbcd419f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd40ffaf0 0x7fbcd4197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 --2- 172.21.15.26:0/3292644529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcd4100ce0 0x7fbcd4197fe0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005740 tx=0x7fbcc8005770 comp rx=0 tx=0).stop 2023-07-13T22:12:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.995+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 >> 172.21.15.26:0/3292644529 conn(0x7fbcd40fb130 msgr2=0x7fbcd41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:17.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.998+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 shutdown_connections 2023-07-13T22:12:17.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:17.998+0000 7fbcda5ff700 1 -- 172.21.15.26:0/3292644529 wait complete. 2023-07-13T22:12:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-07-13T22:12:18.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-13T22:12:18.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:17 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:18.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:17 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:18.195 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:17 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-13T22:12:18.196 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:17 smithi117 ceph-mon[129074]: osdmap e80: 12 total, 12 up, 12 in 2023-07-13T22:12:18.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:17 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:18.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:17 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:18.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:17 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-13T22:12:18.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:17 smithi026 ceph-mon[95647]: osdmap e80: 12 total, 12 up, 12 in 2023-07-13T22:12:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/2106373555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 msgr2=0x7f5bb8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/2106373555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8105260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009990 tx=0x7f5ba4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/2106373555 shutdown_connections 2023-07-13T22:12:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/2106373555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb81057a0 0x7f5bb8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/2106373555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb8100560 0x7f5bb8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/2106373555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8105260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/2106373555 >> 172.21.15.26:0/2106373555 conn(0x7f5bb80fa000 msgr2=0x7f5bb80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/2106373555 shutdown_connections 2023-07-13T22:12:18.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.234+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/2106373555 wait complete. 2023-07-13T22:12:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.235+0000 7f5bbf1ff700 1 Processor -- start 2023-07-13T22:12:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.235+0000 7f5bbf1ff700 1 -- start start 2023-07-13T22:12:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 0x7f5bb819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:18.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819f5f0 con 0x7f5bb8100560 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819f730 con 0x7f5bb8102e80 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbf1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819f8c0 con 0x7f5bb81057a0 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbcf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbcf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56946/0 (socket says 172.21.15.26:56946) 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbcf9b700 1 -- 172.21.15.26:0/761911543 learned_addr learned my addr 172.21.15.26:0/761911543 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bb7fff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.236+0000 7f5bbd79c700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 0x7f5bb819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:18.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 -- 172.21.15.26:0/761911543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 msgr2=0x7f5bb819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 0x7f5bb819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 -- 172.21.15.26:0/761911543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 msgr2=0x7f5bb8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 -- 172.21.15.26:0/761911543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40095f0 con 0x7f5bb8100560 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbd79c700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 0x7f5bb819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bb7fff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbcf9b700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9a0 tx=0x7f5bac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac009790 con 0x7f5bb8100560 2023-07-13T22:12:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb819fba0 con 0x7f5bb8100560 2023-07-13T22:12:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac010430 con 0x7f5bb8100560 2023-07-13T22:12:18.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018680 con 0x7f5bb8100560 2023-07-13T22:12:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.237+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb81a00b0 con 0x7f5bb8100560 2023-07-13T22:12:18.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.239+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb8107360 con 0x7f5bb8100560 2023-07-13T22:12:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.243+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5bac0105a0 con 0x7f5bb8100560 2023-07-13T22:12:18.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.245+0000 7f5bb5ffb700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 0x7f5ba80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.245+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f5bac014070 con 0x7f5bb8100560 2023-07-13T22:12:18.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.245+0000 7f5bb7fff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 0x7f5ba80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.246+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5bac096400 con 0x7f5bb8100560 2023-07-13T22:12:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.246+0000 7f5bb7fff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 0x7f5ba80a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b2e0 tx=0x7f5ba4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:18.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:17 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-13T22:12:18.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:17 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:18.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:17 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-13T22:12:18.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:17 smithi121 ceph-mon[129187]: osdmap e80: 12 total, 12 up, 12 in 2023-07-13T22:12:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:18.515+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5bb802cb00 con 0x7f5bb8100560 2023-07-13T22:12:19.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: pgmap v263: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:19.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3292644529' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: osdmap e81: 12 total, 12 up, 12 in 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-13T22:12:19.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:18 smithi026 ceph-mon[95647]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: pgmap v263: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3292644529' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: osdmap e81: 12 total, 12 up, 12 in 2023-07-13T22:12:19.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-13T22:12:19.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:18 smithi121 ceph-mon[129187]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-13T22:12:19.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: pgmap v263: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:19.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3292644529' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: osdmap e81: 12 total, 12 up, 12 in 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:19.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-13T22:12:19.447 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:18 smithi117 ceph-mon[129074]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-13T22:12:19.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.490+0000 7f5bb5ffb700 1 -- 172.21.15.26:0/761911543 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5bac0260c0 con 0x7f5bb8100560 2023-07-13T22:12:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 msgr2=0x7f5ba80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 0x7f5ba80a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b2e0 tx=0x7f5ba4005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 msgr2=0x7f5bb8197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00e9a0 tx=0x7f5bac00ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 shutdown_connections 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ba80a1150 0x7f5ba80a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bb8100560 0x7f5bb8197a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5bb81057a0 0x7f5bb819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 --2- 172.21.15.26:0/761911543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bb8102e80 0x7f5bb8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.492+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 >> 172.21.15.26:0/761911543 conn(0x7f5bb80fa000 msgr2=0x7f5bb80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.493+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 shutdown_connections 2023-07-13T22:12:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.493+0000 7f5bbf1ff700 1 -- 172.21.15.26:0/761911543 wait complete. 2023-07-13T22:12:19.485 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2023-07-13T22:12:19.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:12:19.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 -- 172.21.15.26:0/867680877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 msgr2=0x7fe7780fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/867680877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780fff10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe774009990 tx=0x7fe774009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 -- 172.21.15.26:0/867680877 shutdown_connections 2023-07-13T22:12:19.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/867680877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780fff10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/867680877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778101680 0x7fe778107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/867680877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778100cf0 0x7fe778101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 -- 172.21.15.26:0/867680877 >> 172.21.15.26:0/867680877 conn(0x7fe7780fb0e0 msgr2=0x7fe7780fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:19.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 -- 172.21.15.26:0/867680877 shutdown_connections 2023-07-13T22:12:19.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.726+0000 7fe77f5ff700 1 -- 172.21.15.26:0/867680877 wait complete. 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.727+0000 7fe77f5ff700 1 Processor -- start 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 -- start start 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778100cf0 0x7fe778072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778101680 0x7fe778072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780735e0 con 0x7fe7780ffb00 2023-07-13T22:12:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778073750 con 0x7fe778100cf0 2023-07-13T22:12:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77f5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77819f8a0 con 0x7fe778101680 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56956/0 (socket says 172.21.15.26:56956) 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.728+0000 7fe77d39b700 1 -- 172.21.15.26:0/721376000 learned_addr learned my addr 172.21.15.26:0/721376000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77cb9a700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778100cf0 0x7fe778072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77db9c700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778101680 0x7fe778072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 -- 172.21.15.26:0/721376000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778101680 msgr2=0x7fe778072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778101680 0x7fe778072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 -- 172.21.15.26:0/721376000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778100cf0 msgr2=0x7fe778072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778100cf0 0x7fe778072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 -- 172.21.15.26:0/721376000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740095f0 con 0x7fe7780ffb00 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77d39b700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b620 tx=0x7fe7740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77401d070 con 0x7fe7780ffb00 2023-07-13T22:12:19.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe774021410 con 0x7fe7780ffb00 2023-07-13T22:12:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77819fb20 con 0x7fe7780ffb00 2023-07-13T22:12:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.729+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774017650 con 0x7fe7780ffb00 2023-07-13T22:12:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.730+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77819ff40 con 0x7fe7780ffb00 2023-07-13T22:12:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.731+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778107070 con 0x7fe7780ffb00 2023-07-13T22:12:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.732+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe774021580 con 0x7fe7780ffb00 2023-07-13T22:12:19.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.734+0000 7fe76e7fc700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 0x7fe7640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:19.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.734+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7740cde20 con 0x7fe7780ffb00 2023-07-13T22:12:19.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.734+0000 7fe77cb9a700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 0x7fe7640a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:19.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.735+0000 7fe77cb9a700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 0x7fe7640a3650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe768009bf0 tx=0x7fe768009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:19.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.738+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe7740976b0 con 0x7fe7780ffb00 2023-07-13T22:12:19.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.948+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe77802c850 con 0x7fe7780ffb00 2023-07-13T22:12:19.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.950+0000 7fe76e7fc700 1 -- 172.21.15.26:0/721376000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+22109 (secure 0 0 0) 0x7fe77409acd0 con 0x7fe7780ffb00 2023-07-13T22:12:19.941 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:19.941 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:18.515878+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 msgr2=0x7fe7640a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 0x7fe7640a3650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe768009bf0 tx=0x7fe768009260 comp rx=0 tx=0).stop 2023-07-13T22:12:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 msgr2=0x7fe7780742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b620 tx=0x7fe7740057f0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 shutdown_connections 2023-07-13T22:12:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7640a11a0 0x7fe7640a3650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7780ffb00 0x7fe7780742c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe778101680 0x7fe778072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 --2- 172.21.15.26:0/721376000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe778100cf0 0x7fe778072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 >> 172.21.15.26:0/721376000 conn(0x7fe7780fb0e0 msgr2=0x7fe7781041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 shutdown_connections 2023-07-13T22:12:19.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:19.952+0000 7fe77f5ff700 1 -- 172.21.15.26:0/721376000 wait complete. 2023-07-13T22:12:19.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:12:20.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 -- 172.21.15.26:0/3024512642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100ffb00 msgr2=0x7fd1100fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 --2- 172.21.15.26:0/3024512642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100ffb00 0x7fd1100fff10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd100009990 tx=0x7fd100009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 -- 172.21.15.26:0/3024512642 shutdown_connections 2023-07-13T22:12:20.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 --2- 172.21.15.26:0/3024512642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100ffb00 0x7fd1100fff10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 --2- 172.21.15.26:0/3024512642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 0x7fd110107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 --2- 172.21.15.26:0/3024512642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd110100cf0 0x7fd110101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 -- 172.21.15.26:0/3024512642 >> 172.21.15.26:0/3024512642 conn(0x7fd1100fb0e0 msgr2=0x7fd1100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 -- 172.21.15.26:0/3024512642 shutdown_connections 2023-07-13T22:12:20.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.177+0000 7fd115fff700 1 -- 172.21.15.26:0/3024512642 wait complete. 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.178+0000 7fd115fff700 1 Processor -- start 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.178+0000 7fd115fff700 1 -- start start 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1100ffb00 0x7fd1100742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 0x7fd110072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100735e0 con 0x7fd110100cf0 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110073750 con 0x7fd110101680 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd115fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11019f8e0 con 0x7fd1100ffb00 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56976/0 (socket says 172.21.15.26:56976) 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 -- 172.21.15.26:0/15636594 learned_addr learned my addr 172.21.15.26:0/15636594 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd10f7fe700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1100ffb00 0x7fd1100742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd10ffff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 0x7fd110072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 -- 172.21.15.26:0/15636594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1100ffb00 msgr2=0x7fd1100742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1100ffb00 0x7fd1100742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 -- 172.21.15.26:0/15636594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 msgr2=0x7fd110072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 0x7fd110072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.179+0000 7fd107fff700 1 -- 172.21.15.26:0/15636594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000095f0 con 0x7fd110100cf0 2023-07-13T22:12:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd107fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800e9a0 tx=0x7fd0f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8009790 con 0x7fd110100cf0 2023-07-13T22:12:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0f800cbd0 con 0x7fd110100cf0 2023-07-13T22:12:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8018680 con 0x7fd110100cf0 2023-07-13T22:12:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11019fbc0 con 0x7fd110100cf0 2023-07-13T22:12:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.180+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1101a0080 con 0x7fd110100cf0 2023-07-13T22:12:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.181+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1101070a0 con 0x7fd110100cf0 2023-07-13T22:12:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.183+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd0f800cd40 con 0x7fd110100cf0 2023-07-13T22:12:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.185+0000 7fd10d7fa700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 0x7fd0fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.185+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0f8014070 con 0x7fd110100cf0 2023-07-13T22:12:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.185+0000 7fd10f7fe700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 0x7fd0fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.185+0000 7fd10f7fe700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 0x7fd0fc0a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd10000b2e0 tx=0x7fd100005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.188+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd0f8051790 con 0x7fd110100cf0 2023-07-13T22:12:20.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: osdmap e82: 12 total, 12 up, 12 in 2023-07-13T22:12:20.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: pgmap v266: 97 pgs: 5 active+clean, 20 creating+peering, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:20.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-13T22:12:20.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-13T22:12:20.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:20.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:20.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-13T22:12:20.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:19 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/721376000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:20.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: osdmap e82: 12 total, 12 up, 12 in 2023-07-13T22:12:20.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: pgmap v266: 97 pgs: 5 active+clean, 20 creating+peering, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:20.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:19 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/721376000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.402+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd11002c8a0 con 0x7fd110100cf0 2023-07-13T22:12:20.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.404+0000 7fd10d7fa700 1 -- 172.21.15.26:0/15636594 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22096 (secure 0 0 0) 0x7fd0f8099e00 con 0x7fd110100cf0 2023-07-13T22:12:20.395 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:20.395 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:19.954949+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 msgr2=0x7fd0fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 0x7fd0fc0a36d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd10000b2e0 tx=0x7fd100005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 msgr2=0x7fd110072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800e9a0 tx=0x7fd0f800ed60 comp rx=0 tx=0).stop 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 shutdown_connections 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd110101680 0x7fd110072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1100ffb00 0x7fd1100742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd0fc0a1220 0x7fd0fc0a36d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 --2- 172.21.15.26:0/15636594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd110100cf0 0x7fd110072900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.406+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 >> 172.21.15.26:0/15636594 conn(0x7fd1100fb0e0 msgr2=0x7fd1101041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.407+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 shutdown_connections 2023-07-13T22:12:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.407+0000 7fd115fff700 1 -- 172.21.15.26:0/15636594 wait complete. 2023-07-13T22:12:20.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-13T22:12:20.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: osdmap e82: 12 total, 12 up, 12 in 2023-07-13T22:12:20.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: pgmap v266: 97 pgs: 5 active+clean, 20 creating+peering, 72 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:12:20.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/761911543' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:12:20.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:19 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/721376000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:20.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 -- 172.21.15.26:0/2428548460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 msgr2=0x7ffb980ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 --2- 172.21.15.26:0/2428548460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb980ffef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c009990 tx=0x7ffb8c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 -- 172.21.15.26:0/2428548460 shutdown_connections 2023-07-13T22:12:20.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 --2- 172.21.15.26:0/2428548460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb980ffef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 --2- 172.21.15.26:0/2428548460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 0x7ffb98107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 --2- 172.21.15.26:0/2428548460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 0x7ffb98101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 -- 172.21.15.26:0/2428548460 >> 172.21.15.26:0/2428548460 conn(0x7ffb980fb0e0 msgr2=0x7ffb980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:20.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 -- 172.21.15.26:0/2428548460 shutdown_connections 2023-07-13T22:12:20.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.633+0000 7ffb9efff700 1 -- 172.21.15.26:0/2428548460 wait complete. 2023-07-13T22:12:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.634+0000 7ffb9efff700 1 Processor -- start 2023-07-13T22:12:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.634+0000 7ffb9efff700 1 -- start start 2023-07-13T22:12:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.634+0000 7ffb9efff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.634+0000 7ffb9efff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 0x7ffb98197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9efff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 0x7ffb9819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9efff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9819f630 con 0x7ffb980ffae0 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9efff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9819f770 con 0x7ffb98100cd0 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9efff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9819f900 con 0x7ffb98101660 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56990/0 (socket says 172.21.15.26:56990) 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 -- 172.21.15.26:0/157498856 learned_addr learned my addr 172.21.15.26:0/157498856 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:20.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb97fff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 0x7ffb98197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9d59c700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 0x7ffb9819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 -- 172.21.15.26:0/157498856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 msgr2=0x7ffb9819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 0x7ffb9819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 -- 172.21.15.26:0/157498856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 msgr2=0x7ffb98197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:20.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 0x7ffb98197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 -- 172.21.15.26:0/157498856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0095f0 con 0x7ffb980ffae0 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.635+0000 7ffb9cd9b700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b620 tx=0x7ffb8c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.636+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c01d070 con 0x7ffb980ffae0 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.636+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb8c021410 con 0x7ffb980ffae0 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.636+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c017650 con 0x7ffb980ffae0 2023-07-13T22:12:20.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.636+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9819fb80 con 0x7ffb980ffae0 2023-07-13T22:12:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.636+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9819ff50 con 0x7ffb980ffae0 2023-07-13T22:12:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.637+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb981921c0 con 0x7ffb980ffae0 2023-07-13T22:12:20.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.639+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffb8c021580 con 0x7ffb980ffae0 2023-07-13T22:12:20.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.640+0000 7ffb95ffb700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 0x7ffb880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:20.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.641+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffb8c0cdd30 con 0x7ffb980ffae0 2023-07-13T22:12:20.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.641+0000 7ffb97fff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 0x7ffb880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:20.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.641+0000 7ffb97fff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 0x7ffb880a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009c60 tx=0x7ffb84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:20.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.644+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffb8c0975c0 con 0x7ffb980ffae0 2023-07-13T22:12:20.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:20.903+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb9802c850 con 0x7ffb980ffae0 2023-07-13T22:12:21.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: osdmap e83: 12 total, 12 up, 12 in 2023-07-13T22:12:21.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/15636594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:21.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:active 2023-07-13T22:12:21.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.246 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.247 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:20 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-13T22:12:21.445 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: osdmap e83: 12 total, 12 up, 12 in 2023-07-13T22:12:21.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/15636594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:21.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:active 2023-07-13T22:12:21.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.446 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:20 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-13T22:12:21.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: osdmap e83: 12 total, 12 up, 12 in 2023-07-13T22:12:21.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/15636594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:21.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] up:active 2023-07-13T22:12:21.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:21.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:20 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-13T22:12:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.504+0000 7ffb95ffb700 1 -- 172.21.15.26:0/157498856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb8c09abe0 con 0x7ffb980ffae0 2023-07-13T22:12:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 msgr2=0x7ffb880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 0x7ffb880a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009c60 tx=0x7ffb84009260 comp rx=0 tx=0).stop 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 msgr2=0x7ffb98197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00b620 tx=0x7ffb8c005740 comp rx=0 tx=0).stop 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 shutdown_connections 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffb880a1150 0x7ffb880a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb980ffae0 0x7ffb98197ab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb98101660 0x7ffb9819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 --2- 172.21.15.26:0/157498856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb98100cd0 0x7ffb98197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 >> 172.21.15.26:0/157498856 conn(0x7ffb980fb0e0 msgr2=0x7ffb981065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 shutdown_connections 2023-07-13T22:12:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.507+0000 7ffb9efff700 1 -- 172.21.15.26:0/157498856 wait complete. 2023-07-13T22:12:21.518 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-07-13T22:12:21.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-07-13T22:12:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 -- 172.21.15.26:0/2629737807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc0ffaf0 msgr2=0x7fb3cc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/2629737807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0fff00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009990 tx=0x7fb3c8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 -- 172.21.15.26:0/2629737807 shutdown_connections 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/2629737807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0fff00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/2629737807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc101670 0x7fb3cc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/2629737807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 -- 172.21.15.26:0/2629737807 >> 172.21.15.26:0/2629737807 conn(0x7fb3cc0fb130 msgr2=0x7fb3cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 -- 172.21.15.26:0/2629737807 shutdown_connections 2023-07-13T22:12:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.739+0000 7fb3d39ff700 1 -- 172.21.15.26:0/2629737807 wait complete. 2023-07-13T22:12:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.740+0000 7fb3d39ff700 1 Processor -- start 2023-07-13T22:12:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.740+0000 7fb3d39ff700 1 -- start start 2023-07-13T22:12:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc0735e0 con 0x7fb3cc101670 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc073750 con 0x7fb3cc100ce0 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d39ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc19f8a0 con 0x7fb3cc0ffaf0 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56998/0 (socket says 172.21.15.26:56998) 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 -- 172.21.15.26:0/790747715 learned_addr learned my addr 172.21.15.26:0/790747715 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d0f9a700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d179b700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 -- 172.21.15.26:0/790747715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc0ffaf0 msgr2=0x7fb3cc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 -- 172.21.15.26:0/790747715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 msgr2=0x7fb3cc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d1f9c700 1 -- 172.21.15.26:0/790747715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80095f0 con 0x7fb3cc101670 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.741+0000 7fb3d0f9a700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3d1f9c700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4007ee0 tx=0x7fb3c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4017070 con 0x7fb3cc101670 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c400f040 con 0x7fb3cc101670 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc19fb80 con 0x7fb3cc101670 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005740 con 0x7fb3cc101670 2023-07-13T22:12:21.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.742+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc1a00e0 con 0x7fb3cc101670 2023-07-13T22:12:21.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.743+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3cc10b2c0 con 0x7fb3cc101670 2023-07-13T22:12:21.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.745+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3c400dc60 con 0x7fb3cc101670 2023-07-13T22:12:21.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.746+0000 7fb3c27fc700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 0x7fb3b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:21.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.746+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3c40cbfb0 con 0x7fb3cc101670 2023-07-13T22:12:21.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.746+0000 7fb3d179b700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 0x7fb3b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:21.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.747+0000 7fb3d179b700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 0x7fb3b80a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:21.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:21.750+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb3c40957b0 con 0x7fb3cc101670 2023-07-13T22:12:22.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.012+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fb3cc1069f0 con 0x7fb3cc101670 2023-07-13T22:12:22.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:22 smithi121 ceph-mon[129187]: pgmap v268: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-13T22:12:22.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-13T22:12:22.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:22 smithi121 ceph-mon[129187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:22.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-13T22:12:22.945 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: 2023-07-13T22:12:22.475+0000 7f709780d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-13T22:12:22.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:22 smithi117 ceph-mon[129074]: pgmap v268: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-13T22:12:22.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-13T22:12:22.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:22 smithi117 ceph-mon[129074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:22.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-13T22:12:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:22 smithi026 ceph-mon[95647]: pgmap v268: 97 pgs: 37 active+clean, 20 creating+peering, 40 unknown; 577 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-07-13T22:12:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/157498856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-13T22:12:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:22 smithi026 ceph-mon[95647]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:22.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-07-13T22:12:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.970+0000 7fb3c27fc700 1 -- 172.21.15.26:0/790747715 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb3c4098dd0 con 0x7fb3cc101670 2023-07-13T22:12:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.973+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 msgr2=0x7fb3b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.973+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 0x7fb3b80a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8006010 tx=0x7fb3c800b260 comp rx=0 tx=0).stop 2023-07-13T22:12:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.973+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 msgr2=0x7fb3cc072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.973+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4007ee0 tx=0x7fb3c400d3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:22.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 shutdown_connections 2023-07-13T22:12:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b80a11d0 0x7fb3b80a3680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc101670 0x7fb3cc072e40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc0ffaf0 0x7fb3cc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 --2- 172.21.15.26:0/790747715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc100ce0 0x7fb3cc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 >> 172.21.15.26:0/790747715 conn(0x7fb3cc0fb130 msgr2=0x7fb3cc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 shutdown_connections 2023-07-13T22:12:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:22.974+0000 7fb3d39ff700 1 -- 172.21.15.26:0/790747715 wait complete. 2023-07-13T22:12:22.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-13T22:12:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 -- 172.21.15.26:0/659537679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 msgr2=0x7f6f4c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 --2- 172.21.15.26:0/659537679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c101120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009990 tx=0x7f6f3c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 -- 172.21.15.26:0/659537679 shutdown_connections 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 --2- 172.21.15.26:0/659537679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 --2- 172.21.15.26:0/659537679 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 0x7f6f4c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 --2- 172.21.15.26:0/659537679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c101120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 -- 172.21.15.26:0/659537679 >> 172.21.15.26:0/659537679 conn(0x7f6f4c0fb0e0 msgr2=0x7f6f4c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 -- 172.21.15.26:0/659537679 shutdown_connections 2023-07-13T22:12:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.213+0000 7f6f547ff700 1 -- 172.21.15.26:0/659537679 wait complete. 2023-07-13T22:12:23.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 Processor -- start 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 -- start start 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 0x7f6f4c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19f5c0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19f700 con 0x7f6f4c101660 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.214+0000 7f6f547ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c19f890 con 0x7f6f4c0ffae0 2023-07-13T22:12:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f5259b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 0x7f6f4c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45804/0 (socket says 172.21.15.26:45804) 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 -- 172.21.15.26:0/1470927688 learned_addr learned my addr 172.21.15.26:0/1470927688 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f52d9c700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 -- 172.21.15.26:0/1470927688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 msgr2=0x7f6f4c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 0x7f6f4c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 -- 172.21.15.26:0/1470927688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 msgr2=0x7f6f4c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 -- 172.21.15.26:0/1470927688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0095f0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f52d9c700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.215+0000 7f6f51d9a700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b620 tx=0x7f6f3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.216+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c01d070 con 0x7f6f4c100cd0 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.216+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c19fb10 con 0x7f6f4c100cd0 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.216+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3c005030 con 0x7f6f4c100cd0 2023-07-13T22:12:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.216+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c004bf0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.216+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c19ff90 con 0x7f6f4c100cd0 2023-07-13T22:12:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.217+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c10b2b0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.219+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6f3c004d50 con 0x7f6f4c100cd0 2023-07-13T22:12:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.220+0000 7f6f437fe700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 0x7f6f380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.220+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f3c0cd8d0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.220+0000 7f6f5259b700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 0x7f6f380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.221+0000 7f6f5259b700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 0x7f6f380a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c100b30 tx=0x7f6f48009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:23.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.224+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6f3c097160 con 0x7f6f4c100cd0 2023-07-13T22:12:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.483+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f6f4c1a05a0 con 0x7f6f4c100cd0 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-07-13T22:12:23.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-07-13T22:12:23.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-13T22:12:23.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-13T22:12:23.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-13T22:12:23.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-13T22:12:23.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:23.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-07-13T22:12:23.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-07-13T22:12:23.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-13T22:12:23.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-13T22:12:23.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/790747715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-07-13T22:12:23.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-13T22:12:23.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-13T22:12:23.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2023-07-13T22:12:23.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: daemon mds.a is now active in filesystem cephfs as rank 1 2023-07-13T22:12:23.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-13T22:12:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.982+0000 7f6f437fe700 1 -- 172.21.15.26:0/1470927688 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6f3c09a780 con 0x7f6f4c100cd0 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 msgr2=0x7f6f380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 0x7f6f380a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c100b30 tx=0x7f6f48009260 comp rx=0 tx=0).stop 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 msgr2=0x7f6f4c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00b620 tx=0x7f6f3c004730 comp rx=0 tx=0).stop 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 shutdown_connections 2023-07-13T22:12:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f4c101660 0x7f6f4c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f4c0ffae0 0x7f6f4c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f380a1150 0x7f6f380a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 --2- 172.21.15.26:0/1470927688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c100cd0 0x7f6f4c197f80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 >> 172.21.15.26:0/1470927688 conn(0x7f6f4c0fb0e0 msgr2=0x7f6f4c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 shutdown_connections 2023-07-13T22:12:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:23.986+0000 7f6f547ff700 1 -- 172.21.15.26:0/1470927688 wait complete. 2023-07-13T22:12:24.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-13T22:12:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 -- 172.21.15.26:0/1451571858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc101660 msgr2=0x7f09bc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 --2- 172.21.15.26:0/1451571858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc101660 0x7f09bc107f20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b0a0 tx=0x7f09b400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 -- 172.21.15.26:0/1451571858 shutdown_connections 2023-07-13T22:12:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 --2- 172.21.15.26:0/1451571858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc101660 0x7f09bc107f20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 --2- 172.21.15.26:0/1451571858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 0x7f09bc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 --2- 172.21.15.26:0/1451571858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc100cd0 0x7f09bc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 -- 172.21.15.26:0/1451571858 >> 172.21.15.26:0/1451571858 conn(0x7f09bc0fb0e0 msgr2=0x7f09bc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 -- 172.21.15.26:0/1451571858 shutdown_connections 2023-07-13T22:12:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.228+0000 7f09c0dff700 1 -- 172.21.15.26:0/1451571858 wait complete. 2023-07-13T22:12:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 Processor -- start 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 -- start start 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 0x7f09bc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc101660 0x7f09bc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0735e0 con 0x7f09bc100cd0 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc073750 con 0x7f09bc101660 2023-07-13T22:12:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.229+0000 7f09c0dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc19f8f0 con 0x7f09bc0ffae0 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45836/0 (socket says 172.21.15.26:45836) 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 -- 172.21.15.26:0/2451727274 learned_addr learned my addr 172.21.15.26:0/2451727274 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09bad9d700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc101660 0x7f09bc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09ba59c700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 0x7f09bc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 -- 172.21.15.26:0/2451727274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 msgr2=0x7f09bc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 0x7f09bc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 -- 172.21.15.26:0/2451727274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc101660 msgr2=0x7f09bc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc101660 0x7f09bc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 -- 172.21.15.26:0/2451727274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4009520 con 0x7f09bc100cd0 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.230+0000 7f09b3fff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f09a400eb30 tx=0x7f09a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.231+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4009880 con 0x7f09bc100cd0 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.231+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09a40044e0 con 0x7f09bc100cd0 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.231+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003d80 con 0x7f09bc100cd0 2023-07-13T22:12:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.231+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b4009d00 con 0x7f09bc100cd0 2023-07-13T22:12:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.231+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc19ff00 con 0x7f09bc100cd0 2023-07-13T22:12:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.232+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc107000 con 0x7f09bc100cd0 2023-07-13T22:12:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.234+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f09a4003f20 con 0x7f09bc100cd0 2023-07-13T22:12:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.236+0000 7f09b37fe700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 0x7f09a80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.237+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f09a4014070 con 0x7f09bc100cd0 2023-07-13T22:12:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.237+0000 7f09ba59c700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 0x7f09a80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.237+0000 7f09ba59c700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 0x7f09a80a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09bc100b30 tx=0x7f09ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.240+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f09a4096590 con 0x7f09bc100cd0 2023-07-13T22:12:24.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:24.502+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f09bc02c850 con 0x7f09bc100cd0 2023-07-13T22:12:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: pgmap v269: 97 pgs: 83 active+clean, 14 creating+peering; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-07-13T22:12:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:active 2023-07-13T22:12:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:active 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-13T22:12:24.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-13T22:12:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-13T22:12:24.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-13T22:12:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: pgmap v269: 97 pgs: 83 active+clean, 14 creating+peering; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-07-13T22:12:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:active 2023-07-13T22:12:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-13T22:12:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:active 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:24.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-13T22:12:24.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-13T22:12:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: pgmap v269: 97 pgs: 83 active+clean, 14 creating+peering; 579 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-07-13T22:12:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] up:active 2023-07-13T22:12:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1470927688' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-13T22:12:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-13T22:12:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:creating} 9 up:standby 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] up:active 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:creating} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:24.997 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-07-13T22:12:24.998 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-13T22:12:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.438+0000 7f09b37fe700 1 -- 172.21.15.26:0/2451727274 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) v1 ==== 115+0+0 (secure 0 0 0) 0x7f09a4094e60 con 0x7f09bc100cd0 2023-07-13T22:12:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.442+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 msgr2=0x7f09a80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.442+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 0x7f09a80a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09bc100b30 tx=0x7f09ac009260 comp rx=0 tx=0).stop 2023-07-13T22:12:25.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.442+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 msgr2=0x7f09bc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.442+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f09a400eb30 tx=0x7f09a400ee40 comp rx=0 tx=0).stop 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 shutdown_connections 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09bc101660 0x7f09bc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09bc0ffae0 0x7f09bc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09a80a1150 0x7f09a80a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 --2- 172.21.15.26:0/2451727274 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09bc100cd0 0x7f09bc072900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 >> 172.21.15.26:0/2451727274 conn(0x7f09bc0fb0e0 msgr2=0x7f09bc104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 shutdown_connections 2023-07-13T22:12:25.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.443+0000 7f09c0dff700 1 -- 172.21.15.26:0/2451727274 wait complete. 2023-07-13T22:12:25.462 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-07-13T22:12:25.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-13T22:12:25.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 -- 172.21.15.26:0/2415736333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c0ffae0 msgr2=0x7ff52c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2415736333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c0ffae0 0x7ff52c0ffef0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009990 tx=0x7ff51c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 -- 172.21.15.26:0/2415736333 shutdown_connections 2023-07-13T22:12:25.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2415736333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c0ffae0 0x7ff52c0ffef0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2415736333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c101660 0x7ff52c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2415736333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c100cd0 0x7ff52c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 -- 172.21.15.26:0/2415736333 >> 172.21.15.26:0/2415736333 conn(0x7ff52c0fb0e0 msgr2=0x7ff52c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:25.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 -- 172.21.15.26:0/2415736333 shutdown_connections 2023-07-13T22:12:25.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.685+0000 7ff5325ff700 1 -- 172.21.15.26:0/2415736333 wait complete. 2023-07-13T22:12:25.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.686+0000 7ff5325ff700 1 Processor -- start 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.686+0000 7ff5325ff700 1 -- start start 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.686+0000 7ff5325ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c0ffae0 0x7ff52c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.686+0000 7ff5325ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff5325ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c101660 0x7ff52c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff5325ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c19f6a0 con 0x7ff52c100cd0 2023-07-13T22:12:25.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff5325ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c19f7e0 con 0x7ff52c101660 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff5325ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c19f970 con 0x7ff52c0ffae0 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45854/0 (socket says 172.21.15.26:45854) 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 -- 172.21.15.26:0/2963610886 learned_addr learned my addr 172.21.15.26:0/2963610886 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff530b9c700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c101660 0x7ff52c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff52bfff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c0ffae0 0x7ff52c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:25.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 -- 172.21.15.26:0/2963610886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c0ffae0 msgr2=0x7ff52c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c0ffae0 0x7ff52c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 -- 172.21.15.26:0/2963610886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c101660 msgr2=0x7ff52c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c101660 0x7ff52c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 -- 172.21.15.26:0/2963610886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0095f0 con 0x7ff52c100cd0 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.687+0000 7ff523fff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.688+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400f890 con 0x7ff52c100cd0 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.688+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c19fc50 con 0x7ff52c100cd0 2023-07-13T22:12:25.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.688+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff514010460 con 0x7ff52c100cd0 2023-07-13T22:12:25.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.688+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400e610 con 0x7ff52c100cd0 2023-07-13T22:12:25.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.688+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1a0160 con 0x7ff52c100cd0 2023-07-13T22:12:25.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.689+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c1921c0 con 0x7ff52c100cd0 2023-07-13T22:12:25.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.691+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff5140108d0 con 0x7ff52c100cd0 2023-07-13T22:12:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.692+0000 7ff529ffb700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.693+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff5140cc050 con 0x7ff52c100cd0 2023-07-13T22:12:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.693+0000 7ff52bfff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:25.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.693+0000 7ff52bfff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 0x7ff5180a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b5c0 tx=0x7ff51c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.696+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff514095850 con 0x7ff52c100cd0 2023-07-13T22:12:25.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.943+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7ff52c060950 con 0x7ff5180a1150 2023-07-13T22:12:25.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.990+0000 7ff529ffb700 1 -- 172.21.15.26:0/2963610886 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff52c060950 con 0x7ff5180a1150 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 msgr2=0x7ff5180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 0x7ff5180a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00b5c0 tx=0x7ff51c005fd0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 msgr2=0x7ff52c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff51400d950 tx=0x7ff51400dc60 comp rx=0 tx=0).stop 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 shutdown_connections 2023-07-13T22:12:25.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff52c101660 0x7ff52c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff52c0ffae0 0x7ff52c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5180a1150 0x7ff5180a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 --2- 172.21.15.26:0/2963610886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52c100cd0 0x7ff52c198060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 >> 172.21.15.26:0/2963610886 conn(0x7ff52c0fb0e0 msgr2=0x7ff52c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 shutdown_connections 2023-07-13T22:12:25.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:25.993+0000 7ff5325ff700 1 -- 172.21.15.26:0/2963610886 wait complete. 2023-07-13T22:12:26.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-13T22:12:26.203 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:12:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:12:25.950+0000 7fb16d20d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-13T22:12:26.203 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:12:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:12:25.950+0000 7fb16d20d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-13T22:12:26.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:12:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:12:25.950+0000 7fb16d20d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-13T22:12:26.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:12:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:12:25.950+0000 7fb16d20d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-13T22:12:26.204 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:12:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:12:25.950+0000 7fb16d20d700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-13T22:12:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 -- 172.21.15.26:0/405433569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 msgr2=0x7f50340fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 --2- 172.21.15.26:0/405433569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340fff00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5024009990 tx=0x7f5024009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 -- 172.21.15.26:0/405433569 shutdown_connections 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 --2- 172.21.15.26:0/405433569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340fff00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 --2- 172.21.15.26:0/405433569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 0x7f5034107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 --2- 172.21.15.26:0/405433569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 0x7f5034101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 -- 172.21.15.26:0/405433569 >> 172.21.15.26:0/405433569 conn(0x7f50340fb130 msgr2=0x7f50340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 -- 172.21.15.26:0/405433569 shutdown_connections 2023-07-13T22:12:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.220+0000 7f50393ff700 1 -- 172.21.15.26:0/405433569 wait complete. 2023-07-13T22:12:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.221+0000 7f50393ff700 1 Processor -- start 2023-07-13T22:12:26.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 -- start start 2023-07-13T22:12:26.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 0x7f5034072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 0x7f5034072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50340735e0 con 0x7f50340ffaf0 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5034073750 con 0x7f5034101670 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f50393ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503419f8e0 con 0x7f5034100ce0 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f5032ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f5032ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45888/0 (socket says 172.21.15.26:45888) 2023-07-13T22:12:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.222+0000 7f5032ffd700 1 -- 172.21.15.26:0/2097820980 learned_addr learned my addr 172.21.15.26:0/2097820980 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f50337fe700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 0x7f5034072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f502bfff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 0x7f5034072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 -- 172.21.15.26:0/2097820980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 msgr2=0x7f5034072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 0x7f5034072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 -- 172.21.15.26:0/2097820980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 msgr2=0x7f5034072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 0x7f5034072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 -- 172.21.15.26:0/2097820980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240095f0 con 0x7f50340ffaf0 2023-07-13T22:12:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5032ffd700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5024005740 tx=0x7f5024005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502401d070 con 0x7f50340ffaf0 2023-07-13T22:12:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5024021410 con 0x7f50340ffaf0 2023-07-13T22:12:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503419fb60 con 0x7f50340ffaf0 2023-07-13T22:12:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.223+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024017650 con 0x7f50340ffaf0 2023-07-13T22:12:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.224+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503419ffe0 con 0x7f50340ffaf0 2023-07-13T22:12:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.225+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50341070b0 con 0x7f50340ffaf0 2023-07-13T22:12:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.226+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5024021580 con 0x7f50340ffaf0 2023-07-13T22:12:26.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.228+0000 7f5030ff9700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 0x7f50200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.228+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50240cd860 con 0x7f50340ffaf0 2023-07-13T22:12:26.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.228+0000 7f502bfff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 0x7f50200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.229+0000 7f502bfff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 0x7f50200a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f501c009c60 tx=0x7f501c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:26.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.232+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f50240972b0 con 0x7f50340ffaf0 2023-07-13T22:12:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.479+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5034060950 con 0x7f50200a11d0 2023-07-13T22:12:26.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.483+0000 7f5030ff9700 1 -- 172.21.15.26:0/2097820980 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f5034060950 con 0x7f50200a11d0 2023-07-13T22:12:26.473 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv_0/bd1f1f7a-d08e-42c2-8ccf-037bcce918b9 2023-07-13T22:12:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 msgr2=0x7f50200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 0x7f50200a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f501c009c60 tx=0x7f501c009260 comp rx=0 tx=0).stop 2023-07-13T22:12:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 msgr2=0x7f50340742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5024005740 tx=0x7f5024005770 comp rx=0 tx=0).stop 2023-07-13T22:12:26.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 shutdown_connections 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50200a11d0 0x7f50200a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50340ffaf0 0x7f50340742c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5034101670 0x7f5034072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 --2- 172.21.15.26:0/2097820980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5034100ce0 0x7f5034072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 >> 172.21.15.26:0/2097820980 conn(0x7f50340fb130 msgr2=0x7f50341041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 shutdown_connections 2023-07-13T22:12:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.485+0000 7f50393ff700 1 -- 172.21.15.26:0/2097820980 wait complete. 2023-07-13T22:12:26.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-13T22:12:26.695 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2023-07-13T22:12:26.695 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:active 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: Cluster is now healthy 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-07-13T22:12:26.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:26 smithi117 ceph-mon[129074]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-13T22:12:26.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2023-07-13T22:12:26.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:active 2023-07-13T22:12:26.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-13T22:12:26.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-13T22:12:26.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-13T22:12:26.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: Cluster is now healthy 2023-07-13T22:12:26.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:26.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-07-13T22:12:26.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:26 smithi026 ceph-mon[95647]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 -- 172.21.15.26:0/2010245028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14100cd0 msgr2=0x7fcc14101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 --2- 172.21.15.26:0/2010245028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14100cd0 0x7fcc14101120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009990 tx=0x7fcbfc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 -- 172.21.15.26:0/2010245028 shutdown_connections 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 --2- 172.21.15.26:0/2010245028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc14101660 0x7fcc14107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 --2- 172.21.15.26:0/2010245028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc140ffae0 0x7fcc140ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 --2- 172.21.15.26:0/2010245028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14100cd0 0x7fcc14101120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 -- 172.21.15.26:0/2010245028 >> 172.21.15.26:0/2010245028 conn(0x7fcc140fb0e0 msgr2=0x7fcc140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 -- 172.21.15.26:0/2010245028 shutdown_connections 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.713+0000 7fcc191ff700 1 -- 172.21.15.26:0/2010245028 wait complete. 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.714+0000 7fcc191ff700 1 Processor -- start 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.714+0000 7fcc191ff700 1 -- start start 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.714+0000 7fcc191ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc140ffae0 0x7fcc140742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.714+0000 7fcc191ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc14100cd0 0x7fcc14072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc191ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc191ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140735e0 con 0x7fcc14101660 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc191ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14073750 con 0x7fcc14100cd0 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc191ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1419f8c0 con 0x7fcc140ffae0 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45894/0 (socket says 172.21.15.26:45894) 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 -- 172.21.15.26:0/3353710073 learned_addr learned my addr 172.21.15.26:0/3353710073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:26.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc0bfff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc14100cd0 0x7fcc14072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc12d9d700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc140ffae0 0x7fcc140742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 -- 172.21.15.26:0/3353710073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc140ffae0 msgr2=0x7fcc140742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc140ffae0 0x7fcc140742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 -- 172.21.15.26:0/3353710073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc14100cd0 msgr2=0x7fcc14072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc14100cd0 0x7fcc14072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 -- 172.21.15.26:0/3353710073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0095f0 con 0x7fcc14101660 2023-07-13T22:12:26.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.715+0000 7fcc1359e700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00ef30 tx=0x7fcc0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.716+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c018070 con 0x7fcc14101660 2023-07-13T22:12:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.716+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc0c00f040 con 0x7fcc14101660 2023-07-13T22:12:26.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.716+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00cd00 con 0x7fcc14101660 2023-07-13T22:12:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.716+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1419fb40 con 0x7fcc14101660 2023-07-13T22:12:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.716+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc141a0050 con 0x7fcc14101660 2023-07-13T22:12:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.717+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1410b300 con 0x7fcc14101660 2023-07-13T22:12:26.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.719+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcc0c003a00 con 0x7fcc14101660 2023-07-13T22:12:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.720+0000 7fcc10d99700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 0x7fcc000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.720+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc0c0cc2c0 con 0x7fcc14101660 2023-07-13T22:12:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.720+0000 7fcc12d9d700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 0x7fcc000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:26.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.721+0000 7fcc12d9d700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 0x7fcc000a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc14100b30 tx=0x7fcc04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:26.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.724+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcc0c095b50 con 0x7fcc14101660 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: pgmap v270: 97 pgs: 97 active+clean; 580 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] up:active 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2451727274' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: Cluster is now healthy 2023-07-13T22:12:26.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=h=up:active,3=f=up:active} 3 up:standby-replay 5 up:standby 2023-07-13T22:12:26.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2023-07-13T22:12:26.747 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:26 smithi121 ceph-mon[129187]: daemon mds.k is now active in filesystem cephfs as rank 4 2023-07-13T22:12:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.967+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc14060950 con 0x7fcc000a11d0 2023-07-13T22:12:26.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.992+0000 7fcc10d99700 1 -- 172.21.15.26:0/3353710073 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcc14060950 con 0x7fcc000a11d0 2023-07-13T22:12:26.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 msgr2=0x7fcc000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 0x7fcc000a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc14100b30 tx=0x7fcc04009260 comp rx=0 tx=0).stop 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 msgr2=0x7fcc14072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00ef30 tx=0x7fcc0c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 shutdown_connections 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcc000a11d0 0x7fcc000a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc14101660 0x7fcc14072e40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcc140ffae0 0x7fcc140742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 --2- 172.21.15.26:0/3353710073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc14100cd0 0x7fcc14072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 >> 172.21.15.26:0/3353710073 conn(0x7fcc140fb0e0 msgr2=0x7fcc140fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 shutdown_connections 2023-07-13T22:12:26.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:26.994+0000 7fcc191ff700 1 -- 172.21.15.26:0/3353710073 wait complete. 2023-07-13T22:12:27.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-13T22:12:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 -- 172.21.15.26:0/2382609409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f81079a0 msgr2=0x7f69f8109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/2382609409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f81079a0 0x7f69f8109e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f69f000b0a0 tx=0x7f69f000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 -- 172.21.15.26:0/2382609409 shutdown_connections 2023-07-13T22:12:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/2382609409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f81079a0 0x7f69f8109e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/2382609409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f8074550 0x7f69f80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/2382609409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8072f70 0x7f69f80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 -- 172.21.15.26:0/2382609409 >> 172.21.15.26:0/2382609409 conn(0x7f69f80fb0e0 msgr2=0x7f69f80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 -- 172.21.15.26:0/2382609409 shutdown_connections 2023-07-13T22:12:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.220+0000 7f69fd5ff700 1 -- 172.21.15.26:0/2382609409 wait complete. 2023-07-13T22:12:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.221+0000 7f69fd5ff700 1 Processor -- start 2023-07-13T22:12:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.221+0000 7f69fd5ff700 1 -- start start 2023-07-13T22:12:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8074550 0x7f69f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f81079a0 0x7f69f819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f819f720 con 0x7f69f8072f70 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f819f860 con 0x7f69f8074550 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69fd5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f819f9f0 con 0x7f69f81079a0 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45916/0 (socket says 172.21.15.26:45916) 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 -- 172.21.15.26:0/1805398436 learned_addr learned my addr 172.21.15.26:0/1805398436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f67fc700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8074550 0x7f69f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f77fe700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f81079a0 0x7f69f819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 -- 172.21.15.26:0/1805398436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f81079a0 msgr2=0x7f69f819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f81079a0 0x7f69f819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 -- 172.21.15.26:0/1805398436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8074550 msgr2=0x7f69f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8074550 0x7f69f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.222+0000 7f69f6ffd700 1 -- 172.21.15.26:0/1805398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009d00 con 0x7f69f8072f70 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69f6ffd700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e970 tx=0x7f69e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8009790 con 0x7f69f8072f70 2023-07-13T22:12:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e800c7e0 con 0x7f69f8072f70 2023-07-13T22:12:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80054e0 con 0x7f69f8072f70 2023-07-13T22:12:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f819fcd0 con 0x7f69f8072f70 2023-07-13T22:12:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.223+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f81a0190 con 0x7f69f8072f70 2023-07-13T22:12:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.224+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8106050 con 0x7f69f8072f70 2023-07-13T22:12:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.226+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f69e800c950 con 0x7f69f8072f70 2023-07-13T22:12:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.227+0000 7f69dffff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 0x7f69e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.227+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f69e8014070 con 0x7f69f8072f70 2023-07-13T22:12:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.228+0000 7f69f67fc700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 0x7f69e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.228+0000 7f69f67fc700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 0x7f69e00a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69f8074270 tx=0x7f69ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.231+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f69e8096270 con 0x7f69f8072f70 2023-07-13T22:12:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.480+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f8060950 con 0x7f69e00a1150 2023-07-13T22:12:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.484+0000 7f69dffff700 1 -- 172.21.15.26:0/1805398436 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f69f8060950 con 0x7f69e00a1150 2023-07-13T22:12:27.474 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv_1/304bc8c3-9709-4ccf-9b77-737ad5afecf4 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 msgr2=0x7f69e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 0x7f69e00a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f69f8074270 tx=0x7f69ec009260 comp rx=0 tx=0).stop 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 msgr2=0x7f69f8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e970 tx=0x7f69e800ec80 comp rx=0 tx=0).stop 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 shutdown_connections 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69e00a1150 0x7f69e00a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f8072f70 0x7f69f8197ba0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f81079a0 0x7f69f819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 --2- 172.21.15.26:0/1805398436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f8074550 0x7f69f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 >> 172.21.15.26:0/1805398436 conn(0x7f69f80fb0e0 msgr2=0x7f69f80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 shutdown_connections 2023-07-13T22:12:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.486+0000 7f69fd5ff700 1 -- 172.21.15.26:0/1805398436 wait complete. 2023-07-13T22:12:27.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:12:27.695 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: 2023-07-13T22:12:27.476+0000 7f709780d700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-07-13T22:12:27.695 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-mon[129074]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:active 2023-07-13T22:12:27.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-mon[129074]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-07-13T22:12:27.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-mon[129074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:12:27.696 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:27 smithi117 ceph-mon[129074]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 -- 172.21.15.26:0/2605569878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c101680 msgr2=0x7f300c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 --2- 172.21.15.26:0/2605569878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c101680 0x7f300c107f40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f300400b0a0 tx=0x7f300400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 -- 172.21.15.26:0/2605569878 shutdown_connections 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 --2- 172.21.15.26:0/2605569878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c101680 0x7f300c107f40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 --2- 172.21.15.26:0/2605569878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c0ffb00 0x7f300c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 --2- 172.21.15.26:0/2605569878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 0x7f300c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 -- 172.21.15.26:0/2605569878 >> 172.21.15.26:0/2605569878 conn(0x7f300c0fb0e0 msgr2=0x7f300c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 -- 172.21.15.26:0/2605569878 shutdown_connections 2023-07-13T22:12:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.708+0000 7f30147ff700 1 -- 172.21.15.26:0/2605569878 wait complete. 2023-07-13T22:12:27.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.709+0000 7f30147ff700 1 Processor -- start 2023-07-13T22:12:27.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 -- start start 2023-07-13T22:12:27.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 0x7f300c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c101680 0x7f300c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c19d480 con 0x7f300c0ffb00 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c19d5c0 con 0x7f300c100cf0 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f30147ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c19d750 con 0x7f300c101680 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f301259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f301259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45936/0 (socket says 172.21.15.26:45936) 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f301259b700 1 -- 172.21.15.26:0/2731407671 learned_addr learned my addr 172.21.15.26:0/2731407671 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f3012d9c700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c101680 0x7f300c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.710+0000 7f3011d9a700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 0x7f300c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 -- 172.21.15.26:0/2731407671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c101680 msgr2=0x7f300c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c101680 0x7f300c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 -- 172.21.15.26:0/2731407671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 msgr2=0x7f300c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 0x7f300c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 -- 172.21.15.26:0/2731407671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004009d00 con 0x7f300c0ffb00 2023-07-13T22:12:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f301259b700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300800e970 tx=0x7f300800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008009790 con 0x7f300c0ffb00 2023-07-13T22:12:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f300800c7e0 con 0x7f300c0ffb00 2023-07-13T22:12:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c19da30 con 0x7f300c0ffb00 2023-07-13T22:12:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008003c70 con 0x7f300c0ffb00 2023-07-13T22:12:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.711+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c19df40 con 0x7f300c0ffb00 2023-07-13T22:12:27.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:27 smithi026 ceph-mon[95647]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:active 2023-07-13T22:12:27.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:27 smithi026 ceph-mon[95647]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-07-13T22:12:27.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:27 smithi026 ceph-mon[95647]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:12:27.705 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:27 smithi026 ceph-mon[95647]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.713+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c10b330 con 0x7f300c0ffb00 2023-07-13T22:12:27.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.715+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f300800cc50 con 0x7f300c0ffb00 2023-07-13T22:12:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.716+0000 7f30037fe700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 0x7f2ff80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.716+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3008014070 con 0x7f300c0ffb00 2023-07-13T22:12:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.716+0000 7f3011d9a700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 0x7f2ff80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.717+0000 7f3011d9a700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 0x7f2ff80a3650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009c60 tx=0x7f2ffc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.720+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f30080961f0 con 0x7f300c0ffb00 2023-07-13T22:12:27.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:27 smithi121 ceph-mon[129187]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] up:active 2023-07-13T22:12:27.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:27 smithi121 ceph-mon[129187]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2023-07-13T22:12:27.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:27 smithi121 ceph-mon[129187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:12:27.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:27 smithi121 ceph-mon[129187]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:27.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.972+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f300c10b550 con 0x7f300c0ffb00 2023-07-13T22:12:27.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.972+0000 7f30037fe700 1 -- 172.21.15.26:0/2731407671 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f3008003dd0 con 0x7f300c0ffb00 2023-07-13T22:12:27.964 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:27.965 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:12:27.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 msgr2=0x7f2ff80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 0x7f2ff80a3650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009c60 tx=0x7f2ffc009260 comp rx=0 tx=0).stop 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 msgr2=0x7f300c195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f300800e970 tx=0x7f300800ec80 comp rx=0 tx=0).stop 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 shutdown_connections 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ff80a11a0 0x7f2ff80a3650 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300c0ffb00 0x7f300c195900 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f300c101680 0x7f300c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 --2- 172.21.15.26:0/2731407671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f300c100cf0 0x7f300c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 >> 172.21.15.26:0/2731407671 conn(0x7f300c0fb0e0 msgr2=0x7f300c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 shutdown_connections 2023-07-13T22:12:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:27.975+0000 7f30147ff700 1 -- 172.21.15.26:0/2731407671 wait complete. 2023-07-13T22:12:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:12:27.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:12:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 -- 172.21.15.26:0/2069684116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa500ffb00 msgr2=0x7faa500fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 --2- 172.21.15.26:0/2069684116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa500ffb00 0x7faa500fff10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faa40009990 tx=0x7faa40009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 -- 172.21.15.26:0/2069684116 shutdown_connections 2023-07-13T22:12:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 --2- 172.21.15.26:0/2069684116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa500ffb00 0x7faa500fff10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 --2- 172.21.15.26:0/2069684116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50101680 0x7faa50107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 --2- 172.21.15.26:0/2069684116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50100cf0 0x7faa50101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 -- 172.21.15.26:0/2069684116 >> 172.21.15.26:0/2069684116 conn(0x7faa500fb0e0 msgr2=0x7faa500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 -- 172.21.15.26:0/2069684116 shutdown_connections 2023-07-13T22:12:28.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.198+0000 7faa54bff700 1 -- 172.21.15.26:0/2069684116 wait complete. 2023-07-13T22:12:28.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.199+0000 7faa54bff700 1 Processor -- start 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.199+0000 7faa54bff700 1 -- start start 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.199+0000 7faa54bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500ffb00 0x7faa50197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa54bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa54bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50101680 0x7faa5019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa54bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019f600 con 0x7faa50100cf0 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa54bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019f740 con 0x7faa500ffb00 2023-07-13T22:12:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa54bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019f8d0 con 0x7faa50101680 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45956/0 (socket says 172.21.15.26:45956) 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 -- 172.21.15.26:0/996396 learned_addr learned my addr 172.21.15.26:0/996396 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4e59c700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500ffb00 0x7faa50197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4ed9d700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50101680 0x7faa5019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 -- 172.21.15.26:0/996396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50101680 msgr2=0x7faa5019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50101680 0x7faa5019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 -- 172.21.15.26:0/996396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500ffb00 msgr2=0x7faa50197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500ffb00 0x7faa50197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 -- 172.21.15.26:0/996396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400095f0 con 0x7faa50100cf0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.200+0000 7faa4dd9b700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e9a0 tx=0x7faa4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.201+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44009790 con 0x7faa50100cf0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.201+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa4400cbd0 con 0x7faa50100cf0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.201+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5019fbb0 con 0x7faa50100cf0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.201+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44018680 con 0x7faa50100cf0 2023-07-13T22:12:28.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.201+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa501a00c0 con 0x7faa50100cf0 2023-07-13T22:12:28.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.202+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa501921e0 con 0x7faa50100cf0 2023-07-13T22:12:28.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.204+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7faa4400cd40 con 0x7faa50100cf0 2023-07-13T22:12:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.205+0000 7faa3f7fe700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 0x7faa380a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.205+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa44014070 con 0x7faa50100cf0 2023-07-13T22:12:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.205+0000 7faa4e59c700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 0x7faa380a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.206+0000 7faa4e59c700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 0x7faa380a3650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b2e0 tx=0x7faa40005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:28.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.209+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faa44051790 con 0x7faa50100cf0 2023-07-13T22:12:28.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.417+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa5002caa0 con 0x7faa50100cf0 2023-07-13T22:12:28.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.419+0000 7faa3f7fe700 1 -- 172.21.15.26:0/996396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22096 (secure 0 0 0) 0x7faa44096300 con 0x7faa50100cf0 2023-07-13T22:12:28.411 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:28.411 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:19.954949+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 msgr2=0x7faa380a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 0x7faa380a3650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b2e0 tx=0x7faa40005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 msgr2=0x7faa50197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faa4400e9a0 tx=0x7faa4400ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 shutdown_connections 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa50101680 0x7faa5019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa500ffb00 0x7faa50197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faa380a11a0 0x7faa380a3650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 --2- 172.21.15.26:0/996396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa50100cf0 0x7faa50197fc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.421+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 >> 172.21.15.26:0/996396 conn(0x7faa500fb0e0 msgr2=0x7faa501065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.422+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 shutdown_connections 2023-07-13T22:12:28.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.422+0000 7faa54bff700 1 -- 172.21.15.26:0/996396 wait complete. 2023-07-13T22:12:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-13T22:12:28.433 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:12:28.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.649+0000 7f1e18bff700 1 -- 172.21.15.26:0/3832662152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14100cf0 msgr2=0x7f1e14101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.649+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3832662152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14100cf0 0x7f1e14101140 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009990 tx=0x7f1dfc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 -- 172.21.15.26:0/3832662152 shutdown_connections 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3832662152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14101680 0x7f1e14107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3832662152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e140fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3832662152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14100cf0 0x7f1e14101140 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 -- 172.21.15.26:0/3832662152 >> 172.21.15.26:0/3832662152 conn(0x7f1e140fb0e0 msgr2=0x7f1e140fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 -- 172.21.15.26:0/3832662152 shutdown_connections 2023-07-13T22:12:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.650+0000 7f1e18bff700 1 -- 172.21.15.26:0/3832662152 wait complete. 2023-07-13T22:12:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 Processor -- start 2023-07-13T22:12:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 -- start start 2023-07-13T22:12:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e14197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14100cf0 0x7f1e14197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419f600 con 0x7f1e14101680 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419f740 con 0x7f1e14100cf0 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.651+0000 7f1e18bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1419f8d0 con 0x7f1e140ffb00 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e1259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e14197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45970/0 (socket says 172.21.15.26:45970) 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e1259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e14197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33826/0 (socket says 172.21.15.26:33826) 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 -- 172.21.15.26:0/3273088161 learned_addr learned my addr 172.21.15.26:0/3273088161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e0bfff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14100cf0 0x7f1e14197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 -- 172.21.15.26:0/3273088161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 msgr2=0x7f1e14197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e14197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 -- 172.21.15.26:0/3273088161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14100cf0 msgr2=0x7f1e14197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14100cf0 0x7f1e14197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 -- 172.21.15.26:0/3273088161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0095f0 con 0x7f1e14101680 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e12d9d700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00ef30 tx=0x7f1e0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.652+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c018070 con 0x7f1e14101680 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.653+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e0c00f040 con 0x7f1e14101680 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.653+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00cd00 con 0x7f1e14101680 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.653+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1419fb50 con 0x7f1e14101680 2023-07-13T22:12:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.653+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e141a0060 con 0x7f1e14101680 2023-07-13T22:12:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.654+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e1410b320 con 0x7f1e14101680 2023-07-13T22:12:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.656+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1e0c003a00 con 0x7f1e14101680 2023-07-13T22:12:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.658+0000 7f1e0b7fe700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 0x7f1e000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.658+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1e0c0cc400 con 0x7f1e14101680 2023-07-13T22:12:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.658+0000 7f1e1259c700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 0x7f1e000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.659+0000 7f1e1259c700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 0x7f1e000a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e14100b50 tx=0x7f1e04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.662+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1e0c095b30 con 0x7f1e14101680 2023-07-13T22:12:28.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:28 smithi026 ceph-mon[95647]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:28 smithi026 ceph-mon[95647]: pgmap v271: 97 pgs: 97 active+clean; 604 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2023-07-13T22:12:28.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:28 smithi026 ceph-mon[95647]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2731407671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:28.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/996396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:28.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:28 smithi121 ceph-mon[129187]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:28 smithi121 ceph-mon[129187]: pgmap v271: 97 pgs: 97 active+clean; 604 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2023-07-13T22:12:28.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:28 smithi121 ceph-mon[129187]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:28 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2731407671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:28.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:28 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/996396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.867+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e14107cb0 con 0x7f1e000a12a0 2023-07-13T22:12:28.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.880+0000 7f1e0b7fe700 1 -- 172.21.15.26:0/3273088161 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335554 (secure 0 0 0) 0x7f1e14107cb0 con 0x7f1e000a12a0 2023-07-13T22:12:28.875 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 msgr2=0x7f1e000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 0x7f1e000a3750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e14100b50 tx=0x7f1e04009260 comp rx=0 tx=0).stop 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 msgr2=0x7f1e1419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00ef30 tx=0x7f1e0c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 shutdown_connections 2023-07-13T22:12:28.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e000a12a0 0x7f1e000a3750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e14101680 0x7f1e1419f030 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e140ffb00 0x7f1e14197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 --2- 172.21.15.26:0/3273088161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e14100cf0 0x7f1e14197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 >> 172.21.15.26:0/3273088161 conn(0x7f1e140fb0e0 msgr2=0x7f1e14106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 shutdown_connections 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:28.885+0000 7f1e18bff700 1 -- 172.21.15.26:0/3273088161 wait complete. 2023-07-13T22:12:28.879 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:12:28.895 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2023-07-13T22:12:27.419401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618331,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":343,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335832,"kb_used_data":12408,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605992,"statfs":{"total":1151940427776,"available":1151596535808,"internally_reserved":0,"allocated":12705792,"data_stored":6911283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27963,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":35,"num_read_kb":45,"num_write":162,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.464160"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.970024+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:12:19.970024+0000","last_peered":"2023-07-13T22:12:19.970024+0000","last_clean":"2023-07-13T22:12:19.970024+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:12:19.970024+0000","last_undegraded":"2023-07-13T22:12:19.970024+0000","last_fullsized":"2023-07-13T22:12:19.970024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:58.393428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253769+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:12:20.253769+0000","last_peered":"2023-07-13T22:12:20.253769+0000","last_clean":"2023-07-13T22:12:20.253769+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:12:20.253769+0000","last_undegraded":"2023-07-13T22:12:20.253769+0000","last_fullsized":"2023-07-13T22:12:20.253769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:51:25.772870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253875+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:12:20.253875+0000","last_peered":"2023-07-13T22:12:20.253875+0000","last_clean":"2023-07-13T22:12:20.253875+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:12:20.253875+0000","last_undegraded":"2023-07-13T22:12:20.253875+0000","last_fullsized":"2023-07-13T22:12:20.253875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:01:13.663502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654966+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:12:20.654966+0000","last_peered":"2023-07-13T22:12:20.654966+0000","last_clean":"2023-07-13T22:12:20.654966+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:12:20.654966+0000","last_undegraded":"2023-07-13T22:12:20.654966+0000","last_fullsized":"2023-07-13T22:12:20.654966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:01.763241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658843+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:12:20.658843+0000","last_peered":"2023-07-13T22:12:20.658843+0000","last_clean":"2023-07-13T22:12:20.658843+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:12:20.658843+0000","last_undegraded":"2023-07-13T22:12:20.658843+0000","last_fullsized":"2023-07-13T22:12:20.658843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:05.836286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.523903+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:12:20.523903+0000","last_peered":"2023-07-13T22:12:20.523903+0000","last_clean":"2023-07-13T22:12:20.523903+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:12:20.523903+0000","last_undegraded":"2023-07-13T22:12:20.523903+0000","last_fullsized":"2023-07-13T22:12:20.523903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.864029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964438+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:12:19.964438+0000","last_peered":"2023-07-13T22:12:19.964438+0000","last_clean":"2023-07-13T22:12:19.964438+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:12:19.964438+0000","last_undegraded":"2023-07-13T22:12:19.964438+0000","last_fullsized":"2023-07-13T22:12:19.964438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:06.441040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.963974+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:12:19.963974+0000","last_peered":"2023-07-13T22:12:19.963974+0000","last_clean":"2023-07-13T22:12:19.963974+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:12:19.963974+0000","last_undegraded":"2023-07-13T22:12:19.963974+0000","last_fullsized":"2023-07-13T22:12:19.963974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:55.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654609+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:12:20.654609+0000","last_peered":"2023-07-13T22:12:20.654609+0000","last_clean":"2023-07-13T22:12:20.654609+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:12:20.654609+0000","last_undegraded":"2023-07-13T22:12:20.654609+0000","last_fullsized":"2023-07-13T22:12:20.654609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:52.248774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.524045+0000","last_change":"2023-07-13T22:12:18.970433+0000","last_active":"2023-07-13T22:12:20.524045+0000","last_peered":"2023-07-13T22:12:20.524045+0000","last_clean":"2023-07-13T22:12:20.524045+0000","last_became_active":"2023-07-13T22:12:18.969903+0000","last_became_peered":"2023-07-13T22:12:18.969903+0000","last_unstale":"2023-07-13T22:12:20.524045+0000","last_undegraded":"2023-07-13T22:12:20.524045+0000","last_fullsized":"2023-07-13T22:12:20.524045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:42:08.002303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.965021+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:12:19.965021+0000","last_peered":"2023-07-13T22:12:19.965021+0000","last_clean":"2023-07-13T22:12:19.965021+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:12:19.965021+0000","last_undegraded":"2023-07-13T22:12:19.965021+0000","last_fullsized":"2023-07-13T22:12:19.965021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:13.816929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654946+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:12:20.654946+0000","last_peered":"2023-07-13T22:12:20.654946+0000","last_clean":"2023-07-13T22:12:20.654946+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:12:20.654946+0000","last_undegraded":"2023-07-13T22:12:20.654946+0000","last_fullsized":"2023-07-13T22:12:20.654946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:57:24.559497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654494+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:12:20.654494+0000","last_peered":"2023-07-13T22:12:20.654494+0000","last_clean":"2023-07-13T22:12:20.654494+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:12:20.654494+0000","last_undegraded":"2023-07-13T22:12:20.654494+0000","last_fullsized":"2023-07-13T22:12:20.654494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:41.432615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253809+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:12:20.253809+0000","last_peered":"2023-07-13T22:12:20.253809+0000","last_clean":"2023-07-13T22:12:20.253809+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:12:20.253809+0000","last_undegraded":"2023-07-13T22:12:20.253809+0000","last_fullsized":"2023-07-13T22:12:20.253809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:53.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253734+0000","last_change":"2023-07-13T22:12:18.972104+0000","last_active":"2023-07-13T22:12:20.253734+0000","last_peered":"2023-07-13T22:12:20.253734+0000","last_clean":"2023-07-13T22:12:20.253734+0000","last_became_active":"2023-07-13T22:12:18.971822+0000","last_became_peered":"2023-07-13T22:12:18.971822+0000","last_unstale":"2023-07-13T22:12:20.253734+0000","last_undegraded":"2023-07-13T22:12:20.253734+0000","last_fullsized":"2023-07-13T22:12:20.253734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:53.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653964+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:12:20.653964+0000","last_peered":"2023-07-13T22:12:20.653964+0000","last_clean":"2023-07-13T22:12:20.653964+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:12:20.653964+0000","last_undegraded":"2023-07-13T22:12:20.653964+0000","last_fullsized":"2023-07-13T22:12:20.653964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:02.543666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.970124+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:12:19.970124+0000","last_peered":"2023-07-13T22:12:19.970124+0000","last_clean":"2023-07-13T22:12:19.970124+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:12:19.970124+0000","last_undegraded":"2023-07-13T22:12:19.970124+0000","last_fullsized":"2023-07-13T22:12:19.970124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:50:30.192327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246333+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:12:20.246333+0000","last_peered":"2023-07-13T22:12:20.246333+0000","last_clean":"2023-07-13T22:12:20.246333+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:12:20.246333+0000","last_undegraded":"2023-07-13T22:12:20.246333+0000","last_fullsized":"2023-07-13T22:12:20.246333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:46.939883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654295+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:12:20.654295+0000","last_peered":"2023-07-13T22:12:20.654295+0000","last_clean":"2023-07-13T22:12:20.654295+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:12:20.654295+0000","last_undegraded":"2023-07-13T22:12:20.654295+0000","last_fullsized":"2023-07-13T22:12:20.654295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:02.671624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'11","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446482+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:12:24.446482+0000","last_peered":"2023-07-13T22:12:24.446482+0000","last_clean":"2023-07-13T22:12:24.446482+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:12:24.446482+0000","last_undegraded":"2023-07-13T22:12:24.446482+0000","last_fullsized":"2023-07-13T22:12:24.446482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:36:55.215409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964681+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:12:19.964681+0000","last_peered":"2023-07-13T22:12:19.964681+0000","last_clean":"2023-07-13T22:12:19.964681+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:12:19.964681+0000","last_undegraded":"2023-07-13T22:12:19.964681+0000","last_fullsized":"2023-07-13T22:12:19.964681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:21:34.769717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.213989+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:20.213989+0000","last_peered":"2023-07-13T22:12:20.213989+0000","last_clean":"2023-07-13T22:12:20.213989+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:20.213989+0000","last_undegraded":"2023-07-13T22:12:20.213989+0000","last_fullsized":"2023-07-13T22:12:20.213989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:27.315219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253952+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:12:20.253952+0000","last_peered":"2023-07-13T22:12:20.253952+0000","last_clean":"2023-07-13T22:12:20.253952+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:12:20.253952+0000","last_undegraded":"2023-07-13T22:12:20.253952+0000","last_fullsized":"2023-07-13T22:12:20.253952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:20.411763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.444701+0000","last_change":"2023-07-13T22:12:17.986791+0000","last_active":"2023-07-13T22:12:24.444701+0000","last_peered":"2023-07-13T22:12:24.444701+0000","last_clean":"2023-07-13T22:12:24.444701+0000","last_became_active":"2023-07-13T22:12:17.986273+0000","last_became_peered":"2023-07-13T22:12:17.986273+0000","last_unstale":"2023-07-13T22:12:24.444701+0000","last_undegraded":"2023-07-13T22:12:24.444701+0000","last_fullsized":"2023-07-13T22:12:24.444701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:15.486007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654149+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:12:20.654149+0000","last_peered":"2023-07-13T22:12:20.654149+0000","last_clean":"2023-07-13T22:12:20.654149+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:12:20.654149+0000","last_undegraded":"2023-07-13T22:12:20.654149+0000","last_fullsized":"2023-07-13T22:12:20.654149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:15.958627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659357+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:20.659357+0000","last_peered":"2023-07-13T22:12:20.659357+0000","last_clean":"2023-07-13T22:12:20.659357+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:20.659357+0000","last_undegraded":"2023-07-13T22:12:20.659357+0000","last_fullsized":"2023-07-13T22:12:20.659357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:03:08.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659110+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:12:20.659110+0000","last_peered":"2023-07-13T22:12:20.659110+0000","last_clean":"2023-07-13T22:12:20.659110+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:12:20.659110+0000","last_undegraded":"2023-07-13T22:12:20.659110+0000","last_fullsized":"2023-07-13T22:12:20.659110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:10:02.649510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.439199+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:12:25.439199+0000","last_peered":"2023-07-13T22:12:25.439199+0000","last_clean":"2023-07-13T22:12:25.439199+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:12:25.439199+0000","last_undegraded":"2023-07-13T22:12:25.439199+0000","last_fullsized":"2023-07-13T22:12:25.439199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:54:45.909242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658675+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:12:20.658675+0000","last_peered":"2023-07-13T22:12:20.658675+0000","last_clean":"2023-07-13T22:12:20.658675+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:12:20.658675+0000","last_undegraded":"2023-07-13T22:12:20.658675+0000","last_fullsized":"2023-07-13T22:12:20.658675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:27.563406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446736+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:24.446736+0000","last_peered":"2023-07-13T22:12:24.446736+0000","last_clean":"2023-07-13T22:12:24.446736+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:24.446736+0000","last_undegraded":"2023-07-13T22:12:24.446736+0000","last_fullsized":"2023-07-13T22:12:24.446736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:09:21.483570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659361+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:12:20.659361+0000","last_peered":"2023-07-13T22:12:20.659361+0000","last_clean":"2023-07-13T22:12:20.659361+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:12:20.659361+0000","last_undegraded":"2023-07-13T22:12:20.659361+0000","last_fullsized":"2023-07-13T22:12:20.659361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:32.512828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:23.996982+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:23.996982+0000","last_peered":"2023-07-13T22:12:23.996982+0000","last_clean":"2023-07-13T22:12:23.996982+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:23.996982+0000","last_undegraded":"2023-07-13T22:12:23.996982+0000","last_fullsized":"2023-07-13T22:12:23.996982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:31.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964105+0000","last_change":"2023-07-13T22:12:18.980968+0000","last_active":"2023-07-13T22:12:19.964105+0000","last_peered":"2023-07-13T22:12:19.964105+0000","last_clean":"2023-07-13T22:12:19.964105+0000","last_became_active":"2023-07-13T22:12:18.980166+0000","last_became_peered":"2023-07-13T22:12:18.980166+0000","last_unstale":"2023-07-13T22:12:19.964105+0000","last_undegraded":"2023-07-13T22:12:19.964105+0000","last_fullsized":"2023-07-13T22:12:19.964105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:25:06.678041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.451173+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:25.451173+0000","last_peered":"2023-07-13T22:12:25.451173+0000","last_clean":"2023-07-13T22:12:25.451173+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:25.451173+0000","last_undegraded":"2023-07-13T22:12:25.451173+0000","last_fullsized":"2023-07-13T22:12:25.451173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:00:50.214242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246083+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:12:20.246083+0000","last_peered":"2023-07-13T22:12:20.246083+0000","last_clean":"2023-07-13T22:12:20.246083+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:12:20.246083+0000","last_undegraded":"2023-07-13T22:12:20.246083+0000","last_fullsized":"2023-07-13T22:12:20.246083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:58:35.345174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.440055+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:12:26.440055+0000","last_peered":"2023-07-13T22:12:26.440055+0000","last_clean":"2023-07-13T22:12:26.440055+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:12:26.440055+0000","last_undegraded":"2023-07-13T22:12:26.440055+0000","last_fullsized":"2023-07-13T22:12:26.440055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:35.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654434+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:12:20.654434+0000","last_peered":"2023-07-13T22:12:20.654434+0000","last_clean":"2023-07-13T22:12:20.654434+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:12:20.654434+0000","last_undegraded":"2023-07-13T22:12:20.654434+0000","last_fullsized":"2023-07-13T22:12:20.654434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:29:22.530212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.986518+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:22.986518+0000","last_peered":"2023-07-13T22:12:22.986518+0000","last_clean":"2023-07-13T22:12:22.986518+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:22.986518+0000","last_undegraded":"2023-07-13T22:12:22.986518+0000","last_fullsized":"2023-07-13T22:12:22.986518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:33.555639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253873+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:12:20.253873+0000","last_peered":"2023-07-13T22:12:20.253873+0000","last_clean":"2023-07-13T22:12:20.253873+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:12:20.253873+0000","last_undegraded":"2023-07-13T22:12:20.253873+0000","last_fullsized":"2023-07-13T22:12:20.253873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:52:47.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.437022+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:24.437022+0000","last_peered":"2023-07-13T22:12:24.437022+0000","last_clean":"2023-07-13T22:12:24.437022+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:24.437022+0000","last_undegraded":"2023-07-13T22:12:24.437022+0000","last_fullsized":"2023-07-13T22:12:24.437022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:09:00.597660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654261+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:12:20.654261+0000","last_peered":"2023-07-13T22:12:20.654261+0000","last_clean":"2023-07-13T22:12:20.654261+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:12:20.654261+0000","last_undegraded":"2023-07-13T22:12:20.654261+0000","last_fullsized":"2023-07-13T22:12:20.654261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:50:54.792816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.441876+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:12:26.441876+0000","last_peered":"2023-07-13T22:12:26.441876+0000","last_clean":"2023-07-13T22:12:26.441876+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:12:26.441876+0000","last_undegraded":"2023-07-13T22:12:26.441876+0000","last_fullsized":"2023-07-13T22:12:26.441876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:29:00.307908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":19,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659474+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:12:20.659474+0000","last_peered":"2023-07-13T22:12:20.659474+0000","last_clean":"2023-07-13T22:12:20.659474+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:12:20.659474+0000","last_undegraded":"2023-07-13T22:12:20.659474+0000","last_fullsized":"2023-07-13T22:12:20.659474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:03:30.825329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.434019+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:24.434019+0000","last_peered":"2023-07-13T22:12:24.434019+0000","last_clean":"2023-07-13T22:12:24.434019+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:24.434019+0000","last_undegraded":"2023-07-13T22:12:24.434019+0000","last_fullsized":"2023-07-13T22:12:24.434019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:17:11.346880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654290+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:12:20.654290+0000","last_peered":"2023-07-13T22:12:20.654290+0000","last_clean":"2023-07-13T22:12:20.654290+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:12:20.654290+0000","last_undegraded":"2023-07-13T22:12:20.654290+0000","last_fullsized":"2023-07-13T22:12:20.654290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:40:19.190355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.451269+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:25.451269+0000","last_peered":"2023-07-13T22:12:25.451269+0000","last_clean":"2023-07-13T22:12:25.451269+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:25.451269+0000","last_undegraded":"2023-07-13T22:12:25.451269+0000","last_fullsized":"2023-07-13T22:12:25.451269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:11:03.850573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964835+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:12:19.964835+0000","last_peered":"2023-07-13T22:12:19.964835+0000","last_clean":"2023-07-13T22:12:19.964835+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:12:19.964835+0000","last_undegraded":"2023-07-13T22:12:19.964835+0000","last_fullsized":"2023-07-13T22:12:19.964835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:10.032807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654853+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:12:20.654853+0000","last_peered":"2023-07-13T22:12:20.654853+0000","last_clean":"2023-07-13T22:12:20.654853+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:12:20.654853+0000","last_undegraded":"2023-07-13T22:12:20.654853+0000","last_fullsized":"2023-07-13T22:12:20.654853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:04.877520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964385+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:12:19.964385+0000","last_peered":"2023-07-13T22:12:19.964385+0000","last_clean":"2023-07-13T22:12:19.964385+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:12:19.964385+0000","last_undegraded":"2023-07-13T22:12:19.964385+0000","last_fullsized":"2023-07-13T22:12:19.964385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:43:04.765916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'16","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.434055+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:24.434055+0000","last_peered":"2023-07-13T22:12:24.434055+0000","last_clean":"2023-07-13T22:12:24.434055+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:24.434055+0000","last_undegraded":"2023-07-13T22:12:24.434055+0000","last_fullsized":"2023-07-13T22:12:24.434055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:49:36.100083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658871+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:12:20.658871+0000","last_peered":"2023-07-13T22:12:20.658871+0000","last_clean":"2023-07-13T22:12:20.658871+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:12:20.658871+0000","last_undegraded":"2023-07-13T22:12:20.658871+0000","last_fullsized":"2023-07-13T22:12:20.658871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:14.031661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659630+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:20.659630+0000","last_peered":"2023-07-13T22:12:20.659630+0000","last_clean":"2023-07-13T22:12:20.659630+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:20.659630+0000","last_undegraded":"2023-07-13T22:12:20.659630+0000","last_fullsized":"2023-07-13T22:12:20.659630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:36:32.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969908+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:12:19.969908+0000","last_peered":"2023-07-13T22:12:19.969908+0000","last_clean":"2023-07-13T22:12:19.969908+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:12:19.969908+0000","last_undegraded":"2023-07-13T22:12:19.969908+0000","last_fullsized":"2023-07-13T22:12:19.969908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:46:21.674730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246220+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:20.246220+0000","last_peered":"2023-07-13T22:12:20.246220+0000","last_clean":"2023-07-13T22:12:20.246220+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:20.246220+0000","last_undegraded":"2023-07-13T22:12:20.246220+0000","last_fullsized":"2023-07-13T22:12:20.246220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:52.330175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:21.739637+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:12:21.739637+0000","last_peered":"2023-07-13T22:12:21.739637+0000","last_clean":"2023-07-13T22:12:21.739637+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:12:21.739637+0000","last_undegraded":"2023-07-13T22:12:21.739637+0000","last_fullsized":"2023-07-13T22:12:21.739637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:17:58.279474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964543+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:12:19.964543+0000","last_peered":"2023-07-13T22:12:19.964543+0000","last_clean":"2023-07-13T22:12:19.964543+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:12:19.964543+0000","last_undegraded":"2023-07-13T22:12:19.964543+0000","last_fullsized":"2023-07-13T22:12:19.964543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:53:50.719647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.523699+0000","last_change":"2023-07-13T22:12:18.983052+0000","last_active":"2023-07-13T22:12:20.523699+0000","last_peered":"2023-07-13T22:12:20.523699+0000","last_clean":"2023-07-13T22:12:20.523699+0000","last_became_active":"2023-07-13T22:12:18.982747+0000","last_became_peered":"2023-07-13T22:12:18.982747+0000","last_unstale":"2023-07-13T22:12:20.523699+0000","last_undegraded":"2023-07-13T22:12:20.523699+0000","last_fullsized":"2023-07-13T22:12:20.523699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:17:25.757531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659033+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:12:20.659033+0000","last_peered":"2023-07-13T22:12:20.659033+0000","last_clean":"2023-07-13T22:12:20.659033+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:12:20.659033+0000","last_undegraded":"2023-07-13T22:12:20.659033+0000","last_fullsized":"2023-07-13T22:12:20.659033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:33.801533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.441639+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:12:26.441639+0000","last_peered":"2023-07-13T22:12:26.441639+0000","last_clean":"2023-07-13T22:12:26.441639+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:12:26.441639+0000","last_undegraded":"2023-07-13T22:12:26.441639+0000","last_fullsized":"2023-07-13T22:12:26.441639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:33.552236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964542+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:12:19.964542+0000","last_peered":"2023-07-13T22:12:19.964542+0000","last_clean":"2023-07-13T22:12:19.964542+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:12:19.964542+0000","last_undegraded":"2023-07-13T22:12:19.964542+0000","last_fullsized":"2023-07-13T22:12:19.964542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:35:31.095800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.655483+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:20.655483+0000","last_peered":"2023-07-13T22:12:20.655483+0000","last_clean":"2023-07-13T22:12:20.655483+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:20.655483+0000","last_undegraded":"2023-07-13T22:12:20.655483+0000","last_fullsized":"2023-07-13T22:12:20.655483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:28:07.554927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964496+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:19.964496+0000","last_peered":"2023-07-13T22:12:19.964496+0000","last_clean":"2023-07-13T22:12:19.964496+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:19.964496+0000","last_undegraded":"2023-07-13T22:12:19.964496+0000","last_fullsized":"2023-07-13T22:12:19.964496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:03.934747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654179+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:12:20.654179+0000","last_peered":"2023-07-13T22:12:20.654179+0000","last_clean":"2023-07-13T22:12:20.654179+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:12:20.654179+0000","last_undegraded":"2023-07-13T22:12:20.654179+0000","last_fullsized":"2023-07-13T22:12:20.654179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:55.597112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964751+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:19.964751+0000","last_peered":"2023-07-13T22:12:19.964751+0000","last_clean":"2023-07-13T22:12:19.964751+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:19.964751+0000","last_undegraded":"2023-07-13T22:12:19.964751+0000","last_fullsized":"2023-07-13T22:12:19.964751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:22.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.445522+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:24.445522+0000","last_peered":"2023-07-13T22:12:24.445522+0000","last_clean":"2023-07-13T22:12:24.445522+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:24.445522+0000","last_undegraded":"2023-07-13T22:12:24.445522+0000","last_fullsized":"2023-07-13T22:12:24.445522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:44:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654358+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:12:20.654358+0000","last_peered":"2023-07-13T22:12:20.654358+0000","last_clean":"2023-07-13T22:12:20.654358+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:12:20.654358+0000","last_undegraded":"2023-07-13T22:12:20.654358+0000","last_fullsized":"2023-07-13T22:12:20.654358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:26.007193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446769+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:24.446769+0000","last_peered":"2023-07-13T22:12:24.446769+0000","last_clean":"2023-07-13T22:12:24.446769+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:24.446769+0000","last_undegraded":"2023-07-13T22:12:24.446769+0000","last_fullsized":"2023-07-13T22:12:24.446769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:11.802625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964328+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:12:19.964328+0000","last_peered":"2023-07-13T22:12:19.964328+0000","last_clean":"2023-07-13T22:12:19.964328+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:12:19.964328+0000","last_undegraded":"2023-07-13T22:12:19.964328+0000","last_fullsized":"2023-07-13T22:12:19.964328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:47.508251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654872+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:20.654872+0000","last_peered":"2023-07-13T22:12:20.654872+0000","last_clean":"2023-07-13T22:12:20.654872+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:20.654872+0000","last_undegraded":"2023-07-13T22:12:20.654872+0000","last_fullsized":"2023-07-13T22:12:20.654872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:23:08.462604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253938+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:12:20.253938+0000","last_peered":"2023-07-13T22:12:20.253938+0000","last_clean":"2023-07-13T22:12:20.253938+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:12:20.253938+0000","last_undegraded":"2023-07-13T22:12:20.253938+0000","last_fullsized":"2023-07-13T22:12:20.253938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:40.442692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.988369+0000","last_change":"2023-07-13T22:12:17.987479+0000","last_active":"2023-07-13T22:12:22.988369+0000","last_peered":"2023-07-13T22:12:22.988369+0000","last_clean":"2023-07-13T22:12:22.988369+0000","last_became_active":"2023-07-13T22:12:17.987141+0000","last_became_peered":"2023-07-13T22:12:17.987141+0000","last_unstale":"2023-07-13T22:12:22.988369+0000","last_undegraded":"2023-07-13T22:12:22.988369+0000","last_fullsized":"2023-07-13T22:12:22.988369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:31.713242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659192+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:12:20.659192+0000","last_peered":"2023-07-13T22:12:20.659192+0000","last_clean":"2023-07-13T22:12:20.659192+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:12:20.659192+0000","last_undegraded":"2023-07-13T22:12:20.659192+0000","last_fullsized":"2023-07-13T22:12:20.659192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:25.212659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446501+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:24.446501+0000","last_peered":"2023-07-13T22:12:24.446501+0000","last_clean":"2023-07-13T22:12:24.446501+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:24.446501+0000","last_undegraded":"2023-07-13T22:12:24.446501+0000","last_fullsized":"2023-07-13T22:12:24.446501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:50:52.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969621+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:12:19.969621+0000","last_peered":"2023-07-13T22:12:19.969621+0000","last_clean":"2023-07-13T22:12:19.969621+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:12:19.969621+0000","last_undegraded":"2023-07-13T22:12:19.969621+0000","last_fullsized":"2023-07-13T22:12:19.969621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:16.827543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.085032+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:24.085032+0000","last_peered":"2023-07-13T22:12:24.085032+0000","last_clean":"2023-07-13T22:12:24.085032+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:24.085032+0000","last_undegraded":"2023-07-13T22:12:24.085032+0000","last_fullsized":"2023-07-13T22:12:24.085032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:25.632958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964616+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:12:19.964616+0000","last_peered":"2023-07-13T22:12:19.964616+0000","last_clean":"2023-07-13T22:12:19.964616+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:12:19.964616+0000","last_undegraded":"2023-07-13T22:12:19.964616+0000","last_fullsized":"2023-07-13T22:12:19.964616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:00.526274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.987233+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:22.987233+0000","last_peered":"2023-07-13T22:12:22.987233+0000","last_clean":"2023-07-13T22:12:22.987233+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:22.987233+0000","last_undegraded":"2023-07-13T22:12:22.987233+0000","last_fullsized":"2023-07-13T22:12:22.987233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:29:44.146326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964781+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:12:19.964781+0000","last_peered":"2023-07-13T22:12:19.964781+0000","last_clean":"2023-07-13T22:12:19.964781+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:12:19.964781+0000","last_undegraded":"2023-07-13T22:12:19.964781+0000","last_fullsized":"2023-07-13T22:12:19.964781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:59.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.444274+0000","last_change":"2023-07-13T22:12:17.991052+0000","last_active":"2023-07-13T22:12:24.444274+0000","last_peered":"2023-07-13T22:12:24.444274+0000","last_clean":"2023-07-13T22:12:24.444274+0000","last_became_active":"2023-07-13T22:12:17.990314+0000","last_became_peered":"2023-07-13T22:12:17.990314+0000","last_unstale":"2023-07-13T22:12:24.444274+0000","last_undegraded":"2023-07-13T22:12:24.444274+0000","last_fullsized":"2023-07-13T22:12:24.444274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:01:36.285647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964898+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:12:19.964898+0000","last_peered":"2023-07-13T22:12:19.964898+0000","last_clean":"2023-07-13T22:12:19.964898+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:12:19.964898+0000","last_undegraded":"2023-07-13T22:12:19.964898+0000","last_fullsized":"2023-07-13T22:12:19.964898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.914208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.445291+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:24.445291+0000","last_peered":"2023-07-13T22:12:24.445291+0000","last_clean":"2023-07-13T22:12:24.445291+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:24.445291+0000","last_undegraded":"2023-07-13T22:12:24.445291+0000","last_fullsized":"2023-07-13T22:12:24.445291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:39:03.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964537+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:12:19.964537+0000","last_peered":"2023-07-13T22:12:19.964537+0000","last_clean":"2023-07-13T22:12:19.964537+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:12:19.964537+0000","last_undegraded":"2023-07-13T22:12:19.964537+0000","last_fullsized":"2023-07-13T22:12:19.964537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:14.493969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.454731+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:25.454731+0000","last_peered":"2023-07-13T22:12:25.454731+0000","last_clean":"2023-07-13T22:12:25.454731+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:25.454731+0000","last_undegraded":"2023-07-13T22:12:25.454731+0000","last_fullsized":"2023-07-13T22:12:25.454731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:28:57.362060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964984+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:12:19.964984+0000","last_peered":"2023-07-13T22:12:19.964984+0000","last_clean":"2023-07-13T22:12:19.964984+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:12:19.964984+0000","last_undegraded":"2023-07-13T22:12:19.964984+0000","last_fullsized":"2023-07-13T22:12:19.964984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:04:08.340928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969615+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:12:19.969615+0000","last_peered":"2023-07-13T22:12:19.969615+0000","last_clean":"2023-07-13T22:12:19.969615+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:12:19.969615+0000","last_undegraded":"2023-07-13T22:12:19.969615+0000","last_fullsized":"2023-07-13T22:12:19.969615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:54.278671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653922+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:12:20.653922+0000","last_peered":"2023-07-13T22:12:20.653922+0000","last_clean":"2023-07-13T22:12:20.653922+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:12:20.653922+0000","last_undegraded":"2023-07-13T22:12:20.653922+0000","last_fullsized":"2023-07-13T22:12:20.653922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:20.741383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653937+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:12:20.653937+0000","last_peered":"2023-07-13T22:12:20.653937+0000","last_clean":"2023-07-13T22:12:20.653937+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:12:20.653937+0000","last_undegraded":"2023-07-13T22:12:20.653937+0000","last_fullsized":"2023-07-13T22:12:20.653937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:22:51.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253940+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:12:20.253940+0000","last_peered":"2023-07-13T22:12:20.253940+0000","last_clean":"2023-07-13T22:12:20.253940+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:12:20.253940+0000","last_undegraded":"2023-07-13T22:12:20.253940+0000","last_fullsized":"2023-07-13T22:12:20.253940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:36:20.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654502+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:12:20.654502+0000","last_peered":"2023-07-13T22:12:20.654502+0000","last_clean":"2023-07-13T22:12:20.654502+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:12:20.654502+0000","last_undegraded":"2023-07-13T22:12:20.654502+0000","last_fullsized":"2023-07-13T22:12:20.654502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:36.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969840+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:12:19.969840+0000","last_peered":"2023-07-13T22:12:19.969840+0000","last_clean":"2023-07-13T22:12:19.969840+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:12:19.969840+0000","last_undegraded":"2023-07-13T22:12:19.969840+0000","last_fullsized":"2023-07-13T22:12:19.969840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:30:48.331390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654409+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:12:20.654409+0000","last_peered":"2023-07-13T22:12:20.654409+0000","last_clean":"2023-07-13T22:12:20.654409+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:12:20.654409+0000","last_undegraded":"2023-07-13T22:12:20.654409+0000","last_fullsized":"2023-07-13T22:12:20.654409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:05:06.409809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964319+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:12:19.964319+0000","last_peered":"2023-07-13T22:12:19.964319+0000","last_clean":"2023-07-13T22:12:19.964319+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:12:19.964319+0000","last_undegraded":"2023-07-13T22:12:19.964319+0000","last_fullsized":"2023-07-13T22:12:19.964319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:39.162208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658654+0000","last_change":"2023-07-13T22:12:18.980792+0000","last_active":"2023-07-13T22:12:20.658654+0000","last_peered":"2023-07-13T22:12:20.658654+0000","last_clean":"2023-07-13T22:12:20.658654+0000","last_became_active":"2023-07-13T22:12:18.980488+0000","last_became_peered":"2023-07-13T22:12:18.980488+0000","last_unstale":"2023-07-13T22:12:20.658654+0000","last_undegraded":"2023-07-13T22:12:20.658654+0000","last_fullsized":"2023-07-13T22:12:20.658654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:54.718460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964433+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:12:19.964433+0000","last_peered":"2023-07-13T22:12:19.964433+0000","last_clean":"2023-07-13T22:12:19.964433+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:12:19.964433+0000","last_undegraded":"2023-07-13T22:12:19.964433+0000","last_fullsized":"2023-07-13T22:12:19.964433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:38:40.316111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654694+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:12:20.654694+0000","last_peered":"2023-07-13T22:12:20.654694+0000","last_clean":"2023-07-13T22:12:20.654694+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:12:20.654694+0000","last_undegraded":"2023-07-13T22:12:20.654694+0000","last_fullsized":"2023-07-13T22:12:20.654694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:05:25.381241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654510+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:12:20.654510+0000","last_peered":"2023-07-13T22:12:20.654510+0000","last_clean":"2023-07-13T22:12:20.654510+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:12:20.654510+0000","last_undegraded":"2023-07-13T22:12:20.654510+0000","last_fullsized":"2023-07-13T22:12:20.654510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:09:11.970190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969676+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:12:19.969676+0000","last_peered":"2023-07-13T22:12:19.969676+0000","last_clean":"2023-07-13T22:12:19.969676+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:12:19.969676+0000","last_undegraded":"2023-07-13T22:12:19.969676+0000","last_fullsized":"2023-07-13T22:12:19.969676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:51:17.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27840,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":35,"num_read_kb":45,"num_write":161,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":37002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":10,"up_from":71,"seq":304942678034,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1560576,"data_stored":1036926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.3220000000000001,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.2800000000000002,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.234,"5min":5.234,"15min":5.234},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.753}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.746},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.5},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.84699999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.876},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.85899999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.95799999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103194,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.745}]},{"osd":2,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.876},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77300000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1016768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.57999999999999996,"5min":1.48,"15min":1.48},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82799999999999996,"5min":0.871,"15min":0.871},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92800000000000005,"5min":1.056,"15min":1.056},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94099999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.86499999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.83399999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495650,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91400000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.751},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.014,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.45000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.085,"15min":1.085},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.503,"15min":0.503},"max":{"1min":1.0149999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.01,"5min":1.726,"15min":1.726},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1020000000000001,"5min":1.369,"15min":1.369},"last":0.879},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.085,"5min":1.573,"15min":1.573},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.83499999999999996},"min":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0780000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.78000000000000003},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70299999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.84599999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":427382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.36299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.499,"15min":0.46400000000000002},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.55300000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.31,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80200000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.90600000000000003}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.98599999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.042},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":1.026}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.014}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.318,"15min":0.318},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85399999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.64300000000000002,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.65900000000000003,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jul 13 22:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.52400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86699999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72499999999999998,"15min":0.69999999999999996},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.878}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.443}]},{"osd":9,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.59899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":428015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629}]},{"osd":1,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.875},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78200000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.2469999999999999,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0209999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.054}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":425895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.41399999999999998},"min":{"1min":0.33600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63700000000000001,"5min":0.751,"15min":0.751},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.35399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.63500000000000001,"15min":0.59699999999999998},"min":{"1min":0.57299999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61899999999999999,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.65100000000000002,"15min":0.621},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.67700000000000005,"15min":0.63400000000000001},"min":{"1min":0.51500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.70499999999999996,"15min":0.66400000000000003},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.006},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65000000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.85899999999999999,"15min":0.84899999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Jul 13 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83399999999999996,"5min":1.081,"15min":1.081},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.65900000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97499999999999998,"5min":1.095,"15min":1.095},"last":0.443},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Jul 13 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.6459999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.754}]},{"osd":10,"last update":"Thu Jul 13 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1527808,"data_stored":1018958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.7170000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":3.2360000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":3.0009999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81799999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.665}]},{"osd":3,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":2.9590000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.764},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.9159999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":2.032}]},{"osd":7,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":2.875},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.625,"15min":0.628},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":3.9929999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":2.8279999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88100000000000001}]},{"osd":9,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":2.0880000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":3.1139999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":3.0470000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:12:28.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:12:28.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:28 smithi117 ceph-mon[129074]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:28 smithi117 ceph-mon[129074]: pgmap v271: 97 pgs: 97 active+clean; 604 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2023-07-13T22:12:28.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:28 smithi117 ceph-mon[129074]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:12:28.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:28 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2731407671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:12:28.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:28 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/996396' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:29.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 -- 172.21.15.26:0/751553107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc924101680 msgr2=0x7fc924107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:29.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 --2- 172.21.15.26:0/751553107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc924101680 0x7fc924107f40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009f90 tx=0x7fc91c00b330 comp rx=0 tx=0).stop 2023-07-13T22:12:29.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 -- 172.21.15.26:0/751553107 shutdown_connections 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 --2- 172.21.15.26:0/751553107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc924101680 0x7fc924107f40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 --2- 172.21.15.26:0/751553107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9240ffb00 0x7fc9240fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 --2- 172.21.15.26:0/751553107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924100cf0 0x7fc924101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.147+0000 7fc928dff700 1 -- 172.21.15.26:0/751553107 >> 172.21.15.26:0/751553107 conn(0x7fc9240fb0e0 msgr2=0x7fc9240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.148+0000 7fc928dff700 1 -- 172.21.15.26:0/751553107 shutdown_connections 2023-07-13T22:12:29.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.148+0000 7fc928dff700 1 -- 172.21.15.26:0/751553107 wait complete. 2023-07-13T22:12:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 Processor -- start 2023-07-13T22:12:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 -- start start 2023-07-13T22:12:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 0x7fc924198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924101680 0x7fc92419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419f6c0 con 0x7fc9240ffb00 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419f800 con 0x7fc924101680 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc928dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419f990 con 0x7fc924100cf0 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc92259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc92259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45988/0 (socket says 172.21.15.26:45988) 2023-07-13T22:12:29.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc92259c700 1 -- 172.21.15.26:0/3781644369 learned_addr learned my addr 172.21.15.26:0/3781644369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.149+0000 7fc921d9b700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 0x7fc924198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc922d9d700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924101680 0x7fc92419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 -- 172.21.15.26:0/3781644369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 msgr2=0x7fc924198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 0x7fc924198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 -- 172.21.15.26:0/3781644369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924101680 msgr2=0x7fc92419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924101680 0x7fc92419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 -- 172.21.15.26:0/3781644369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c009c40 con 0x7fc9240ffb00 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc921d9b700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 0x7fc924198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc92259c700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914004220 con 0x7fc9240ffb00 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914004380 con 0x7fc9240ffb00 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914003c00 con 0x7fc9240ffb00 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92419fc70 con 0x7fc9240ffb00 2023-07-13T22:12:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.150+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9241a0130 con 0x7fc9240ffb00 2023-07-13T22:12:29.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.151+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92410b330 con 0x7fc9240ffb00 2023-07-13T22:12:29.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.153+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc91400c7c0 con 0x7fc9240ffb00 2023-07-13T22:12:29.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.155+0000 7fc9137fe700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 0x7fc90c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:29.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.155+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc914014070 con 0x7fc9240ffb00 2023-07-13T22:12:29.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.155+0000 7fc921d9b700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 0x7fc90c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:29.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.156+0000 7fc921d9b700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 0x7fc90c0a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc918011490 tx=0x7fc918009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:29.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.158+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc914096410 con 0x7fc9240ffb00 2023-07-13T22:12:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.361+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc924107bf0 con 0x7fc90c0a1220 2023-07-13T22:12:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.372+0000 7fc9137fe700 1 -- 172.21.15.26:0/3781644369 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335554 (secure 0 0 0) 0x7fc924107bf0 con 0x7fc90c0a1220 2023-07-13T22:12:29.367 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 msgr2=0x7fc90c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 0x7fc90c0a36d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc918011490 tx=0x7fc918009260 comp rx=0 tx=0).stop 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 msgr2=0x7fc924197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc91400e9f0 tx=0x7fc91400ed00 comp rx=0 tx=0).stop 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 shutdown_connections 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc90c0a1220 0x7fc90c0a36d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9240ffb00 0x7fc924197b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc924101680 0x7fc92419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 --2- 172.21.15.26:0/3781644369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc924100cf0 0x7fc924198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.376+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 >> 172.21.15.26:0/3781644369 conn(0x7fc9240fb0e0 msgr2=0x7fc9241064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.377+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 shutdown_connections 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:29.377+0000 7fc928dff700 1 -- 172.21.15.26:0/3781644369 wait complete. 2023-07-13T22:12:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:12:29.387 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2023-07-13T22:12:27.419401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":618331,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":161,"num_read_kb":154,"num_write":343,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335832,"kb_used_data":12408,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605992,"statfs":{"total":1151940427776,"available":1151596535808,"internally_reserved":0,"allocated":12705792,"data_stored":6911283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27963,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":35,"num_read_kb":45,"num_write":162,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.464160"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.970024+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:12:19.970024+0000","last_peered":"2023-07-13T22:12:19.970024+0000","last_clean":"2023-07-13T22:12:19.970024+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:12:19.970024+0000","last_undegraded":"2023-07-13T22:12:19.970024+0000","last_fullsized":"2023-07-13T22:12:19.970024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:52:58.393428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253769+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:12:20.253769+0000","last_peered":"2023-07-13T22:12:20.253769+0000","last_clean":"2023-07-13T22:12:20.253769+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:12:20.253769+0000","last_undegraded":"2023-07-13T22:12:20.253769+0000","last_fullsized":"2023-07-13T22:12:20.253769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:51:25.772870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253875+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:12:20.253875+0000","last_peered":"2023-07-13T22:12:20.253875+0000","last_clean":"2023-07-13T22:12:20.253875+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:12:20.253875+0000","last_undegraded":"2023-07-13T22:12:20.253875+0000","last_fullsized":"2023-07-13T22:12:20.253875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:01:13.663502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654966+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:12:20.654966+0000","last_peered":"2023-07-13T22:12:20.654966+0000","last_clean":"2023-07-13T22:12:20.654966+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:12:20.654966+0000","last_undegraded":"2023-07-13T22:12:20.654966+0000","last_fullsized":"2023-07-13T22:12:20.654966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:01.763241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658843+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:12:20.658843+0000","last_peered":"2023-07-13T22:12:20.658843+0000","last_clean":"2023-07-13T22:12:20.658843+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:12:20.658843+0000","last_undegraded":"2023-07-13T22:12:20.658843+0000","last_fullsized":"2023-07-13T22:12:20.658843+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:21:05.836286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.523903+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:12:20.523903+0000","last_peered":"2023-07-13T22:12:20.523903+0000","last_clean":"2023-07-13T22:12:20.523903+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:12:20.523903+0000","last_undegraded":"2023-07-13T22:12:20.523903+0000","last_fullsized":"2023-07-13T22:12:20.523903+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:16.864029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964438+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:12:19.964438+0000","last_peered":"2023-07-13T22:12:19.964438+0000","last_clean":"2023-07-13T22:12:19.964438+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:12:19.964438+0000","last_undegraded":"2023-07-13T22:12:19.964438+0000","last_fullsized":"2023-07-13T22:12:19.964438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:06.441040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.963974+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:12:19.963974+0000","last_peered":"2023-07-13T22:12:19.963974+0000","last_clean":"2023-07-13T22:12:19.963974+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:12:19.963974+0000","last_undegraded":"2023-07-13T22:12:19.963974+0000","last_fullsized":"2023-07-13T22:12:19.963974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:39:55.463767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654609+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:12:20.654609+0000","last_peered":"2023-07-13T22:12:20.654609+0000","last_clean":"2023-07-13T22:12:20.654609+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:12:20.654609+0000","last_undegraded":"2023-07-13T22:12:20.654609+0000","last_fullsized":"2023-07-13T22:12:20.654609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:07:52.248774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.524045+0000","last_change":"2023-07-13T22:12:18.970433+0000","last_active":"2023-07-13T22:12:20.524045+0000","last_peered":"2023-07-13T22:12:20.524045+0000","last_clean":"2023-07-13T22:12:20.524045+0000","last_became_active":"2023-07-13T22:12:18.969903+0000","last_became_peered":"2023-07-13T22:12:18.969903+0000","last_unstale":"2023-07-13T22:12:20.524045+0000","last_undegraded":"2023-07-13T22:12:20.524045+0000","last_fullsized":"2023-07-13T22:12:20.524045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:42:08.002303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.965021+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:12:19.965021+0000","last_peered":"2023-07-13T22:12:19.965021+0000","last_clean":"2023-07-13T22:12:19.965021+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:12:19.965021+0000","last_undegraded":"2023-07-13T22:12:19.965021+0000","last_fullsized":"2023-07-13T22:12:19.965021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:13.816929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654946+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:12:20.654946+0000","last_peered":"2023-07-13T22:12:20.654946+0000","last_clean":"2023-07-13T22:12:20.654946+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:12:20.654946+0000","last_undegraded":"2023-07-13T22:12:20.654946+0000","last_fullsized":"2023-07-13T22:12:20.654946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:57:24.559497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654494+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:12:20.654494+0000","last_peered":"2023-07-13T22:12:20.654494+0000","last_clean":"2023-07-13T22:12:20.654494+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:12:20.654494+0000","last_undegraded":"2023-07-13T22:12:20.654494+0000","last_fullsized":"2023-07-13T22:12:20.654494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:44:41.432615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253809+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:12:20.253809+0000","last_peered":"2023-07-13T22:12:20.253809+0000","last_clean":"2023-07-13T22:12:20.253809+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:12:20.253809+0000","last_undegraded":"2023-07-13T22:12:20.253809+0000","last_fullsized":"2023-07-13T22:12:20.253809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:30:53.828609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253734+0000","last_change":"2023-07-13T22:12:18.972104+0000","last_active":"2023-07-13T22:12:20.253734+0000","last_peered":"2023-07-13T22:12:20.253734+0000","last_clean":"2023-07-13T22:12:20.253734+0000","last_became_active":"2023-07-13T22:12:18.971822+0000","last_became_peered":"2023-07-13T22:12:18.971822+0000","last_unstale":"2023-07-13T22:12:20.253734+0000","last_undegraded":"2023-07-13T22:12:20.253734+0000","last_fullsized":"2023-07-13T22:12:20.253734+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:05:53.713229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653964+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:12:20.653964+0000","last_peered":"2023-07-13T22:12:20.653964+0000","last_clean":"2023-07-13T22:12:20.653964+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:12:20.653964+0000","last_undegraded":"2023-07-13T22:12:20.653964+0000","last_fullsized":"2023-07-13T22:12:20.653964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:02.543666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.970124+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:12:19.970124+0000","last_peered":"2023-07-13T22:12:19.970124+0000","last_clean":"2023-07-13T22:12:19.970124+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:12:19.970124+0000","last_undegraded":"2023-07-13T22:12:19.970124+0000","last_fullsized":"2023-07-13T22:12:19.970124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:50:30.192327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246333+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:12:20.246333+0000","last_peered":"2023-07-13T22:12:20.246333+0000","last_clean":"2023-07-13T22:12:20.246333+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:12:20.246333+0000","last_undegraded":"2023-07-13T22:12:20.246333+0000","last_fullsized":"2023-07-13T22:12:20.246333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:46:46.939883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654295+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:12:20.654295+0000","last_peered":"2023-07-13T22:12:20.654295+0000","last_clean":"2023-07-13T22:12:20.654295+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:12:20.654295+0000","last_undegraded":"2023-07-13T22:12:20.654295+0000","last_fullsized":"2023-07-13T22:12:20.654295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:56:02.671624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'11","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446482+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:12:24.446482+0000","last_peered":"2023-07-13T22:12:24.446482+0000","last_clean":"2023-07-13T22:12:24.446482+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:12:24.446482+0000","last_undegraded":"2023-07-13T22:12:24.446482+0000","last_fullsized":"2023-07-13T22:12:24.446482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:36:55.215409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964681+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:12:19.964681+0000","last_peered":"2023-07-13T22:12:19.964681+0000","last_clean":"2023-07-13T22:12:19.964681+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:12:19.964681+0000","last_undegraded":"2023-07-13T22:12:19.964681+0000","last_fullsized":"2023-07-13T22:12:19.964681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:21:34.769717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.213989+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:20.213989+0000","last_peered":"2023-07-13T22:12:20.213989+0000","last_clean":"2023-07-13T22:12:20.213989+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:20.213989+0000","last_undegraded":"2023-07-13T22:12:20.213989+0000","last_fullsized":"2023-07-13T22:12:20.213989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:27.315219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253952+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:12:20.253952+0000","last_peered":"2023-07-13T22:12:20.253952+0000","last_clean":"2023-07-13T22:12:20.253952+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:12:20.253952+0000","last_undegraded":"2023-07-13T22:12:20.253952+0000","last_fullsized":"2023-07-13T22:12:20.253952+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:20.411763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.444701+0000","last_change":"2023-07-13T22:12:17.986791+0000","last_active":"2023-07-13T22:12:24.444701+0000","last_peered":"2023-07-13T22:12:24.444701+0000","last_clean":"2023-07-13T22:12:24.444701+0000","last_became_active":"2023-07-13T22:12:17.986273+0000","last_became_peered":"2023-07-13T22:12:17.986273+0000","last_unstale":"2023-07-13T22:12:24.444701+0000","last_undegraded":"2023-07-13T22:12:24.444701+0000","last_fullsized":"2023-07-13T22:12:24.444701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:15.486007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654149+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:12:20.654149+0000","last_peered":"2023-07-13T22:12:20.654149+0000","last_clean":"2023-07-13T22:12:20.654149+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:12:20.654149+0000","last_undegraded":"2023-07-13T22:12:20.654149+0000","last_fullsized":"2023-07-13T22:12:20.654149+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:14:15.958627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659357+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:20.659357+0000","last_peered":"2023-07-13T22:12:20.659357+0000","last_clean":"2023-07-13T22:12:20.659357+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:20.659357+0000","last_undegraded":"2023-07-13T22:12:20.659357+0000","last_fullsized":"2023-07-13T22:12:20.659357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:03:08.364201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659110+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:12:20.659110+0000","last_peered":"2023-07-13T22:12:20.659110+0000","last_clean":"2023-07-13T22:12:20.659110+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:12:20.659110+0000","last_undegraded":"2023-07-13T22:12:20.659110+0000","last_fullsized":"2023-07-13T22:12:20.659110+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:10:02.649510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.439199+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:12:25.439199+0000","last_peered":"2023-07-13T22:12:25.439199+0000","last_clean":"2023-07-13T22:12:25.439199+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:12:25.439199+0000","last_undegraded":"2023-07-13T22:12:25.439199+0000","last_fullsized":"2023-07-13T22:12:25.439199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:54:45.909242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658675+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:12:20.658675+0000","last_peered":"2023-07-13T22:12:20.658675+0000","last_clean":"2023-07-13T22:12:20.658675+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:12:20.658675+0000","last_undegraded":"2023-07-13T22:12:20.658675+0000","last_fullsized":"2023-07-13T22:12:20.658675+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:27.563406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446736+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:24.446736+0000","last_peered":"2023-07-13T22:12:24.446736+0000","last_clean":"2023-07-13T22:12:24.446736+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:24.446736+0000","last_undegraded":"2023-07-13T22:12:24.446736+0000","last_fullsized":"2023-07-13T22:12:24.446736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:09:21.483570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659361+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:12:20.659361+0000","last_peered":"2023-07-13T22:12:20.659361+0000","last_clean":"2023-07-13T22:12:20.659361+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:12:20.659361+0000","last_undegraded":"2023-07-13T22:12:20.659361+0000","last_fullsized":"2023-07-13T22:12:20.659361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:32.512828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:23.996982+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:23.996982+0000","last_peered":"2023-07-13T22:12:23.996982+0000","last_clean":"2023-07-13T22:12:23.996982+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:23.996982+0000","last_undegraded":"2023-07-13T22:12:23.996982+0000","last_fullsized":"2023-07-13T22:12:23.996982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:09:31.802556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964105+0000","last_change":"2023-07-13T22:12:18.980968+0000","last_active":"2023-07-13T22:12:19.964105+0000","last_peered":"2023-07-13T22:12:19.964105+0000","last_clean":"2023-07-13T22:12:19.964105+0000","last_became_active":"2023-07-13T22:12:18.980166+0000","last_became_peered":"2023-07-13T22:12:18.980166+0000","last_unstale":"2023-07-13T22:12:19.964105+0000","last_undegraded":"2023-07-13T22:12:19.964105+0000","last_fullsized":"2023-07-13T22:12:19.964105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:25:06.678041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.451173+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:25.451173+0000","last_peered":"2023-07-13T22:12:25.451173+0000","last_clean":"2023-07-13T22:12:25.451173+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:25.451173+0000","last_undegraded":"2023-07-13T22:12:25.451173+0000","last_fullsized":"2023-07-13T22:12:25.451173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:00:50.214242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246083+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:12:20.246083+0000","last_peered":"2023-07-13T22:12:20.246083+0000","last_clean":"2023-07-13T22:12:20.246083+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:12:20.246083+0000","last_undegraded":"2023-07-13T22:12:20.246083+0000","last_fullsized":"2023-07-13T22:12:20.246083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:58:35.345174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.440055+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:12:26.440055+0000","last_peered":"2023-07-13T22:12:26.440055+0000","last_clean":"2023-07-13T22:12:26.440055+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:12:26.440055+0000","last_undegraded":"2023-07-13T22:12:26.440055+0000","last_fullsized":"2023-07-13T22:12:26.440055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:17:35.796379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654434+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:12:20.654434+0000","last_peered":"2023-07-13T22:12:20.654434+0000","last_clean":"2023-07-13T22:12:20.654434+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:12:20.654434+0000","last_undegraded":"2023-07-13T22:12:20.654434+0000","last_fullsized":"2023-07-13T22:12:20.654434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:29:22.530212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.986518+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:22.986518+0000","last_peered":"2023-07-13T22:12:22.986518+0000","last_clean":"2023-07-13T22:12:22.986518+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:22.986518+0000","last_undegraded":"2023-07-13T22:12:22.986518+0000","last_fullsized":"2023-07-13T22:12:22.986518+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:55:33.555639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253873+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:12:20.253873+0000","last_peered":"2023-07-13T22:12:20.253873+0000","last_clean":"2023-07-13T22:12:20.253873+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:12:20.253873+0000","last_undegraded":"2023-07-13T22:12:20.253873+0000","last_fullsized":"2023-07-13T22:12:20.253873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:52:47.750308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.437022+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:24.437022+0000","last_peered":"2023-07-13T22:12:24.437022+0000","last_clean":"2023-07-13T22:12:24.437022+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:24.437022+0000","last_undegraded":"2023-07-13T22:12:24.437022+0000","last_fullsized":"2023-07-13T22:12:24.437022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:09:00.597660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654261+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:12:20.654261+0000","last_peered":"2023-07-13T22:12:20.654261+0000","last_clean":"2023-07-13T22:12:20.654261+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:12:20.654261+0000","last_undegraded":"2023-07-13T22:12:20.654261+0000","last_fullsized":"2023-07-13T22:12:20.654261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:50:54.792816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.441876+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:12:26.441876+0000","last_peered":"2023-07-13T22:12:26.441876+0000","last_clean":"2023-07-13T22:12:26.441876+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:12:26.441876+0000","last_undegraded":"2023-07-13T22:12:26.441876+0000","last_fullsized":"2023-07-13T22:12:26.441876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:29:00.307908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23542,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":19,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659474+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:12:20.659474+0000","last_peered":"2023-07-13T22:12:20.659474+0000","last_clean":"2023-07-13T22:12:20.659474+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:12:20.659474+0000","last_undegraded":"2023-07-13T22:12:20.659474+0000","last_fullsized":"2023-07-13T22:12:20.659474+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:03:30.825329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'7","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.434019+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:24.434019+0000","last_peered":"2023-07-13T22:12:24.434019+0000","last_clean":"2023-07-13T22:12:24.434019+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:24.434019+0000","last_undegraded":"2023-07-13T22:12:24.434019+0000","last_fullsized":"2023-07-13T22:12:24.434019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:17:11.346880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654290+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:12:20.654290+0000","last_peered":"2023-07-13T22:12:20.654290+0000","last_clean":"2023-07-13T22:12:20.654290+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:12:20.654290+0000","last_undegraded":"2023-07-13T22:12:20.654290+0000","last_fullsized":"2023-07-13T22:12:20.654290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:40:19.190355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.451269+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:25.451269+0000","last_peered":"2023-07-13T22:12:25.451269+0000","last_clean":"2023-07-13T22:12:25.451269+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:25.451269+0000","last_undegraded":"2023-07-13T22:12:25.451269+0000","last_fullsized":"2023-07-13T22:12:25.451269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:11:03.850573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964835+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:12:19.964835+0000","last_peered":"2023-07-13T22:12:19.964835+0000","last_clean":"2023-07-13T22:12:19.964835+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:12:19.964835+0000","last_undegraded":"2023-07-13T22:12:19.964835+0000","last_fullsized":"2023-07-13T22:12:19.964835+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:10.032807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654853+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:12:20.654853+0000","last_peered":"2023-07-13T22:12:20.654853+0000","last_clean":"2023-07-13T22:12:20.654853+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:12:20.654853+0000","last_undegraded":"2023-07-13T22:12:20.654853+0000","last_fullsized":"2023-07-13T22:12:20.654853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:52:04.877520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964385+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:12:19.964385+0000","last_peered":"2023-07-13T22:12:19.964385+0000","last_clean":"2023-07-13T22:12:19.964385+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:12:19.964385+0000","last_undegraded":"2023-07-13T22:12:19.964385+0000","last_fullsized":"2023-07-13T22:12:19.964385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:43:04.765916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'16","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.434055+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:24.434055+0000","last_peered":"2023-07-13T22:12:24.434055+0000","last_clean":"2023-07-13T22:12:24.434055+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:24.434055+0000","last_undegraded":"2023-07-13T22:12:24.434055+0000","last_fullsized":"2023-07-13T22:12:24.434055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:49:36.100083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658871+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:12:20.658871+0000","last_peered":"2023-07-13T22:12:20.658871+0000","last_clean":"2023-07-13T22:12:20.658871+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:12:20.658871+0000","last_undegraded":"2023-07-13T22:12:20.658871+0000","last_fullsized":"2023-07-13T22:12:20.658871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:57:14.031661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659630+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:20.659630+0000","last_peered":"2023-07-13T22:12:20.659630+0000","last_clean":"2023-07-13T22:12:20.659630+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:20.659630+0000","last_undegraded":"2023-07-13T22:12:20.659630+0000","last_fullsized":"2023-07-13T22:12:20.659630+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:36:32.688356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969908+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:12:19.969908+0000","last_peered":"2023-07-13T22:12:19.969908+0000","last_clean":"2023-07-13T22:12:19.969908+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:12:19.969908+0000","last_undegraded":"2023-07-13T22:12:19.969908+0000","last_fullsized":"2023-07-13T22:12:19.969908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:46:21.674730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.246220+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:20.246220+0000","last_peered":"2023-07-13T22:12:20.246220+0000","last_clean":"2023-07-13T22:12:20.246220+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:20.246220+0000","last_undegraded":"2023-07-13T22:12:20.246220+0000","last_fullsized":"2023-07-13T22:12:20.246220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:22:52.330175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":13,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:21.739637+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:12:21.739637+0000","last_peered":"2023-07-13T22:12:21.739637+0000","last_clean":"2023-07-13T22:12:21.739637+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:12:21.739637+0000","last_undegraded":"2023-07-13T22:12:21.739637+0000","last_fullsized":"2023-07-13T22:12:21.739637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:17:58.279474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964543+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:12:19.964543+0000","last_peered":"2023-07-13T22:12:19.964543+0000","last_clean":"2023-07-13T22:12:19.964543+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:12:19.964543+0000","last_undegraded":"2023-07-13T22:12:19.964543+0000","last_fullsized":"2023-07-13T22:12:19.964543+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:53:50.719647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.523699+0000","last_change":"2023-07-13T22:12:18.983052+0000","last_active":"2023-07-13T22:12:20.523699+0000","last_peered":"2023-07-13T22:12:20.523699+0000","last_clean":"2023-07-13T22:12:20.523699+0000","last_became_active":"2023-07-13T22:12:18.982747+0000","last_became_peered":"2023-07-13T22:12:18.982747+0000","last_unstale":"2023-07-13T22:12:20.523699+0000","last_undegraded":"2023-07-13T22:12:20.523699+0000","last_fullsized":"2023-07-13T22:12:20.523699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:17:25.757531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659033+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:12:20.659033+0000","last_peered":"2023-07-13T22:12:20.659033+0000","last_clean":"2023-07-13T22:12:20.659033+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:12:20.659033+0000","last_undegraded":"2023-07-13T22:12:20.659033+0000","last_fullsized":"2023-07-13T22:12:20.659033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:31:33.801533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:26.441639+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:12:26.441639+0000","last_peered":"2023-07-13T22:12:26.441639+0000","last_clean":"2023-07-13T22:12:26.441639+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:12:26.441639+0000","last_undegraded":"2023-07-13T22:12:26.441639+0000","last_fullsized":"2023-07-13T22:12:26.441639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:33.552236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964542+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:12:19.964542+0000","last_peered":"2023-07-13T22:12:19.964542+0000","last_clean":"2023-07-13T22:12:19.964542+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:12:19.964542+0000","last_undegraded":"2023-07-13T22:12:19.964542+0000","last_fullsized":"2023-07-13T22:12:19.964542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:35:31.095800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.655483+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:20.655483+0000","last_peered":"2023-07-13T22:12:20.655483+0000","last_clean":"2023-07-13T22:12:20.655483+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:20.655483+0000","last_undegraded":"2023-07-13T22:12:20.655483+0000","last_fullsized":"2023-07-13T22:12:20.655483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:28:07.554927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":330,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964496+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:19.964496+0000","last_peered":"2023-07-13T22:12:19.964496+0000","last_clean":"2023-07-13T22:12:19.964496+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:19.964496+0000","last_undegraded":"2023-07-13T22:12:19.964496+0000","last_fullsized":"2023-07-13T22:12:19.964496+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:15:03.934747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654179+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:12:20.654179+0000","last_peered":"2023-07-13T22:12:20.654179+0000","last_clean":"2023-07-13T22:12:20.654179+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:12:20.654179+0000","last_undegraded":"2023-07-13T22:12:20.654179+0000","last_fullsized":"2023-07-13T22:12:20.654179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:47:55.597112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964751+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:19.964751+0000","last_peered":"2023-07-13T22:12:19.964751+0000","last_clean":"2023-07-13T22:12:19.964751+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:19.964751+0000","last_undegraded":"2023-07-13T22:12:19.964751+0000","last_fullsized":"2023-07-13T22:12:19.964751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:33:22.737711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.445522+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:24.445522+0000","last_peered":"2023-07-13T22:12:24.445522+0000","last_clean":"2023-07-13T22:12:24.445522+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:24.445522+0000","last_undegraded":"2023-07-13T22:12:24.445522+0000","last_fullsized":"2023-07-13T22:12:24.445522+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:44:59.864576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654358+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:12:20.654358+0000","last_peered":"2023-07-13T22:12:20.654358+0000","last_clean":"2023-07-13T22:12:20.654358+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:12:20.654358+0000","last_undegraded":"2023-07-13T22:12:20.654358+0000","last_fullsized":"2023-07-13T22:12:20.654358+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:16:26.007193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'7","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446769+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:24.446769+0000","last_peered":"2023-07-13T22:12:24.446769+0000","last_clean":"2023-07-13T22:12:24.446769+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:24.446769+0000","last_undegraded":"2023-07-13T22:12:24.446769+0000","last_fullsized":"2023-07-13T22:12:24.446769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:39:11.802625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964328+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:12:19.964328+0000","last_peered":"2023-07-13T22:12:19.964328+0000","last_clean":"2023-07-13T22:12:19.964328+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:12:19.964328+0000","last_undegraded":"2023-07-13T22:12:19.964328+0000","last_fullsized":"2023-07-13T22:12:19.964328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:02:47.508251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654872+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:20.654872+0000","last_peered":"2023-07-13T22:12:20.654872+0000","last_clean":"2023-07-13T22:12:20.654872+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:20.654872+0000","last_undegraded":"2023-07-13T22:12:20.654872+0000","last_fullsized":"2023-07-13T22:12:20.654872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:23:08.462604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253938+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:12:20.253938+0000","last_peered":"2023-07-13T22:12:20.253938+0000","last_clean":"2023-07-13T22:12:20.253938+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:12:20.253938+0000","last_undegraded":"2023-07-13T22:12:20.253938+0000","last_fullsized":"2023-07-13T22:12:20.253938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:41:40.442692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.988369+0000","last_change":"2023-07-13T22:12:17.987479+0000","last_active":"2023-07-13T22:12:22.988369+0000","last_peered":"2023-07-13T22:12:22.988369+0000","last_clean":"2023-07-13T22:12:22.988369+0000","last_became_active":"2023-07-13T22:12:17.987141+0000","last_became_peered":"2023-07-13T22:12:17.987141+0000","last_unstale":"2023-07-13T22:12:22.988369+0000","last_undegraded":"2023-07-13T22:12:22.988369+0000","last_fullsized":"2023-07-13T22:12:22.988369+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:31.713242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.659192+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:12:20.659192+0000","last_peered":"2023-07-13T22:12:20.659192+0000","last_clean":"2023-07-13T22:12:20.659192+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:12:20.659192+0000","last_undegraded":"2023-07-13T22:12:20.659192+0000","last_fullsized":"2023-07-13T22:12:20.659192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:39:25.212659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.446501+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:24.446501+0000","last_peered":"2023-07-13T22:12:24.446501+0000","last_clean":"2023-07-13T22:12:24.446501+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:24.446501+0000","last_undegraded":"2023-07-13T22:12:24.446501+0000","last_fullsized":"2023-07-13T22:12:24.446501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:50:52.980622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969621+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:12:19.969621+0000","last_peered":"2023-07-13T22:12:19.969621+0000","last_clean":"2023-07-13T22:12:19.969621+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:12:19.969621+0000","last_undegraded":"2023-07-13T22:12:19.969621+0000","last_fullsized":"2023-07-13T22:12:19.969621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:54:16.827543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.085032+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:24.085032+0000","last_peered":"2023-07-13T22:12:24.085032+0000","last_clean":"2023-07-13T22:12:24.085032+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:24.085032+0000","last_undegraded":"2023-07-13T22:12:24.085032+0000","last_fullsized":"2023-07-13T22:12:24.085032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:00:25.632958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964616+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:12:19.964616+0000","last_peered":"2023-07-13T22:12:19.964616+0000","last_clean":"2023-07-13T22:12:19.964616+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:12:19.964616+0000","last_undegraded":"2023-07-13T22:12:19.964616+0000","last_fullsized":"2023-07-13T22:12:19.964616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:14:00.526274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:22.987233+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:22.987233+0000","last_peered":"2023-07-13T22:12:22.987233+0000","last_clean":"2023-07-13T22:12:22.987233+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:22.987233+0000","last_undegraded":"2023-07-13T22:12:22.987233+0000","last_fullsized":"2023-07-13T22:12:22.987233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:29:44.146326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964781+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:12:19.964781+0000","last_peered":"2023-07-13T22:12:19.964781+0000","last_clean":"2023-07-13T22:12:19.964781+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:12:19.964781+0000","last_undegraded":"2023-07-13T22:12:19.964781+0000","last_fullsized":"2023-07-13T22:12:19.964781+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:54:59.263980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.444274+0000","last_change":"2023-07-13T22:12:17.991052+0000","last_active":"2023-07-13T22:12:24.444274+0000","last_peered":"2023-07-13T22:12:24.444274+0000","last_clean":"2023-07-13T22:12:24.444274+0000","last_became_active":"2023-07-13T22:12:17.990314+0000","last_became_peered":"2023-07-13T22:12:17.990314+0000","last_unstale":"2023-07-13T22:12:24.444274+0000","last_undegraded":"2023-07-13T22:12:24.444274+0000","last_fullsized":"2023-07-13T22:12:24.444274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:01:36.285647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964898+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:12:19.964898+0000","last_peered":"2023-07-13T22:12:19.964898+0000","last_clean":"2023-07-13T22:12:19.964898+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:12:19.964898+0000","last_undegraded":"2023-07-13T22:12:19.964898+0000","last_fullsized":"2023-07-13T22:12:19.964898+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:40:48.914208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:24.445291+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:24.445291+0000","last_peered":"2023-07-13T22:12:24.445291+0000","last_clean":"2023-07-13T22:12:24.445291+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:24.445291+0000","last_undegraded":"2023-07-13T22:12:24.445291+0000","last_fullsized":"2023-07-13T22:12:24.445291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:39:03.087977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964537+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:12:19.964537+0000","last_peered":"2023-07-13T22:12:19.964537+0000","last_clean":"2023-07-13T22:12:19.964537+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:12:19.964537+0000","last_undegraded":"2023-07-13T22:12:19.964537+0000","last_fullsized":"2023-07-13T22:12:19.964537+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:45:14.493969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:25.454731+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:25.454731+0000","last_peered":"2023-07-13T22:12:25.454731+0000","last_clean":"2023-07-13T22:12:25.454731+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:25.454731+0000","last_undegraded":"2023-07-13T22:12:25.454731+0000","last_fullsized":"2023-07-13T22:12:25.454731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:28:57.362060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964984+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:12:19.964984+0000","last_peered":"2023-07-13T22:12:19.964984+0000","last_clean":"2023-07-13T22:12:19.964984+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:12:19.964984+0000","last_undegraded":"2023-07-13T22:12:19.964984+0000","last_fullsized":"2023-07-13T22:12:19.964984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:04:08.340928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969615+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:12:19.969615+0000","last_peered":"2023-07-13T22:12:19.969615+0000","last_clean":"2023-07-13T22:12:19.969615+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:12:19.969615+0000","last_undegraded":"2023-07-13T22:12:19.969615+0000","last_fullsized":"2023-07-13T22:12:19.969615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:54:54.278671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653922+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:12:20.653922+0000","last_peered":"2023-07-13T22:12:20.653922+0000","last_clean":"2023-07-13T22:12:20.653922+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:12:20.653922+0000","last_undegraded":"2023-07-13T22:12:20.653922+0000","last_fullsized":"2023-07-13T22:12:20.653922+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:20.741383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.653937+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:12:20.653937+0000","last_peered":"2023-07-13T22:12:20.653937+0000","last_clean":"2023-07-13T22:12:20.653937+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:12:20.653937+0000","last_undegraded":"2023-07-13T22:12:20.653937+0000","last_fullsized":"2023-07-13T22:12:20.653937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:22:51.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.253940+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:12:20.253940+0000","last_peered":"2023-07-13T22:12:20.253940+0000","last_clean":"2023-07-13T22:12:20.253940+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:12:20.253940+0000","last_undegraded":"2023-07-13T22:12:20.253940+0000","last_fullsized":"2023-07-13T22:12:20.253940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:36:20.517694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654502+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:12:20.654502+0000","last_peered":"2023-07-13T22:12:20.654502+0000","last_clean":"2023-07-13T22:12:20.654502+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:12:20.654502+0000","last_undegraded":"2023-07-13T22:12:20.654502+0000","last_fullsized":"2023-07-13T22:12:20.654502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:21:36.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969840+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:12:19.969840+0000","last_peered":"2023-07-13T22:12:19.969840+0000","last_clean":"2023-07-13T22:12:19.969840+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:12:19.969840+0000","last_undegraded":"2023-07-13T22:12:19.969840+0000","last_fullsized":"2023-07-13T22:12:19.969840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:30:48.331390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654409+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:12:20.654409+0000","last_peered":"2023-07-13T22:12:20.654409+0000","last_clean":"2023-07-13T22:12:20.654409+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:12:20.654409+0000","last_undegraded":"2023-07-13T22:12:20.654409+0000","last_fullsized":"2023-07-13T22:12:20.654409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:05:06.409809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964319+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:12:19.964319+0000","last_peered":"2023-07-13T22:12:19.964319+0000","last_clean":"2023-07-13T22:12:19.964319+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:12:19.964319+0000","last_undegraded":"2023-07-13T22:12:19.964319+0000","last_fullsized":"2023-07-13T22:12:19.964319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:20:39.162208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.658654+0000","last_change":"2023-07-13T22:12:18.980792+0000","last_active":"2023-07-13T22:12:20.658654+0000","last_peered":"2023-07-13T22:12:20.658654+0000","last_clean":"2023-07-13T22:12:20.658654+0000","last_became_active":"2023-07-13T22:12:18.980488+0000","last_became_peered":"2023-07-13T22:12:18.980488+0000","last_unstale":"2023-07-13T22:12:20.658654+0000","last_undegraded":"2023-07-13T22:12:20.658654+0000","last_fullsized":"2023-07-13T22:12:20.658654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:21:54.718460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.964433+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:12:19.964433+0000","last_peered":"2023-07-13T22:12:19.964433+0000","last_clean":"2023-07-13T22:12:19.964433+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:12:19.964433+0000","last_undegraded":"2023-07-13T22:12:19.964433+0000","last_fullsized":"2023-07-13T22:12:19.964433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:38:40.316111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654694+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:12:20.654694+0000","last_peered":"2023-07-13T22:12:20.654694+0000","last_clean":"2023-07-13T22:12:20.654694+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:12:20.654694+0000","last_undegraded":"2023-07-13T22:12:20.654694+0000","last_fullsized":"2023-07-13T22:12:20.654694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:05:25.381241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:20.654510+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:12:20.654510+0000","last_peered":"2023-07-13T22:12:20.654510+0000","last_clean":"2023-07-13T22:12:20.654510+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:12:20.654510+0000","last_undegraded":"2023-07-13T22:12:20.654510+0000","last_fullsized":"2023-07-13T22:12:20.654510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:09:11.970190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-07-13T22:12:19.969676+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:12:19.969676+0000","last_peered":"2023-07-13T22:12:19.969676+0000","last_clean":"2023-07-13T22:12:19.969676+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:12:19.969676+0000","last_undegraded":"2023-07-13T22:12:19.969676+0000","last_fullsized":"2023-07-13T22:12:19.969676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:51:17.934954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":27840,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":35,"num_read_kb":45,"num_write":161,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":37002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":425818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":10,"up_from":71,"seq":304942678034,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1560576,"data_stored":1036926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.3220000000000001,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.2800000000000002,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.234,"5min":5.234,"15min":5.234},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72399999999999998}]},{"osd":9,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.753}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841556,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":426602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.746},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.5},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.84699999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":0.876},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.85899999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.95799999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103194,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.745}]},{"osd":2,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.876},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.77300000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.81799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332126,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28404,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716748,"statfs":{"total":95995035648,"available":95965949952,"internally_reserved":0,"allocated":1486848,"data_stored":1016768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.57999999999999996,"5min":1.48,"15min":1.48},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82799999999999996,"5min":0.871,"15min":0.871},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92800000000000005,"5min":1.056,"15min":1.056},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94099999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.86499999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.83399999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.96499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495650,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":426615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.39500000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.84999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91400000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.751},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.014,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.45000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.085,"15min":1.085},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.505,"5min":0.503,"15min":0.503},"max":{"1min":1.0149999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.01,"5min":1.726,"15min":1.726},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1020000000000001,"5min":1.369,"15min":1.369},"last":0.879},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81499999999999995,"15min":0.80700000000000005},"min":{"1min":0.58499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.085,"5min":1.573,"15min":1.573},"last":0.58299999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.83499999999999996},"min":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0780000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.78000000000000003},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.70299999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.84599999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.59999999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":427382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52800000000000002,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.78400000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.874,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.88700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.36299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.499,"15min":0.46400000000000002},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57599999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:11:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.51200000000000001},"min":{"1min":0.499,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.57199999999999995,"15min":0.54000000000000004},"min":{"1min":0.55300000000000005,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.83499999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.31,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80200000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93600000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.90600000000000003}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.98599999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":1.042},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":1.026}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1.014}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.318,"15min":0.318},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.746},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.45200000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85399999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.64300000000000002,"15min":0.59899999999999998},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.65900000000000003,"15min":0.61599999999999999},"min":{"1min":0.45200000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Jul 13 22:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.52400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86699999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39800000000000002}]},{"osd":6,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.72499999999999998,"15min":0.69999999999999996},"min":{"1min":0.49099999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.45300000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.878}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.443}]},{"osd":9,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.66000000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.59899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":428015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629}]},{"osd":1,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.875},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78200000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:11:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.64900000000000002,"15min":0.625},"min":{"1min":0.47199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.2469999999999999,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.46000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.92200000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64900000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89200000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:26 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":1.0209999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":4.21,"5min":4.21,"15min":4.21},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.90500000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.054}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":425895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.41399999999999998},"min":{"1min":0.33600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63700000000000001,"5min":0.751,"15min":0.751},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.441,"15min":0.42699999999999999},"min":{"1min":0.35399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.63500000000000001,"15min":0.59699999999999998},"min":{"1min":0.57299999999999995,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61899999999999999,"15min":0.58599999999999997},"min":{"1min":0.45100000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.65100000000000002,"15min":0.621},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1,"5min":1.046,"15min":1.046},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.67700000000000005,"15min":0.63400000000000001},"min":{"1min":0.51500000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.70499999999999996,"15min":0.66400000000000003},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.006},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65000000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67000000000000004,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.71399999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.60099999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.85899999999999999,"15min":0.84899999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Jul 13 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.86599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920812,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":425840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60099999999999998,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71199999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60899999999999999,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.83399999999999996,"5min":1.081,"15min":1.081},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.5},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.66400000000000003,"15min":0.65900000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97499999999999998,"5min":1.095,"15min":1.095},"last":0.443},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.76800000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:11:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68999999999999995}]},{"osd":9,"last update":"Thu Jul 13 22:11:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.6459999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.754}]},{"osd":10,"last update":"Thu Jul 13 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757288,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1527808,"data_stored":1018958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":1.7170000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":3.2360000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":3.0009999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81799999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":2.4460000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.665}]},{"osd":3,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":2.9590000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.764},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":2.9159999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63300000000000001,"15min":0.61899999999999999},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.46400000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":2.032}]},{"osd":7,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":2.875},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.625,"15min":0.628},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":3.9929999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:11:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":2.8279999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88100000000000001}]},{"osd":9,"last update":"Thu Jul 13 22:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":2.0880000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":3.1139999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":3.0470000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.8089999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":25230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:12:29.396 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-13T22:12:29.396 INFO:tasks.cephadm:Setting up client nodes... 2023-07-13T22:12:29.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-13T22:12:29.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:29 smithi026 ceph-mon[95647]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-13T22:12:29.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:29 smithi121 ceph-mon[129187]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-13T22:12:29.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:29 smithi117 ceph-mon[129074]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-07-13T22:12:30.159 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:30.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:30 smithi121 ceph-mon[129187]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:30 smithi121 ceph-mon[129187]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:30 smithi121 ceph-mon[129187]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:30.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:30 smithi026 ceph-mon[95647]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:30 smithi026 ceph-mon[95647]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.802 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:30 smithi026 ceph-mon[95647]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:30.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:30 smithi117 ceph-mon[129074]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:30 smithi117 ceph-mon[129074]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:12:30.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:30 smithi117 ceph-mon[129074]: pgmap v272: 97 pgs: 97 active+clean; 605 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 -- 172.21.15.26:0/3008436869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c072f70 msgr2=0x7f920c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 --2- 172.21.15.26:0/3008436869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c072f70 0x7f920c0733e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009990 tx=0x7f91fc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 -- 172.21.15.26:0/3008436869 shutdown_connections 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 --2- 172.21.15.26:0/3008436869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c107a80 0x7f920c109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 --2- 172.21.15.26:0/3008436869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 0x7f920c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 --2- 172.21.15.26:0/3008436869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c072f70 0x7f920c0733e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 -- 172.21.15.26:0/3008436869 >> 172.21.15.26:0/3008436869 conn(0x7f920c0fb200 msgr2=0x7f920c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.809+0000 7f9213b0b700 1 -- 172.21.15.26:0/3008436869 shutdown_connections 2023-07-13T22:12:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.810+0000 7f9213b0b700 1 -- 172.21.15.26:0/3008436869 wait complete. 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.810+0000 7f9213b0b700 1 Processor -- start 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 -- start start 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c072f70 0x7f920c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 0x7f920c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19f620 con 0x7f920c107a80 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19f760 con 0x7f920c074540 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f9213b0b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c19f8f0 con 0x7f920c072f70 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f92120a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f92120a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46012/0 (socket says 172.21.15.26:46012) 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f92118a7700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c072f70 0x7f920c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f92120a8700 1 -- 172.21.15.26:0/2942228986 learned_addr learned my addr 172.21.15.26:0/2942228986 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.811+0000 7f92110a6700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 0x7f920c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 -- 172.21.15.26:0/2942228986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c072f70 msgr2=0x7f920c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c072f70 0x7f920c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 -- 172.21.15.26:0/2942228986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 msgr2=0x7f920c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 0x7f920c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 -- 172.21.15.26:0/2942228986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0095f0 con 0x7f920c107a80 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f92120a8700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9204007f10 tx=0x7f920400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400dd20 con 0x7f920c107a80 2023-07-13T22:12:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f920400f040 con 0x7f920c107a80 2023-07-13T22:12:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040048a0 con 0x7f920c107a80 2023-07-13T22:12:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.812+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c19fbd0 con 0x7f920c107a80 2023-07-13T22:12:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.813+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c1a0080 con 0x7f920c107a80 2023-07-13T22:12:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.814+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c106130 con 0x7f920c107a80 2023-07-13T22:12:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.815+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9204003a00 con 0x7f920c107a80 2023-07-13T22:12:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.817+0000 7f9202ffd700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 0x7f91f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.817+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f92040cc170 con 0x7f920c107a80 2023-07-13T22:12:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.817+0000 7f92118a7700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 0x7f91f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.818+0000 7f92118a7700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 0x7f91f80a3710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9208007c60 tx=0x7f92080073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:31.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:31.821+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f920404e790 con 0x7f920c107a80 2023-07-13T22:12:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.108+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f920c02ca60 con 0x7f920c107a80 2023-07-13T22:12:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.113+0000 7f9202ffd700 1 -- 172.21.15.26:0/2942228986 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9204095810 con 0x7f920c107a80 2023-07-13T22:12:32.104 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2023-07-13T22:12:32.104 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDQdrBk54WQBhAAXc3mAikiJbLO324mZCw+vQ== 2023-07-13T22:12:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 msgr2=0x7f91f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 0x7f91f80a3710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9208007c60 tx=0x7f92080073d0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 msgr2=0x7f920c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9204007f10 tx=0x7f920400d3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 shutdown_connections 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91f80a1260 0x7f91f80a3710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f920c107a80 0x7f920c19f050 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f920c072f70 0x7f920c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 --2- 172.21.15.26:0/2942228986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f920c074540 0x7f920c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 >> 172.21.15.26:0/2942228986 conn(0x7f920c0fb200 msgr2=0x7f920c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 shutdown_connections 2023-07-13T22:12:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:32.115+0000 7f9213b0b700 1 -- 172.21.15.26:0/2942228986 wait complete. 2023-07-13T22:12:32.543 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:12:32.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-13T22:12:32.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-13T22:12:32.646 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-13T22:12:32.647 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-13T22:12:32.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph mgr dump --format=json 2023-07-13T22:12:32.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:32 smithi121 ceph-mon[129187]: pgmap v273: 97 pgs: 97 active+clean; 606 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-13T22:12:32.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:32 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-13T22:12:32.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:32 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-13T22:12:32.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:32 smithi026 ceph-mon[95647]: pgmap v273: 97 pgs: 97 active+clean; 606 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-13T22:12:32.747 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:32 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-13T22:12:32.747 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:32 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-13T22:12:32.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:32 smithi117 ceph-mon[129074]: pgmap v273: 97 pgs: 97 active+clean; 606 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.6 KiB/s wr, 22 op/s 2023-07-13T22:12:32.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:32 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-13T22:12:32.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:32 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2942228986' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-13T22:12:33.417 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:34.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:34 smithi026 ceph-mon[95647]: pgmap v274: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-07-13T22:12:34.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:34 smithi121 ceph-mon[129187]: pgmap v274: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-07-13T22:12:34.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:34 smithi117 ceph-mon[129074]: pgmap v274: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2023-07-13T22:12:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 -- 172.21.15.26:0/2790452201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f8104ab0 msgr2=0x7fe7f8104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:34.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 --2- 172.21.15.26:0/2790452201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f8104ab0 0x7fe7f8104e80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009990 tx=0x7fe7f4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 -- 172.21.15.26:0/2790452201 shutdown_connections 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 --2- 172.21.15.26:0/2790452201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f8104ab0 0x7fe7f8104e80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 --2- 172.21.15.26:0/2790452201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f80ff460 0x7fe7f810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 --2- 172.21.15.26:0/2790452201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f80fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 -- 172.21.15.26:0/2790452201 >> 172.21.15.26:0/2790452201 conn(0x7fe7f8075b70 msgr2=0x7fe7f8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 -- 172.21.15.26:0/2790452201 shutdown_connections 2023-07-13T22:12:34.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.966+0000 7fe800691700 1 -- 172.21.15.26:0/2790452201 wait complete. 2023-07-13T22:12:34.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.967+0000 7fe800691700 1 Processor -- start 2023-07-13T22:12:34.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 -- start start 2023-07-13T22:12:34.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:34.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f80ff460 0x7fe7f8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:34.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f8104ab0 0x7fe7f819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819cea0 con 0x7fe7f80ff460 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819d010 con 0x7fe7f8104ab0 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.968+0000 7fe800691700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f819d1d0 con 0x7fe7f80feab0 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60078/0 (socket says 172.21.15.26:60078) 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 -- 172.21.15.26:0/412193761 learned_addr learned my addr 172.21.15.26:0/412193761 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fec2e700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f8104ab0 0x7fe7f819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:34.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fdc2c700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f80ff460 0x7fe7f8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 -- 172.21.15.26:0/412193761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f8104ab0 msgr2=0x7fe7f819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f8104ab0 0x7fe7f819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 -- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f80ff460 msgr2=0x7fe7f8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f80ff460 0x7fe7f8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:34.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 -- 172.21.15.26:0/412193761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40095f0 con 0x7fe7f80feab0 2023-07-13T22:12:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.969+0000 7fe7fe42d700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000c00 tx=0x7fe7f4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.970+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f401d070 con 0x7fe7f80feab0 2023-07-13T22:12:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.970+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f819d450 con 0x7fe7f80feab0 2023-07-13T22:12:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.970+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7f4021410 con 0x7fe7f80feab0 2023-07-13T22:12:34.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.970+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017650 con 0x7fe7f80feab0 2023-07-13T22:12:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.970+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f81a4d10 con 0x7fe7f80feab0 2023-07-13T22:12:34.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.971+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f804e210 con 0x7fe7f80feab0 2023-07-13T22:12:34.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.974+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe7f4021580 con 0x7fe7f80feab0 2023-07-13T22:12:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.976+0000 7fe7ef7fe700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 0x7fe7e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:34.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.976+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7f40ce380 con 0x7fe7f80feab0 2023-07-13T22:12:34.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.976+0000 7fe7fdc2c700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 0x7fe7e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:34.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.977+0000 7fe7fdc2c700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 0x7fe7e40a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8199500 tx=0x7fe7e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:34.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:34.978+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe7f4097ab0 con 0x7fe7f80feab0 2023-07-13T22:12:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.244+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe7f81a4ea0 con 0x7fe7f80feab0 2023-07-13T22:12:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.249+0000 7fe7ef7fe700 1 -- 172.21.15.26:0/412193761 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260125 (secure 0 0 0) 0x7fe7f409b0d0 con 0x7fe7f80feab0 2023-07-13T22:12:35.241 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 msgr2=0x7fe7e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 0x7fe7e40a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8199500 tx=0x7fe7e8009260 comp rx=0 tx=0).stop 2023-07-13T22:12:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 msgr2=0x7fe7f8197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000c00 tx=0x7fe7f4005740 comp rx=0 tx=0).stop 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 shutdown_connections 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7f8104ab0 0x7fe7f819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe7f80feab0 0x7fe7f8197ee0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe7e40a11e0 0x7fe7e40a3690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 --2- 172.21.15.26:0/412193761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7f80ff460 0x7fe7f8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 >> 172.21.15.26:0/412193761 conn(0x7fe7f8075b70 msgr2=0x7fe7f80fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 shutdown_connections 2023-07-13T22:12:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:35.254+0000 7fe800691700 1 -- 172.21.15.26:0/412193761 wait complete. 2023-07-13T22:12:35.575 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:35 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/412193761' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-13T22:12:35.601 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":4053809718},{"type":"v1","addr":"172.21.15.26:6801","nonce":4053809718}]},"active_addr":"172.21.15.26:6801/4053809718","active_change":"2023-07-13T22:05:51.339378+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.26:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3478184650}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3406498384}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":1918471153}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":935946693}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.26:0","nonce":3443084201}]}]} 2023-07-13T22:12:35.608 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-13T22:12:35.608 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-13T22:12:35.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-13T22:12:35.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:35 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/412193761' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-13T22:12:35.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:35 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/412193761' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-13T22:12:36.375 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:36.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:36 smithi121 ceph-mon[129187]: pgmap v275: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:36.746 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:36 smithi026 ceph-mon[95647]: pgmap v275: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:36.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:36 smithi117 ceph-mon[129074]: pgmap v275: 97 pgs: 97 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2023-07-13T22:12:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 -- 172.21.15.26:0/2760726134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 msgr2=0x7f83bc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 --2- 172.21.15.26:0/2760726134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc10ca20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b0d0 tx=0x7f83b400b3e0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 -- 172.21.15.26:0/2760726134 shutdown_connections 2023-07-13T22:12:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 --2- 172.21.15.26:0/2760726134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc10ca20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 --2- 172.21.15.26:0/2760726134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc0ffe00 0x7f83bc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 --2- 172.21.15.26:0/2760726134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc1007a0 0x7f83bc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.981+0000 7f83c4629700 1 -- 172.21.15.26:0/2760726134 >> 172.21.15.26:0/2760726134 conn(0x7f83bc0fb660 msgr2=0x7f83bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.982+0000 7f83c4629700 1 -- 172.21.15.26:0/2760726134 shutdown_connections 2023-07-13T22:12:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.982+0000 7f83c4629700 1 -- 172.21.15.26:0/2760726134 wait complete. 2023-07-13T22:12:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.983+0000 7f83c4629700 1 Processor -- start 2023-07-13T22:12:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.983+0000 7f83c4629700 1 -- start start 2023-07-13T22:12:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 0x7f83bc1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc19ad20 con 0x7f83bc104ea0 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc19ae90 con 0x7f83bc0ffe00 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c4629700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc19b050 con 0x7f83bc1007a0 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c2bc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c1bc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 0x7f83bc1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c2bc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49970/0 (socket says 172.21.15.26:49970) 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c2bc6700 1 -- 172.21.15.26:0/3568387536 learned_addr learned my addr 172.21.15.26:0/3568387536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 -- 172.21.15.26:0/3568387536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 msgr2=0x7f83bc1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 0x7f83bc1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 -- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 msgr2=0x7f83bc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c23c5700 1 -- 172.21.15.26:0/3568387536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009d00 con 0x7f83bc0ffe00 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.984+0000 7f83c1bc4700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 0x7f83bc1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83c23c5700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9f0 tx=0x7f83b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b800c9c0 con 0x7f83bc0ffe00 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b8010430 con 0x7f83bc0ffe00 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc19b330 con 0x7f83bc0ffe00 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8018680 con 0x7f83bc0ffe00 2023-07-13T22:12:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.985+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc1a2b90 con 0x7f83bc0ffe00 2023-07-13T22:12:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.986+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc04e210 con 0x7f83bc0ffe00 2023-07-13T22:12:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.988+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f83b8010860 con 0x7f83bc0ffe00 2023-07-13T22:12:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.989+0000 7f83b37fe700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 0x7f83a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.990+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f83b8014070 con 0x7f83bc0ffe00 2023-07-13T22:12:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.990+0000 7f83c1bc4700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 0x7f83a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.990+0000 7f83c1bc4700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 0x7f83a80a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83bc197380 tx=0x7f83ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:37.994+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f83b8051790 con 0x7f83bc0ffe00 2023-07-13T22:12:38.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.202+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83bc1a2e20 con 0x7f83bc0ffe00 2023-07-13T22:12:38.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.204+0000 7f83b37fe700 1 -- 172.21.15.26:0/3568387536 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22096 (secure 0 0 0) 0x7f83b80961b0 con 0x7f83bc0ffe00 2023-07-13T22:12:38.196 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:38.196 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:19.954949+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.206+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 msgr2=0x7f83a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.206+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 0x7f83a80a3690 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83bc197380 tx=0x7f83ac009260 comp rx=0 tx=0).stop 2023-07-13T22:12:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.206+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 msgr2=0x7f83bc195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.206+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f83b800e9f0 tx=0x7f83b800ed00 comp rx=0 tx=0).stop 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 shutdown_connections 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f83a80a11e0 0x7f83a80a3690 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc104ea0 0x7f83bc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195d60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 --2- 172.21.15.26:0/3568387536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83bc1007a0 0x7f83bc1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 >> 172.21.15.26:0/3568387536 conn(0x7f83bc0fb660 msgr2=0x7f83bc0fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 shutdown_connections 2023-07-13T22:12:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:38.207+0000 7f83c4629700 1 -- 172.21.15.26:0/3568387536 wait complete. 2023-07-13T22:12:38.653 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-13T22:12:38.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd dump --format=json 2023-07-13T22:12:38.722 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:38 smithi026 ceph-mon[95647]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2023-07-13T22:12:38.723 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:38 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3568387536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:38.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:38 smithi121 ceph-mon[129187]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2023-07-13T22:12:38.746 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:38 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3568387536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:38.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:38 smithi117 ceph-mon[129074]: pgmap v276: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 KiB/s wr, 21 op/s 2023-07-13T22:12:38.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:38 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3568387536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:39.390 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:40.627 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:40 smithi026 ceph-mon[95647]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-07-13T22:12:40.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:40 smithi121 ceph-mon[129187]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-07-13T22:12:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 -- 172.21.15.26:0/4250226144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf241007a0 msgr2=0x7fbf24104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4250226144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf241007a0 0x7fbf24104770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf180099c0 tx=0x7fbf18009cd0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 -- 172.21.15.26:0/4250226144 shutdown_connections 2023-07-13T22:12:40.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4250226144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf24104ea0 0x7fbf2410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4250226144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf240ffe00 0x7fbf241001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4250226144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf241007a0 0x7fbf24104770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 -- 172.21.15.26:0/4250226144 >> 172.21.15.26:0/4250226144 conn(0x7fbf240fb660 msgr2=0x7fbf240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 -- 172.21.15.26:0/4250226144 shutdown_connections 2023-07-13T22:12:40.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.872+0000 7fbf28a21700 1 -- 172.21.15.26:0/4250226144 wait complete. 2023-07-13T22:12:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 Processor -- start 2023-07-13T22:12:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 -- start start 2023-07-13T22:12:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:40.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 0x7fbf24072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 0x7fbf240732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241a41a0 con 0x7fbf24104ea0 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241a4310 con 0x7fbf241007a0 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.874+0000 7fbf28a21700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241a44d0 con 0x7fbf240ffe00 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 0x7fbf240732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:40.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf21d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 0x7fbf24072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf21d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 0x7fbf24072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39252/0 (socket says 172.21.15.26:39252) 2023-07-13T22:12:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf2259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60112/0 (socket says 172.21.15.26:60112) 2023-07-13T22:12:40.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf21d9b700 1 -- 172.21.15.26:0/4019251496 learned_addr learned my addr 172.21.15.26:0/4019251496 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 -- 172.21.15.26:0/4019251496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 msgr2=0x7fbf24072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 -- 172.21.15.26:0/4019251496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 msgr2=0x7fbf24072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 0x7fbf24072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 -- 172.21.15.26:0/4019251496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180095f0 con 0x7fbf24104ea0 2023-07-13T22:12:40.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf2259c700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf22d9d700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 0x7fbf240732d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00efc0 tx=0x7fbf1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.875+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c018070 con 0x7fbf24104ea0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.876+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf241a4750 con 0x7fbf24104ea0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.876+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1c00f040 con 0x7fbf24104ea0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.876+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c00cd40 con 0x7fbf24104ea0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.876+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf241a4ca0 con 0x7fbf24104ea0 2023-07-13T22:12:40.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.877+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2404e210 con 0x7fbf24104ea0 2023-07-13T22:12:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.879+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbf1c003a00 con 0x7fbf24104ea0 2023-07-13T22:12:40.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.880+0000 7fbf137fe700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 0x7fbf0c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.880+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbf1c0cc460 con 0x7fbf24104ea0 2023-07-13T22:12:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.880+0000 7fbf2259c700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 0x7fbf0c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:40.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.881+0000 7fbf2259c700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 0x7fbf0c0a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf14011490 tx=0x7fbf14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:40.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:40.884+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf1c095b90 con 0x7fbf24104ea0 2023-07-13T22:12:40.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:40 smithi117 ceph-mon[129074]: pgmap v277: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2023-07-13T22:12:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.093+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf241a4f60 con 0x7fbf24104ea0 2023-07-13T22:12:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.094+0000 7fbf137fe700 1 -- 172.21.15.26:0/4019251496 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22096 (secure 0 0 0) 0x7fbf1c0991b0 con 0x7fbf24104ea0 2023-07-13T22:12:41.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:12:41.087 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:19.954949+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:12:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 msgr2=0x7fbf0c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 0x7fbf0c0a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbf14011490 tx=0x7fbf14009260 comp rx=0 tx=0).stop 2023-07-13T22:12:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 msgr2=0x7fbf240732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 0x7fbf240732d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00efc0 tx=0x7fbf1c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:12:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 shutdown_connections 2023-07-13T22:12:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf0c0a1260 0x7fbf0c0a3710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf24104ea0 0x7fbf240732d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf240ffe00 0x7fbf24072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 --2- 172.21.15.26:0/4019251496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf241007a0 0x7fbf24072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 >> 172.21.15.26:0/4019251496 conn(0x7fbf240fb660 msgr2=0x7fbf240fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:41.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 shutdown_connections 2023-07-13T22:12:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:41.097+0000 7fbf28a21700 1 -- 172.21.15.26:0/4019251496 wait complete. 2023-07-13T22:12:41.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-13T22:12:41.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-13T22:12:41.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-13T22:12:41.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-13T22:12:41.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-13T22:12:41.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-13T22:12:41.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-13T22:12:41.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-13T22:12:41.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-13T22:12:41.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-13T22:12:41.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-13T22:12:41.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-13T22:12:41.736 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:41 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4019251496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:41.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:41 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4019251496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:41.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:41 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4019251496' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:12:42.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:42 smithi117 ceph-mon[129074]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-07-13T22:12:42.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:42 smithi026 ceph-mon[95647]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-07-13T22:12:42.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:42 smithi121 ceph-mon[129187]: pgmap v278: 97 pgs: 97 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2023-07-13T22:12:43.077 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:43.775 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:43.964 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:44.002 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:44.644 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:44 smithi026 ceph-mon[95647]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-07-13T22:12:44.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:44 smithi117 ceph-mon[129074]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-07-13T22:12:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:44.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:44 smithi121 ceph-mon[129187]: pgmap v279: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2023-07-13T22:12:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:46.087 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:46.570 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:46 smithi026 ceph-mon[95647]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-13T22:12:46.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:46 smithi117 ceph-mon[129074]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-13T22:12:46.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:46 smithi121 ceph-mon[129187]: pgmap v280: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-07-13T22:12:48.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:48 smithi026 ceph-mon[95647]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-13T22:12:48.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:48 smithi117 ceph-mon[129074]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-13T22:12:48.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:48 smithi121 ceph-mon[129187]: pgmap v281: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-07-13T22:12:49.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4205780711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8101700 msgr2=0x7fa6f8109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4205780711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8101700 0x7fa6f8109c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009990 tx=0x7fa6ec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4205780711 shutdown_connections 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4205780711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f810a140 0x7fa6f810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4205780711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8100df0 0x7fa6f81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4205780711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f8101700 0x7fa6f8109c00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4205780711 >> 172.21.15.26:0/4205780711 conn(0x7fa6f8075b70 msgr2=0x7fa6f8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4205780711 shutdown_connections 2023-07-13T22:12:49.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.603+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4205780711 wait complete. 2023-07-13T22:12:49.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.604+0000 7fa6fce3d700 1 Processor -- start 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 -- start start 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 0x7fa6f81a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8101700 0x7fa6f81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f810a140 0x7fa6f8108840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8108f90 con 0x7fa6f810a140 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8109100 con 0x7fa6f8101700 2023-07-13T22:12:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.605+0000 7fa6fce3d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81092c0 con 0x7fa6f8100df0 2023-07-13T22:12:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 0x7fa6f81a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 0x7fa6f81a0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50546/0 (socket says 172.21.15.26:50546) 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 -- 172.21.15.26:0/4043134244 learned_addr learned my addr 172.21.15.26:0/4043134244 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f6d9d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f810a140 0x7fa6f8108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f5d9b700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8101700 0x7fa6f81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 -- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8101700 msgr2=0x7fa6f81a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8101700 0x7fa6f81a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 -- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f810a140 msgr2=0x7fa6f8108840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6f810a140 0x7fa6f8108840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec0095f0 con 0x7fa6f8100df0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6f659c700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 0x7fa6f81a0f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800e9a0 tx=0x7fa6e800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.606+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8009790 con 0x7fa6f8100df0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.607+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e8010430 con 0x7fa6f8100df0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.607+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8109540 con 0x7fa6f8100df0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.607+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8018680 con 0x7fa6f8100df0 2023-07-13T22:12:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.607+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f81a9040 con 0x7fa6f8100df0 2023-07-13T22:12:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.607+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6f810fd70 con 0x7fa6f8100df0 2023-07-13T22:12:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.610+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa6e80105a0 con 0x7fa6f8100df0 2023-07-13T22:12:49.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.611+0000 7fa6e77fe700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa6e00a11e0 0x7fa6e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.611+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa6e8014070 con 0x7fa6f8100df0 2023-07-13T22:12:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.611+0000 7fa6f5d9b700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa6e00a11e0 0x7fa6e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.612+0000 7fa6e77fe700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] conn(0x7fa6e00aa9b0 0x7fa6e00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.612+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6e00ad470 con 0x7fa6e00aa9b0 2023-07-13T22:12:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.612+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6e80cd230 con 0x7fa6f8100df0 2023-07-13T22:12:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.612+0000 7fa6f6d9d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] conn(0x7fa6e00aa9b0 0x7fa6e00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.612+0000 7fa6f5d9b700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa6e00a11e0 0x7fa6e00a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b2e0 tx=0x7fa6ec005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.613+0000 7fa6f6d9d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] conn(0x7fa6e00aa9b0 0x7fa6e00acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.613+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== osd.4 v2:172.21.15.117:6808/4231906293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa6e00ad470 con 0x7fa6e00aa9b0 2023-07-13T22:12:49.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.630+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 --> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6f804e210 con 0x7fa6e00aa9b0 2023-07-13T22:12:49.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.631+0000 7fa6e77fe700 1 -- 172.21.15.26:0/4043134244 <== osd.4 v2:172.21.15.117:6808/4231906293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa6f804e210 con 0x7fa6e00aa9b0 2023-07-13T22:12:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.631+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] conn(0x7fa6e00aa9b0 msgr2=0x7fa6e00acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.631+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] conn(0x7fa6e00aa9b0 0x7fa6e00acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa6e00a11e0 msgr2=0x7fa6e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa6e00a11e0 0x7fa6e00a3690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b2e0 tx=0x7fa6ec005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 msgr2=0x7fa6f81a0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 --2- 172.21.15.26:0/4043134244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6f8100df0 0x7fa6f81a0f40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800e9a0 tx=0x7fa6e800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6f6d9d700 1 -- 172.21.15.26:0/4043134244 reap_dead start 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 shutdown_connections 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 >> 172.21.15.26:0/4043134244 conn(0x7fa6f8075b70 msgr2=0x7fa6f80fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 shutdown_connections 2023-07-13T22:12:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.632+0000 7fa6fce3d700 1 -- 172.21.15.26:0/4043134244 wait complete. 2023-07-13T22:12:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 -- 172.21.15.26:0/2875447628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590100df0 msgr2=0x7fe5901011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 --2- 172.21.15.26:0/2875447628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590100df0 0x7fe5901011c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe580009990 tx=0x7fe580009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 -- 172.21.15.26:0/2875447628 shutdown_connections 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 --2- 172.21.15.26:0/2875447628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590100df0 0x7fe5901011c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 --2- 172.21.15.26:0/2875447628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe59010a140 0x7fe59010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 --2- 172.21.15.26:0/2875447628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590101700 0x7fe590109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 -- 172.21.15.26:0/2875447628 >> 172.21.15.26:0/2875447628 conn(0x7fe590075b70 msgr2=0x7fe590075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 -- 172.21.15.26:0/2875447628 shutdown_connections 2023-07-13T22:12:49.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.881+0000 7fe595799700 1 -- 172.21.15.26:0/2875447628 wait complete. 2023-07-13T22:12:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.882+0000 7fe595799700 1 Processor -- start 2023-07-13T22:12:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.882+0000 7fe595799700 1 -- start start 2023-07-13T22:12:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 0x7fe59019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590101700 0x7fe59019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe59010a140 0x7fe590195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590196150 con 0x7fe59010a140 2023-07-13T22:12:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901962c0 con 0x7fe590100df0 2023-07-13T22:12:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe595799700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe590196480 con 0x7fe590101700 2023-07-13T22:12:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 0x7fe59019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe59010a140 0x7fe590195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 0x7fe59019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59220/0 (socket says 172.21.15.26:59220) 2023-07-13T22:12:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58effd700 1 -- 172.21.15.26:0/210937665 learned_addr learned my addr 172.21.15.26:0/210937665 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:49.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590101700 0x7fe59019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.883+0000 7fe58effd700 1 -- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590101700 msgr2=0x7fe59019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe58effd700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590101700 0x7fe59019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe58effd700 1 -- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe59010a140 msgr2=0x7fe590195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe58effd700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe59010a140 0x7fe590195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe58effd700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800095f0 con 0x7fe590100df0 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe58effd700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 0x7fe59019ba60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe580005910 tx=0x7fe580005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58001d070 con 0x7fe590100df0 2023-07-13T22:12:49.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590196700 con 0x7fe590100df0 2023-07-13T22:12:49.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe580022470 con 0x7fe590100df0 2023-07-13T22:12:49.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580017650 con 0x7fe590100df0 2023-07-13T22:12:49.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.884+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5901a2a50 con 0x7fe590100df0 2023-07-13T22:12:49.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.885+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe59010fd30 con 0x7fe590100df0 2023-07-13T22:12:49.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.887+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe5800225e0 con 0x7fe590100df0 2023-07-13T22:12:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe577fff700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1260 0x7fe5780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe5800ce730 con 0x7fe590100df0 2023-07-13T22:12:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe577fff700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] conn(0x7fe5780aaa30 0x7fe5780ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1260 0x7fe5780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5780ad4f0 con 0x7fe5780aaa30 2023-07-13T22:12:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.889+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5800ceaa0 con 0x7fe590100df0 2023-07-13T22:12:49.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.890+0000 7fe58f7fe700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] conn(0x7fe5780aaa30 0x7fe5780ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.890+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1260 0x7fe5780a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe59010c900 tx=0x7fe584009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.890+0000 7fe58f7fe700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] conn(0x7fe5780aaa30 0x7fe5780ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.891+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== osd.8 v2:172.21.15.121:6816/4171528294 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe5780ad4f0 con 0x7fe5780aaa30 2023-07-13T22:12:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 --> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe590060b90 con 0x7fe5780aaa30 2023-07-13T22:12:49.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 -- 172.21.15.26:0/94828542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 msgr2=0x7f2f30109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 --2- 172.21.15.26:0/94828542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f30109c00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009990 tx=0x7f2f20009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 -- 172.21.15.26:0/94828542 shutdown_connections 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 --2- 172.21.15.26:0/94828542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f30100df0 0x7f2f301011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 --2- 172.21.15.26:0/94828542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f3010a140 0x7f2f3010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 --2- 172.21.15.26:0/94828542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f30109c00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7f2f38786700 1 -- 172.21.15.26:0/94828542 >> 172.21.15.26:0/94828542 conn(0x7f2f30075b70 msgr2=0x7f2f30075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.908+0000 7fe577fff700 1 -- 172.21.15.26:0/210937665 <== osd.8 v2:172.21.15.121:6816/4171528294 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe590060b90 con 0x7fe5780aaa30 2023-07-13T22:12:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7f2f38786700 1 -- 172.21.15.26:0/94828542 shutdown_connections 2023-07-13T22:12:49.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7f2f38786700 1 -- 172.21.15.26:0/94828542 wait complete. 2023-07-13T22:12:49.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] conn(0x7fe5780aaa30 msgr2=0x7fe5780ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] conn(0x7fe5780aaa30 0x7fe5780ace40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1260 msgr2=0x7fe5780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1260 0x7fe5780a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe59010c900 tx=0x7fe584009260 comp rx=0 tx=0).stop 2023-07-13T22:12:49.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 msgr2=0x7fe59019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 --2- 172.21.15.26:0/210937665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590100df0 0x7fe59019ba60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe580005910 tx=0x7fe580005720 comp rx=0 tx=0).stop 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe58f7fe700 1 -- 172.21.15.26:0/210937665 reap_dead start 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 shutdown_connections 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 >> 172.21.15.26:0/210937665 conn(0x7fe590075b70 msgr2=0x7fe590101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.909+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 shutdown_connections 2023-07-13T22:12:49.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.910+0000 7fe595799700 1 -- 172.21.15.26:0/210937665 wait complete. 2023-07-13T22:12:49.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.910+0000 7f2f38786700 1 Processor -- start 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.910+0000 7f2f38786700 1 -- start start 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.910+0000 7f2f38786700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 0x7f2f3019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.910+0000 7f2f38786700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f3019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f38786700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3010a140 0x7f2f30195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f38786700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301960f0 con 0x7f2f3010a140 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f38786700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30196260 con 0x7f2f30100df0 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f38786700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30196420 con 0x7f2f30101700 2023-07-13T22:12:49.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 0x7f2f3019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 0x7f2f3019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59230/0 (socket says 172.21.15.26:59230) 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 -- 172.21.15.26:0/2643009799 learned_addr learned my addr 172.21.15.26:0/2643009799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36d23700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3010a140 0x7f2f30195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f35d21700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f3019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 -- 172.21.15.26:0/2643009799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 msgr2=0x7f2f3019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f3019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 -- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3010a140 msgr2=0x7f2f30195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3010a140 0x7f2f30195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36522700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200095f0 con 0x7f2f30100df0 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f35d21700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f30101700 0x7f2f3019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.911+0000 7f2f36d23700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3010a140 0x7f2f30195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f36522700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 0x7f2f3019ba00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9a0 tx=0x7f2f2c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c009790 con 0x7f2f30100df0 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f30196700 con 0x7f2f30100df0 2023-07-13T22:12:49.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f2c00cbd0 con 0x7f2f30100df0 2023-07-13T22:12:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c018680 con 0x7f2f30100df0 2023-07-13T22:12:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.912+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f301a29f0 con 0x7f2f30100df0 2023-07-13T22:12:49.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.913+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f3010fd80 con 0x7f2f30100df0 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.915+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2f2c00cd40 con 0x7f2f30100df0 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f277fe700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f1c0a11e0 0x7f2f1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2f2c014070 con 0x7f2f30100df0 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f35d21700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f1c0a11e0 0x7f2f1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f277fe700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] conn(0x7f2f1c0aa9b0 0x7f2f1c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f1c0ad470 con 0x7f2f1c0aa9b0 2023-07-13T22:12:49.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.917+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2c0cd390 con 0x7f2f30100df0 2023-07-13T22:12:49.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.918+0000 7f2f36d23700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] conn(0x7f2f1c0aa9b0 0x7f2f1c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:49.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.918+0000 7f2f35d21700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f1c0a11e0 0x7f2f1c0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f300fcef0 tx=0x7f2f20005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.918+0000 7f2f36d23700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] conn(0x7f2f1c0aa9b0 0x7f2f1c0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:49.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.919+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== osd.3 v2:172.21.15.26:6810/3949990107 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2f1c0ad470 con 0x7f2f1c0aa9b0 2023-07-13T22:12:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.937+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 --> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f300fdd10 con 0x7f2f1c0aa9b0 2023-07-13T22:12:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.937+0000 7f2f277fe700 1 -- 172.21.15.26:0/2643009799 <== osd.3 v2:172.21.15.26:6810/3949990107 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2f300fdd10 con 0x7f2f1c0aa9b0 2023-07-13T22:12:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] conn(0x7f2f1c0aa9b0 msgr2=0x7f2f1c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] conn(0x7f2f1c0aa9b0 0x7f2f1c0acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f1c0a11e0 msgr2=0x7f2f1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f1c0a11e0 0x7f2f1c0a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f300fcef0 tx=0x7f2f20005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 msgr2=0x7f2f3019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 --2- 172.21.15.26:0/2643009799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f30100df0 0x7f2f3019ba00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9a0 tx=0x7f2f2c00ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f36d23700 1 -- 172.21.15.26:0/2643009799 reap_dead start 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 shutdown_connections 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 >> 172.21.15.26:0/2643009799 conn(0x7f2f30075b70 msgr2=0x7f2f300fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 shutdown_connections 2023-07-13T22:12:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:49.938+0000 7f2f38786700 1 -- 172.21.15.26:0/2643009799 wait complete. 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 -- 172.21.15.26:0/3432776095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 msgr2=0x7f448410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 --2- 172.21.15.26:0/3432776095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448410ca20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b0d0 tx=0x7f447c00b3e0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 -- 172.21.15.26:0/3432776095 shutdown_connections 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 --2- 172.21.15.26:0/3432776095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448410ca20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 --2- 172.21.15.26:0/3432776095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44840ffe00 0x7f44841001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 --2- 172.21.15.26:0/3432776095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 -- 172.21.15.26:0/3432776095 >> 172.21.15.26:0/3432776095 conn(0x7f44840fb660 msgr2=0x7f44840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 -- 172.21.15.26:0/3432776095 shutdown_connections 2023-07-13T22:12:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.005+0000 7f4488c5b700 1 -- 172.21.15.26:0/3432776095 wait complete. 2023-07-13T22:12:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.006+0000 7f4488c5b700 1 Processor -- start 2023-07-13T22:12:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.006+0000 7f4488c5b700 1 -- start start 2023-07-13T22:12:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44840ffe00 0x7f4484197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419cf00 con 0x7f4484104ea0 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419d070 con 0x7f44841007a0 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4488c5b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448419d230 con 0x7f44840ffe00 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4481d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59248/0 (socket says 172.21.15.26:59248) 2023-07-13T22:12:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f448259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44840ffe00 0x7f4484197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4481d9b700 1 -- 172.21.15.26:0/1550804654 learned_addr learned my addr 172.21.15.26:0/1550804654 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51104/0 (socket says 172.21.15.26:51104) 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 -- 172.21.15.26:0/1550804654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44840ffe00 msgr2=0x7f4484197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44840ffe00 0x7f4484197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 -- 172.21.15.26:0/1550804654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 msgr2=0x7f4484198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.007+0000 7f4482d9d700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009d00 con 0x7f4484104ea0 2023-07-13T22:12:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f4481d9b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44841007a0 0x7f4484198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f4482d9d700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448419c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b770 tx=0x7f447c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00e040 con 0x7f4484104ea0 2023-07-13T22:12:50.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447c01d930 con 0x7f4484104ea0 2023-07-13T22:12:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c01b820 con 0x7f4484104ea0 2023-07-13T22:12:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448419d4b0 con 0x7f4484104ea0 2023-07-13T22:12:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.008+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44841a4d70 con 0x7f4484104ea0 2023-07-13T22:12:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.009+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f448419d790 con 0x7f4484104ea0 2023-07-13T22:12:50.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.011+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f447c01d470 con 0x7f4484104ea0 2023-07-13T22:12:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f44737fe700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f446c0a11e0 0x7f446c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f447c0ce4d0 con 0x7f4484104ea0 2023-07-13T22:12:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f448259c700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f446c0a11e0 0x7f446c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f44737fe700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] conn(0x7f446c0aa9b0 0x7f446c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f446c0ad470 con 0x7f446c0aa9b0 2023-07-13T22:12:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f447c0ce840 con 0x7f4484104ea0 2023-07-13T22:12:50.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.013+0000 7f4481d9b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] conn(0x7f446c0aa9b0 0x7f446c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.014+0000 7f448259c700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f446c0a11e0 0x7f446c0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4474005950 tx=0x7f447400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.014+0000 7f4481d9b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] conn(0x7f446c0aa9b0 0x7f446c0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.015+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== osd.0 v2:172.21.15.26:6802/1617573741 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f446c0ad470 con 0x7f446c0aa9b0 2023-07-13T22:12:50.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.032+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 --> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f448410fd20 con 0x7f446c0aa9b0 2023-07-13T22:12:50.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.033+0000 7f44737fe700 1 -- 172.21.15.26:0/1550804654 <== osd.0 v2:172.21.15.26:6802/1617573741 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f448410fd20 con 0x7f446c0aa9b0 2023-07-13T22:12:50.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] conn(0x7f446c0aa9b0 msgr2=0x7f446c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] conn(0x7f446c0aa9b0 0x7f446c0acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f446c0a11e0 msgr2=0x7f446c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f446c0a11e0 0x7f446c0a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4474005950 tx=0x7f447400b410 comp rx=0 tx=0).stop 2023-07-13T22:12:50.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 msgr2=0x7f448419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 --2- 172.21.15.26:0/1550804654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4484104ea0 0x7f448419c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b770 tx=0x7f447c007870 comp rx=0 tx=0).stop 2023-07-13T22:12:50.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4482d9d700 1 -- 172.21.15.26:0/1550804654 reap_dead start 2023-07-13T22:12:50.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 shutdown_connections 2023-07-13T22:12:50.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.034+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 >> 172.21.15.26:0/1550804654 conn(0x7f44840fb660 msgr2=0x7f44840fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:50.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.035+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 shutdown_connections 2023-07-13T22:12:50.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.035+0000 7f4488c5b700 1 -- 172.21.15.26:0/1550804654 wait complete. 2023-07-13T22:12:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/1932944603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b70104e90 msgr2=0x7f0b7010ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- 172.21.15.26:0/1932944603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b70104e90 0x7f0b7010ca10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800b0a0 tx=0x7f0b6800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/1932944603 shutdown_connections 2023-07-13T22:12:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- 172.21.15.26:0/1932944603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b70104e90 0x7f0b7010ca10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- 172.21.15.26:0/1932944603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b700ffdf0 0x7f0b701001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- 172.21.15.26:0/1932944603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b70100790 0x7f0b70104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/1932944603 >> 172.21.15.26:0/1932944603 conn(0x7f0b700fb670 msgr2=0x7f0b700fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/1932944603 shutdown_connections 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/1932944603 wait complete. 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 Processor -- start 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- start start 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 0x7f0b70072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b70100790 0x7f0b70072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b70104e90 0x7f0b701a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70073640 con 0x7f0b70100790 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701a4530 con 0x7f0b700ffdf0 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701a46a0 con 0x7f0b70104e90 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b70100790 0x7f0b70072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 0x7f0b70072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 0x7f0b70072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59268/0 (socket says 172.21.15.26:59268) 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 -- 172.21.15.26:0/2618992942 learned_addr learned my addr 172.21.15.26:0/2618992942 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b74d2a700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b70104e90 0x7f0b701a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 -- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b70104e90 msgr2=0x7f0b701a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b70104e90 0x7f0b701a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 -- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b70100790 msgr2=0x7f0b70072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b70100790 0x7f0b70072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009d00 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6ffff700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 0x7f0b70072810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400e9a0 tx=0x7f0b6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64009790 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b64010430 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b701a4920 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64018680 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b701a4e20 con 0x7f0b700ffdf0 2023-07-13T22:12:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.205+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b7010fcc0 con 0x7f0b700ffdf0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.214+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0b640105a0 con 0x7f0b700ffdf0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6d7fa700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b580a11e0 0x7f0b580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0b64014070 con 0x7f0b700ffdf0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6f7fe700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b580a11e0 0x7f0b580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6d7fa700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] conn(0x7f0b580aa9b0 0x7f0b580acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b580ad470 con 0x7f0b580aa9b0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.216+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b640cd230 con 0x7f0b700ffdf0 2023-07-13T22:12:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.217+0000 7f0b6f7fe700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b580a11e0 0x7f0b580a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b70074000 tx=0x7f0b60009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.217+0000 7f0b74d2a700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] conn(0x7f0b580aa9b0 0x7f0b580acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.217+0000 7f0b74d2a700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] conn(0x7f0b580aa9b0 0x7f0b580acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.218+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== osd.2 v2:172.21.15.121:6800/1910431229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b580ad470 con 0x7f0b580aa9b0 2023-07-13T22:12:50.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.236+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 --> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0b7004e210 con 0x7f0b580aa9b0 2023-07-13T22:12:50.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.236+0000 7f0b6d7fa700 1 -- 172.21.15.26:0/2618992942 <== osd.2 v2:172.21.15.121:6800/1910431229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0b7004e210 con 0x7f0b580aa9b0 2023-07-13T22:12:50.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] conn(0x7f0b580aa9b0 msgr2=0x7f0b580acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] conn(0x7f0b580aa9b0 0x7f0b580acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b580a11e0 msgr2=0x7f0b580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b580a11e0 0x7f0b580a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b70074000 tx=0x7f0b60009260 comp rx=0 tx=0).stop 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 msgr2=0x7f0b70072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 --2- 172.21.15.26:0/2618992942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b700ffdf0 0x7f0b70072810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400e9a0 tx=0x7f0b6400ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b74d2a700 1 -- 172.21.15.26:0/2618992942 reap_dead start 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 shutdown_connections 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 >> 172.21.15.26:0/2618992942 conn(0x7f0b700fb670 msgr2=0x7f0b700fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:50.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 shutdown_connections 2023-07-13T22:12:50.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.237+0000 7f0b7678d700 1 -- 172.21.15.26:0/2618992942 wait complete. 2023-07-13T22:12:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/1525587991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd40ffe00 msgr2=0x7f7cd41001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/1525587991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd40ffe00 0x7f7cd41001d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00099c0 tx=0x7f7cd0009cd0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/1525587991 shutdown_connections 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/1525587991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd40ffe00 0x7f7cd41001d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/1525587991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd4104ea0 0x7f7cd410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/1525587991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd41007a0 0x7f7cd4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/1525587991 >> 172.21.15.26:0/1525587991 conn(0x7f7cd40fb660 msgr2=0x7f7cd40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/1525587991 shutdown_connections 2023-07-13T22:12:50.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.906+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/1525587991 wait complete. 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.907+0000 7f7cdbc3f700 1 Processor -- start 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.907+0000 7f7cdbc3f700 1 -- start start 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd40ffe00 0x7f7cd4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd41007a0 0x7f7cd4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 0x7f7cd40732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a4220 con 0x7f7cd4104ea0 2023-07-13T22:12:50.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a4390 con 0x7f7cd40ffe00 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cdbc3f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41a4550 con 0x7f7cd41007a0 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cda1dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 0x7f7cd40732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cd91da700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd41007a0 0x7f7cd4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cda1dc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 0x7f7cd40732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51056/0 (socket says 172.21.15.26:51056) 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.908+0000 7f7cda1dc700 1 -- 172.21.15.26:0/353840690 learned_addr learned my addr 172.21.15.26:0/353840690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cd99db700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd40ffe00 0x7f7cd4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 -- 172.21.15.26:0/353840690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd41007a0 msgr2=0x7f7cd4072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd41007a0 0x7f7cd4072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 -- 172.21.15.26:0/353840690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd40ffe00 msgr2=0x7f7cd4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cd40ffe00 0x7f7cd4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00095f0 con 0x7f7cd4104ea0 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cd91da700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cd41007a0 0x7f7cd4072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cda1dc700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 0x7f7cd40732d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00ef30 tx=0x7f7ccc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc018070 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ccc00f040 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00cd00 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.909+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd41a47d0 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.910+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd41a4c80 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.910+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cd410fd60 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.912+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7ccc003a00 con 0x7f7cd4104ea0 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7ccaffd700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7cc00a11e0 0x7f7cc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ccc0ccde0 con 0x7f7cd4104ea0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7cd99db700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7cc00a11e0 0x7f7cc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7ccaffd700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] conn(0x7f7cc00aa9b0 0x7f7cc00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7cc00ad470 con 0x7f7cc00aa9b0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ccc0cd150 con 0x7f7cd4104ea0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.914+0000 7f7cd91da700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] conn(0x7f7cc00aa9b0 0x7f7cc00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.915+0000 7f7cd99db700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7cc00a11e0 0x7f7cc00a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009990 tx=0x7f7cd0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.915+0000 7f7cd91da700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] conn(0x7f7cc00aa9b0 0x7f7cc00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.916+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== osd.6 v2:172.21.15.26:6818/38495361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7cc00ad470 con 0x7f7cc00aa9b0 2023-07-13T22:12:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.932+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 --> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7cd404e210 con 0x7f7cc00aa9b0 2023-07-13T22:12:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7ccaffd700 1 -- 172.21.15.26:0/353840690 <== osd.6 v2:172.21.15.26:6818/38495361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7cd404e210 con 0x7f7cc00aa9b0 2023-07-13T22:12:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] conn(0x7f7cc00aa9b0 msgr2=0x7f7cc00acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] conn(0x7f7cc00aa9b0 0x7f7cc00acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7cc00a11e0 msgr2=0x7f7cc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7cc00a11e0 0x7f7cc00a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009990 tx=0x7f7cd0005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 msgr2=0x7f7cd40732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 --2- 172.21.15.26:0/353840690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd4104ea0 0x7f7cd40732d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00ef30 tx=0x7f7ccc00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:12:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cda1dc700 1 -- 172.21.15.26:0/353840690 reap_dead start 2023-07-13T22:12:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 shutdown_connections 2023-07-13T22:12:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.933+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 >> 172.21.15.26:0/353840690 conn(0x7f7cd40fb660 msgr2=0x7f7cd40fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.934+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 shutdown_connections 2023-07-13T22:12:50.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:48.934+0000 7f7cdbc3f700 1 -- 172.21.15.26:0/353840690 wait complete. 2023-07-13T22:12:50.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:50 smithi117 ceph-mon[129074]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-13T22:12:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:50 smithi026 ceph-mon[95647]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-13T22:12:50.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:50 smithi121 ceph-mon[129187]: pgmap v282: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 341 B/s wr, 9 op/s 2023-07-13T22:12:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 -- 172.21.15.26:0/1751278175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de410c530 msgr2=0x7f4de410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 --2- 172.21.15.26:0/1751278175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de410c530 0x7f4de410e910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009f90 tx=0x7f4ddc00b330 comp rx=0 tx=0).stop 2023-07-13T22:12:51.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 -- 172.21.15.26:0/1751278175 shutdown_connections 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 --2- 172.21.15.26:0/1751278175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de410c530 0x7f4de410e910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 --2- 172.21.15.26:0/1751278175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de40fe980 0x7f4de40fed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 --2- 172.21.15.26:0/1751278175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40ff290 0x7f4de410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 -- 172.21.15.26:0/1751278175 >> 172.21.15.26:0/1751278175 conn(0x7f4de4075b70 msgr2=0x7f4de4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 -- 172.21.15.26:0/1751278175 shutdown_connections 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.313+0000 7f4deadb8700 1 -- 172.21.15.26:0/1751278175 wait complete. 2023-07-13T22:12:51.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.314+0000 7f4deadb8700 1 Processor -- start 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 -- start start 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40fe980 0x7f4de4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 0x7f4de41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de410c530 0x7f4de419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de419cf70 con 0x7f4de40ff290 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de419d0e0 con 0x7f4de410c530 2023-07-13T22:12:51.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4deadb8700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de419d2a0 con 0x7f4de40fe980 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4de3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 0x7f4de41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4de8b54700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40fe980 0x7f4de4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4de3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 0x7f4de41984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51158/0 (socket says 172.21.15.26:51158) 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4de8b54700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40fe980 0x7f4de4197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50640/0 (socket says 172.21.15.26:50640) 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.315+0000 7f4de3fff700 1 -- 172.21.15.26:0/2925616284 learned_addr learned my addr 172.21.15.26:0/2925616284 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de9355700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de410c530 0x7f4de419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 -- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40fe980 msgr2=0x7f4de4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4de40fe980 0x7f4de4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 -- 172.21.15.26:0/2925616284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de410c530 msgr2=0x7f4de419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de410c530 0x7f4de419c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009c40 con 0x7f4de40ff290 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de9355700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4de410c530 0x7f4de419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de3fff700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 0x7f4de41984f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000e9c0 tx=0x7f4dd000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000cbd0 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de419d580 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd000eeb0 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0005580 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.316+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de41a4de0 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.317+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4de4111f50 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.319+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4dd0004750 con 0x7f4de40ff290 2023-07-13T22:12:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de1ffb700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dd40a1260 0x7f4dd40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4dd0014070 con 0x7f4de40ff290 2023-07-13T22:12:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de8b54700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dd40a1260 0x7f4dd40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de1ffb700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] conn(0x7f4dd40aaa30 0x7f4dd40ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4dd40ad4f0 con 0x7f4dd40aaa30 2023-07-13T22:12:51.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.321+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd00cd4a0 con 0x7f4de40ff290 2023-07-13T22:12:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.322+0000 7f4de9355700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] conn(0x7f4dd40aaa30 0x7f4dd40ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.322+0000 7f4de8b54700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dd40a1260 0x7f4dd40a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009c60 tx=0x7f4dd8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.322+0000 7f4de9355700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] conn(0x7f4dd40aaa30 0x7f4dd40ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.323+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== osd.5 v2:172.21.15.121:6808/37316284 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4dd40ad4f0 con 0x7f4dd40aaa30 2023-07-13T22:12:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.340+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 --> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4de404e210 con 0x7f4dd40aaa30 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4de1ffb700 1 -- 172.21.15.26:0/2925616284 <== osd.5 v2:172.21.15.121:6808/37316284 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4de404e210 con 0x7f4dd40aaa30 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] conn(0x7f4dd40aaa30 msgr2=0x7f4dd40ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] conn(0x7f4dd40aaa30 0x7f4dd40ace40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dd40a1260 msgr2=0x7f4dd40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4dd40a1260 0x7f4dd40a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009c60 tx=0x7f4dd8009260 comp rx=0 tx=0).stop 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 msgr2=0x7f4de41984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 --2- 172.21.15.26:0/2925616284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de40ff290 0x7f4de41984f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000e9c0 tx=0x7f4dd000ecd0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4de9355700 1 -- 172.21.15.26:0/2925616284 reap_dead start 2023-07-13T22:12:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 shutdown_connections 2023-07-13T22:12:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 >> 172.21.15.26:0/2925616284 conn(0x7f4de4075b70 msgr2=0x7f4de40fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 shutdown_connections 2023-07-13T22:12:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:51.341+0000 7f4deadb8700 1 -- 172.21.15.26:0/2925616284 wait complete. 2023-07-13T22:12:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 -- 172.21.15.26:0/1024896227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91580ffe00 msgr2=0x7f91581001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 --2- 172.21.15.26:0/1024896227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91580ffe00 0x7f91581001d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f91540099c0 tx=0x7f9154009cd0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 -- 172.21.15.26:0/1024896227 shutdown_connections 2023-07-13T22:12:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 --2- 172.21.15.26:0/1024896227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91580ffe00 0x7f91581001d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 --2- 172.21.15.26:0/1024896227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9158104ea0 0x7f915810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 --2- 172.21.15.26:0/1024896227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91581007a0 0x7f9158104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 -- 172.21.15.26:0/1024896227 >> 172.21.15.26:0/1024896227 conn(0x7f91580fb660 msgr2=0x7f91580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.664+0000 7f915fe8b700 1 -- 172.21.15.26:0/1024896227 shutdown_connections 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.665+0000 7f915fe8b700 1 -- 172.21.15.26:0/1024896227 wait complete. 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 Processor -- start 2023-07-13T22:12:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 -- start start 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91580ffe00 0x7f9158197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 0x7f91581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9158104ea0 0x7f915819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915819ce70 con 0x7f91581007a0 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915819cfe0 con 0x7f91580ffe00 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.666+0000 7f915fe8b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915819d1a0 con 0x7f9158104ea0 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 0x7f91581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 0x7f91581983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51136/0 (socket says 172.21.15.26:51136) 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 -- 172.21.15.26:0/707762386 learned_addr learned my addr 172.21.15.26:0/707762386 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915dc27700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91580ffe00 0x7f9158197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915e428700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9158104ea0 0x7f915819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 -- 172.21.15.26:0/707762386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9158104ea0 msgr2=0x7f915819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9158104ea0 0x7f915819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 -- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91580ffe00 msgr2=0x7f9158197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91580ffe00 0x7f9158197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540095f0 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f915d426700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 0x7f91581983f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f914800e970 tx=0x7f914800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.667+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148009790 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.668+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f914800c7e0 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.668+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915819d480 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.668+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480054e0 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.668+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91581a4ce0 con 0x7f91581007a0 2023-07-13T22:12:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.668+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f915810fdb0 con 0x7f91581007a0 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.671+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f914800c950 con 0x7f91581007a0 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f914effd700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91440a11e0 0x7f91440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9148014070 con 0x7f91581007a0 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f915dc27700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91440a11e0 0x7f91440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f914effd700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] conn(0x7f91440aa9b0 0x7f91440acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91440ad470 con 0x7f91440aa9b0 2023-07-13T22:12:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.672+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91480cd210 con 0x7f91581007a0 2023-07-13T22:12:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.673+0000 7f915e428700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] conn(0x7f91440aa9b0 0x7f91440acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.673+0000 7f915dc27700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91440a11e0 0x7f91440a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9154009990 tx=0x7f9154005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.673+0000 7f915e428700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] conn(0x7f91440aa9b0 0x7f91440acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.674+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== osd.10 v2:172.21.15.117:6824/1101018481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f91440ad470 con 0x7f91440aa9b0 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.694+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 --> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f915804e210 con 0x7f91440aa9b0 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.694+0000 7f914effd700 1 -- 172.21.15.26:0/707762386 <== osd.10 v2:172.21.15.117:6824/1101018481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f915804e210 con 0x7f91440aa9b0 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] conn(0x7f91440aa9b0 msgr2=0x7f91440acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] conn(0x7f91440aa9b0 0x7f91440acdc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91440a11e0 msgr2=0x7f91440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f91440a11e0 0x7f91440a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9154009990 tx=0x7f9154005fb0 comp rx=0 tx=0).stop 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 msgr2=0x7f91581983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 --2- 172.21.15.26:0/707762386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91581007a0 0x7f91581983f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f914800e970 tx=0x7f914800ec80 comp rx=0 tx=0).stop 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915e428700 1 -- 172.21.15.26:0/707762386 reap_dead start 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 shutdown_connections 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 >> 172.21.15.26:0/707762386 conn(0x7f91580fb660 msgr2=0x7f91580fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 shutdown_connections 2023-07-13T22:12:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:50.695+0000 7f915fe8b700 1 -- 172.21.15.26:0/707762386 wait complete. 2023-07-13T22:12:51.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-13T22:12:51.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2023-07-13T22:12:51.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-07-13T22:12:51.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-13T22:12:51.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-13T22:12:51.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2023-07-13T22:12:51.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-13T22:12:51.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-07-13T22:12:51.797 INFO:teuthology.orchestra.run.smithi026.stdout:141733920818 2023-07-13T22:12:51.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-13T22:12:51.859 INFO:teuthology.orchestra.run.smithi026.stdout:249108103200 2023-07-13T22:12:51.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-13T22:12:51.861 INFO:teuthology.orchestra.run.smithi026.stdout:107374182453 2023-07-13T22:12:51.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-13T22:12:51.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-13T22:12:51.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2023-07-13T22:12:51.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-07-13T22:12:51.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-13T22:12:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-13T22:12:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2023-07-13T22:12:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-13T22:12:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-07-13T22:12:51.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2023-07-13T22:12:51.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-07-13T22:12:52.094 INFO:teuthology.orchestra.run.smithi026.stdout:197568495655 2023-07-13T22:12:52.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-13T22:12:52.149 INFO:teuthology.orchestra.run.smithi026.stdout:34359738435 2023-07-13T22:12:52.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-13T22:12:52.411 INFO:teuthology.orchestra.run.smithi026.stdout:304942678039 2023-07-13T22:12:52.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: pgmap v283: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-13T22:12:52.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-13T22:12:52.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2023-07-13T22:12:52.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-13T22:12:52.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-07-13T22:12:52.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:52 smithi026 ceph-mon[95647]: osdmap e84: 12 total, 12 up, 12 in 2023-07-13T22:12:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 -- 172.21.15.26:0/1713296668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 msgr2=0x7fda3410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/1713296668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3410ca20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b0d0 tx=0x7fda2c00b3e0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 -- 172.21.15.26:0/1713296668 shutdown_connections 2023-07-13T22:12:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/1713296668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3410ca20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/1713296668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda340ffe00 0x7fda341001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/1713296668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda341007a0 0x7fda34104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.729+0000 7fda3c1d6700 1 -- 172.21.15.26:0/1713296668 >> 172.21.15.26:0/1713296668 conn(0x7fda340fb660 msgr2=0x7fda340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.730+0000 7fda3c1d6700 1 -- 172.21.15.26:0/1713296668 shutdown_connections 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.730+0000 7fda3c1d6700 1 -- 172.21.15.26:0/1713296668 wait complete. 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 Processor -- start 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 -- start start 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 0x7fda34197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda341007a0 0x7fda341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419cf50 con 0x7fda34104ea0 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419d0c0 con 0x7fda340ffe00 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.731+0000 7fda3c1d6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3419d280 con 0x7fda341007a0 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 0x7fda34197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda3a773700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 0x7fda34197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59316/0 (socket says 172.21.15.26:59316) 2023-07-13T22:12:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda3a773700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51182/0 (socket says 172.21.15.26:51182) 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 -- 172.21.15.26:0/3902825751 learned_addr learned my addr 172.21.15.26:0/3902825751 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39771700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda341007a0 0x7fda341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 -- 172.21.15.26:0/3902825751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda341007a0 msgr2=0x7fda341984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda341007a0 0x7fda341984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 -- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 msgr2=0x7fda3419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda39f72700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009d00 con 0x7fda340ffe00 2023-07-13T22:12:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.732+0000 7fda3a773700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda34104ea0 0x7fda3419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda39f72700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 0x7fda34197f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9a0 tx=0x7fda3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30009790 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3419d560 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda30010430 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30018680 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.733+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda341a4dc0 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.734+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda3404e210 con 0x7fda340ffe00 2023-07-13T22:12:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.737+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fda300105a0 con 0x7fda340ffe00 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.738+0000 7fda2affd700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda200a11e0 0x7fda200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.738+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fda30014070 con 0x7fda340ffe00 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.738+0000 7fda39771700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda200a11e0 0x7fda200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.739+0000 7fda2affd700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] conn(0x7fda200aad10 0x7fda200ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.739+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda200ad870 con 0x7fda200aad10 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.739+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda300cd2f0 con 0x7fda340ffe00 2023-07-13T22:12:52.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.739+0000 7fda39771700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda200a11e0 0x7fda200a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda341995b0 tx=0x7fda24009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.739+0000 7fda3a773700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] conn(0x7fda200aad10 0x7fda200ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.740+0000 7fda3a773700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] conn(0x7fda200aad10 0x7fda200ad1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.741+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== osd.1 v2:172.21.15.117:6800/2207798137 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fda200ad870 con 0x7fda200aad10 2023-07-13T22:12:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.759+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 --> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda3410fdb0 con 0x7fda200aad10 2023-07-13T22:12:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda2affd700 1 -- 172.21.15.26:0/3902825751 <== osd.1 v2:172.21.15.117:6800/2207798137 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda2c009b80 con 0x7fda200aad10 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] conn(0x7fda200aad10 msgr2=0x7fda200ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] conn(0x7fda200aad10 0x7fda200ad1a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda200a11e0 msgr2=0x7fda200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fda200a11e0 0x7fda200a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda341995b0 tx=0x7fda24009260 comp rx=0 tx=0).stop 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 msgr2=0x7fda34197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 --2- 172.21.15.26:0/3902825751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda340ffe00 0x7fda34197f90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda3000e9a0 tx=0x7fda3000ecb0 comp rx=0 tx=0).stop 2023-07-13T22:12:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3a773700 1 -- 172.21.15.26:0/3902825751 reap_dead start 2023-07-13T22:12:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 shutdown_connections 2023-07-13T22:12:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 >> 172.21.15.26:0/3902825751 conn(0x7fda340fb660 msgr2=0x7fda340fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 shutdown_connections 2023-07-13T22:12:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:52.760+0000 7fda3c1d6700 1 -- 172.21.15.26:0/3902825751 wait complete. 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: pgmap v283: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-13T22:12:52.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2023-07-13T22:12:52.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-13T22:12:52.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-07-13T22:12:52.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:52 smithi117 ceph-mon[129074]: osdmap e84: 12 total, 12 up, 12 in 2023-07-13T22:12:52.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: pgmap v283: 97 pgs: 97 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 11 op/s 2023-07-13T22:12:52.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2023-07-13T22:12:52.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-07-13T22:12:52.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:52 smithi121 ceph-mon[129187]: osdmap e84: 12 total, 12 up, 12 in 2023-07-13T22:12:53.092 INFO:teuthology.orchestra.run.smithi026.stdout:77309411386 2023-07-13T22:12:53.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-13T22:12:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:53.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:53 smithi026 ceph-mon[95647]: osdmap e85: 12 total, 12 up, 12 in 2023-07-13T22:12:53.923 INFO:teuthology.orchestra.run.smithi026.stdout:55834574912 2023-07-13T22:12:53.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-13T22:12:53.924 INFO:teuthology.orchestra.run.smithi026.stdout:163208757293 2023-07-13T22:12:53.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-13T22:12:53.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:53 smithi117 ceph-mon[129074]: osdmap e85: 12 total, 12 up, 12 in 2023-07-13T22:12:53.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:53 smithi121 ceph-mon[129187]: osdmap e85: 12 total, 12 up, 12 in 2023-07-13T22:12:54.641 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:54 smithi026 ceph-mon[95647]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:54.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:54 smithi117 ceph-mon[129074]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:54.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:54 smithi121 ceph-mon[129187]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 -- 172.21.15.26:0/2041735556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 msgr2=0x7f17941011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 --2- 172.21.15.26:0/2041735556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f17941011c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1784009990 tx=0x7f1784009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 -- 172.21.15.26:0/2041735556 shutdown_connections 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 --2- 172.21.15.26:0/2041735556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f17941011c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 --2- 172.21.15.26:0/2041735556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 0x7f179410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 --2- 172.21.15.26:0/2041735556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f1794109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 -- 172.21.15.26:0/2041735556 >> 172.21.15.26:0/2041735556 conn(0x7f1794075b70 msgr2=0x7f1794075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 -- 172.21.15.26:0/2041735556 shutdown_connections 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.729+0000 7f1799397700 1 -- 172.21.15.26:0/2041735556 wait complete. 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.730+0000 7f1799397700 1 Processor -- start 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 -- start start 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f1794197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f17941983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 0x7f179419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419ce30 con 0x7f1794100df0 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419cfa0 con 0x7f179410a140 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1799397700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179419d160 con 0x7f1794101700 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1792ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f1794197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f17927fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f17941983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1792ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f1794197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54440/0 (socket says 172.21.15.26:54440) 2023-07-13T22:12:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f1792ffd700 1 -- 172.21.15.26:0/2114983247 learned_addr learned my addr 172.21.15.26:0/2114983247 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f17927fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f17941983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38132/0 (socket says 172.21.15.26:38132) 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.731+0000 7f17937fe700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 0x7f179419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 -- 172.21.15.26:0/2114983247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 msgr2=0x7f17941983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f17941983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 -- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 msgr2=0x7f179419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 0x7f179419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840095f0 con 0x7f1794100df0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f17927fc700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1794101700 0x7f17941983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f17937fe700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179410a140 0x7f179419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1792ffd700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f1794197e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1784005910 tx=0x7f1784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178401d070 con 0x7f1794100df0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1784022470 con 0x7f1794100df0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179419d3e0 con 0x7f1794100df0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784017650 con 0x7f1794100df0 2023-07-13T22:12:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.732+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17941a4ca0 con 0x7f1794100df0 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.733+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f179404e210 con 0x7f1794100df0 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.736+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f178400fad0 con 0x7f1794100df0 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f177bfff700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f177c0a11e0 0x7f177c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17840ce710 con 0x7f1794100df0 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f17927fc700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f177c0a11e0 0x7f177c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f177bfff700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] conn(0x7f177c0aad10 0x7f177c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f177c0ad870 con 0x7f177c0aad10 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.737+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17840cea80 con 0x7f1794100df0 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.738+0000 7f17937fe700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] conn(0x7f177c0aad10 0x7f177c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.738+0000 7f17927fc700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f177c0a11e0 0x7f177c0a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1794199490 tx=0x7f1788009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.738+0000 7f17937fe700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] conn(0x7f177c0aad10 0x7f177c0ad1a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.739+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== osd.7 v2:172.21.15.117:6816/2078940606 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f177c0ad870 con 0x7f177c0aad10 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.756+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 --> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f179410fd70 con 0x7f177c0aad10 2023-07-13T22:12:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.756+0000 7f177bfff700 1 -- 172.21.15.26:0/2114983247 <== osd.7 v2:172.21.15.117:6816/2078940606 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f178c007500 con 0x7f177c0aad10 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.756+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] conn(0x7f177c0aad10 msgr2=0x7f177c0ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.756+0000 7f1799397700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] conn(0x7f177c0aad10 0x7f177c0ad1a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f177c0a11e0 msgr2=0x7f177c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f177c0a11e0 0x7f177c0a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1794199490 tx=0x7f1788009260 comp rx=0 tx=0).stop 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 msgr2=0x7f1794197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 --2- 172.21.15.26:0/2114983247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1794100df0 0x7f1794197e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1784005910 tx=0x7f1784004730 comp rx=0 tx=0).stop 2023-07-13T22:12:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f17937fe700 1 -- 172.21.15.26:0/2114983247 reap_dead start 2023-07-13T22:12:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 shutdown_connections 2023-07-13T22:12:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 >> 172.21.15.26:0/2114983247 conn(0x7f1794075b70 msgr2=0x7f17940fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 shutdown_connections 2023-07-13T22:12:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:54.757+0000 7f1799397700 1 -- 172.21.15.26:0/2114983247 wait complete. 2023-07-13T22:12:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:55.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 -- 172.21.15.26:0/835501642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc1007a0 msgr2=0x7faefc104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 --2- 172.21.15.26:0/835501642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc1007a0 0x7faefc104770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faee8009990 tx=0x7faee8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 -- 172.21.15.26:0/835501642 shutdown_connections 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 --2- 172.21.15.26:0/835501642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc0ffe00 0x7faefc1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 --2- 172.21.15.26:0/835501642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 --2- 172.21.15.26:0/835501642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc1007a0 0x7faefc104770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 -- 172.21.15.26:0/835501642 >> 172.21.15.26:0/835501642 conn(0x7faefc0fb660 msgr2=0x7faefc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:55.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.335+0000 7faf03146700 1 -- 172.21.15.26:0/835501642 shutdown_connections 2023-07-13T22:12:55.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.336+0000 7faf03146700 1 -- 172.21.15.26:0/835501642 wait complete. 2023-07-13T22:12:55.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.337+0000 7faf03146700 1 Processor -- start 2023-07-13T22:12:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.337+0000 7faf03146700 1 -- start start 2023-07-13T22:12:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc0ffe00 0x7faefc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1007a0 0x7faefc1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc19acb0 con 0x7faefc1007a0 2023-07-13T22:12:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc19ae20 con 0x7faefc0ffe00 2023-07-13T22:12:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf03146700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc19afe0 con 0x7faefc104ea0 2023-07-13T22:12:55.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf016e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf016e3700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38162/0 (socket says 172.21.15.26:38162) 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf00ee2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc0ffe00 0x7faefc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faf016e3700 1 -- 172.21.15.26:0/1807829237 learned_addr learned my addr 172.21.15.26:0/1807829237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.338+0000 7faefbfff700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1007a0 0x7faefc1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 -- 172.21.15.26:0/1807829237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc0ffe00 msgr2=0x7faefc195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faefc0ffe00 0x7faefc195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 -- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1007a0 msgr2=0x7faefc1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1007a0 0x7faefc1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009c40 con 0x7faefc104ea0 2023-07-13T22:12:55.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faefbfff700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faefc1007a0 0x7faefc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:12:55.332 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf016e3700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc19a6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faef400c370 tx=0x7faef400c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4017070 con 0x7faefc104ea0 2023-07-13T22:12:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.339+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee80095f0 con 0x7faefc104ea0 2023-07-13T22:12:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.340+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc1a2950 con 0x7faefc104ea0 2023-07-13T22:12:55.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.340+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faef400f040 con 0x7faefc104ea0 2023-07-13T22:12:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.340+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400e050 con 0x7faefc104ea0 2023-07-13T22:12:55.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.341+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faefc04e210 con 0x7faefc104ea0 2023-07-13T22:12:55.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.343+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faef4004520 con 0x7faefc104ea0 2023-07-13T22:12:55.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.345+0000 7faef9ffb700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faeec0a1260 0x7faeec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.345+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faef40ccdc0 con 0x7faefc104ea0 2023-07-13T22:12:55.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.345+0000 7faef9ffb700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] conn(0x7faeec0aad90 0x7faeec0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:55.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.345+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faeec0ad8f0 con 0x7faeec0aad90 2023-07-13T22:12:55.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.345+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef40cd130 con 0x7faefc104ea0 2023-07-13T22:12:55.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.346+0000 7faefbfff700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] conn(0x7faeec0aad90 0x7faeec0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.347+0000 7faefbfff700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] conn(0x7faeec0aad90 0x7faeec0ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.348+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== osd.9 v2:172.21.15.26:6826/411448379 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faeec0ad8f0 con 0x7faeec0aad90 2023-07-13T22:12:55.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.349+0000 7faf00ee2700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faeec0a1260 0x7faeec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:55.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.350+0000 7faf00ee2700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faeec0a1260 0x7faeec0a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faef0009c60 tx=0x7faef0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.381+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 --> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faefc10fd90 con 0x7faeec0aad90 2023-07-13T22:12:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.383+0000 7faef9ffb700 1 -- 172.21.15.26:0/1807829237 <== osd.9 v2:172.21.15.26:6826/411448379 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faee801e040 con 0x7faeec0aad90 2023-07-13T22:12:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] conn(0x7faeec0aad90 msgr2=0x7faeec0ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] conn(0x7faeec0aad90 0x7faeec0ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faeec0a1260 msgr2=0x7faeec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faeec0a1260 0x7faeec0a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faef0009c60 tx=0x7faef0009260 comp rx=0 tx=0).stop 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 msgr2=0x7faefc19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 --2- 172.21.15.26:0/1807829237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faefc104ea0 0x7faefc19a6a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faef400c370 tx=0x7faef400c680 comp rx=0 tx=0).stop 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf016e3700 1 -- 172.21.15.26:0/1807829237 reap_dead start 2023-07-13T22:12:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 shutdown_connections 2023-07-13T22:12:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.384+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 >> 172.21.15.26:0/1807829237 conn(0x7faefc0fb660 msgr2=0x7faefc0fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.385+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 shutdown_connections 2023-07-13T22:12:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:55.385+0000 7faf03146700 1 -- 172.21.15.26:0/1807829237 wait complete. 2023-07-13T22:12:55.534 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:55.535 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 -- 172.21.15.26:0/581204215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560101700 msgr2=0x7fb560109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 --2- 172.21.15.26:0/581204215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560101700 0x7fb560109c00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0098f0 tx=0x7fb54c009c00 comp rx=0 tx=0).stop 2023-07-13T22:12:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 -- 172.21.15.26:0/581204215 shutdown_connections 2023-07-13T22:12:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 --2- 172.21.15.26:0/581204215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb56010a140 0x7fb56010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 --2- 172.21.15.26:0/581204215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb560100df0 0x7fb5601011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 --2- 172.21.15.26:0/581204215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560101700 0x7fb560109c00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 -- 172.21.15.26:0/581204215 >> 172.21.15.26:0/581204215 conn(0x7fb560075b70 msgr2=0x7fb560075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 -- 172.21.15.26:0/581204215 shutdown_connections 2023-07-13T22:12:56.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.442+0000 7fb566ada700 1 -- 172.21.15.26:0/581204215 wait complete. 2023-07-13T22:12:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.443+0000 7fb566ada700 1 Processor -- start 2023-07-13T22:12:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 -- start start 2023-07-13T22:12:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560100df0 0x7fb560195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 0x7fb560196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb56010a140 0x7fb5601995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560199bd0 con 0x7fb560101700 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560199d40 con 0x7fb56010a140 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb566ada700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560199f00 con 0x7fb560100df0 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 0x7fb560196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb564876700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560100df0 0x7fb560195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 0x7fb560196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54472/0 (socket says 172.21.15.26:54472) 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb55ffff700 1 -- 172.21.15.26:0/1197677086 learned_addr learned my addr 172.21.15.26:0/1197677086 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.444+0000 7fb565077700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb56010a140 0x7fb5601995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 -- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560100df0 msgr2=0x7fb560195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb560100df0 0x7fb560195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 -- 172.21.15.26:0/1197677086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb56010a140 msgr2=0x7fb5601995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb56010a140 0x7fb5601995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009520 con 0x7fb560101700 2023-07-13T22:12:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55ffff700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 0x7fb560196210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009810 tx=0x7fb54c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c01c070 con 0x7fb560101700 2023-07-13T22:12:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56019a180 con 0x7fb560101700 2023-07-13T22:12:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54c00bbd0 con 0x7fb560101700 2023-07-13T22:12:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.445+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004930 con 0x7fb560101700 2023-07-13T22:12:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.446+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a2b30 con 0x7fb560101700 2023-07-13T22:12:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.447+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb56004e210 con 0x7fb560101700 2023-07-13T22:12:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.449+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb54c020070 con 0x7fb560101700 2023-07-13T22:12:56.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.450+0000 7fb55dffb700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb5500a11e0 0x7fb5500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb54c0cde30 con 0x7fb560101700 2023-07-13T22:12:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb564876700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb5500a11e0 0x7fb5500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:56.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb55dffb700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] conn(0x7fb5500aad10 0x7fb5500ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:12:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5500ad870 con 0x7fb5500aad10 2023-07-13T22:12:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54c0ce1a0 con 0x7fb560101700 2023-07-13T22:12:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.451+0000 7fb565077700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] conn(0x7fb5500aad10 0x7fb5500ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:12:56.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.452+0000 7fb564876700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb5500a11e0 0x7fb5500a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb554005fd0 tx=0x7fb554009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:56.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.452+0000 7fb565077700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] conn(0x7fb5500aad10 0x7fb5500ad1a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:12:56.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.453+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== osd.11 v2:172.21.15.121:6824/1646753631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb5500ad870 con 0x7fb5500aad10 2023-07-13T22:12:56.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.471+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 --> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb56010fd80 con 0x7fb5500aad10 2023-07-13T22:12:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb55dffb700 1 -- 172.21.15.26:0/1197677086 <== osd.11 v2:172.21.15.121:6824/1646753631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb55800f040 con 0x7fb5500aad10 2023-07-13T22:12:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] conn(0x7fb5500aad10 msgr2=0x7fb5500ad1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] conn(0x7fb5500aad10 0x7fb5500ad1a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:12:56.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb5500a11e0 msgr2=0x7fb5500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb5500a11e0 0x7fb5500a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb554005fd0 tx=0x7fb554009310 comp rx=0 tx=0).stop 2023-07-13T22:12:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 msgr2=0x7fb560196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:12:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 --2- 172.21.15.26:0/1197677086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb560101700 0x7fb560196210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009810 tx=0x7fb54c00fa00 comp rx=0 tx=0).stop 2023-07-13T22:12:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb565077700 1 -- 172.21.15.26:0/1197677086 reap_dead start 2023-07-13T22:12:56.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 shutdown_connections 2023-07-13T22:12:56.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.472+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 >> 172.21.15.26:0/1197677086 conn(0x7fb560075b70 msgr2=0x7fb560101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:12:56.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.473+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 shutdown_connections 2023-07-13T22:12:56.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:12:56.473+0000 7fb566ada700 1 -- 172.21.15.26:0/1197677086 wait complete. 2023-07-13T22:12:56.689 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:56 smithi026 ceph-mon[95647]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:56.692 INFO:teuthology.orchestra.run.smithi026.stdout:219043332133 2023-07-13T22:12:56.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-13T22:12:56.791 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:56.833 INFO:teuthology.orchestra.run.smithi026.stdout:283467841563 2023-07-13T22:12:56.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-13T22:12:56.840 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:56.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:56 smithi117 ceph-mon[129074]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:56.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:56 smithi121 ceph-mon[129187]: pgmap v287: 97 pgs: 4 peering, 93 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 188 B/s, 0 objects/s recovering 2023-07-13T22:12:57.470 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:57.746 INFO:teuthology.orchestra.run.smithi026.stdout:335007449107 2023-07-13T22:12:57.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-13T22:12:58.464 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:12:58.657 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:12:58 smithi026 ceph-mon[95647]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:12:58.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:12:58 smithi117 ceph-mon[129074]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:12:58.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:12:58 smithi121 ceph-mon[129187]: pgmap v288: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 -- 172.21.15.26:0/1692080015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 msgr2=0x7fe4a80fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/1692080015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a80fefb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009990 tx=0x7fe49c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 -- 172.21.15.26:0/1692080015 shutdown_connections 2023-07-13T22:13:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/1692080015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a8103f10 0x7fe4a8104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/1692080015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a80fefb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/1692080015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 -- 172.21.15.26:0/1692080015 >> 172.21.15.26:0/1692080015 conn(0x7fe4a8075b70 msgr2=0x7fe4a8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 -- 172.21.15.26:0/1692080015 shutdown_connections 2023-07-13T22:13:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.451+0000 7fe4aebc4700 1 -- 172.21.15.26:0/1692080015 wait complete. 2023-07-13T22:13:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.452+0000 7fe4aebc4700 1 Processor -- start 2023-07-13T22:13:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.452+0000 7fe4aebc4700 1 -- start start 2023-07-13T22:13:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a8103f10 0x7fe4a81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a1250 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a13c0 con 0x7fe4a8103f10 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4aebc4700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81a1580 con 0x7fe4a80febe0 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4ac960700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4ac960700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38238/0 (socket says 172.21.15.26:38238) 2023-07-13T22:13:00.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.453+0000 7fe4ac960700 1 -- 172.21.15.26:0/3012148638 learned_addr learned my addr 172.21.15.26:0/3012148638 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4ad161700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a8103f10 0x7fe4a81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 -- 172.21.15.26:0/3012148638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 msgr2=0x7fe4a819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 -- 172.21.15.26:0/3012148638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a8103f10 msgr2=0x7fe4a81a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a8103f10 0x7fe4a81a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 -- 172.21.15.26:0/3012148638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0095f0 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4ac960700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.454+0000 7fe4a7fff700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a819c7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d750 tx=0x7fe49400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.455+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494004220 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.455+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe494004cf0 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.455+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400f690 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.456+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a81a1860 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.456+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a81a90c0 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.457+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a804e210 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.461+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe494004750 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.463+0000 7fe4a5ffb700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 0x7fe4980a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.463+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe4940ccd90 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.464+0000 7fe4ac960700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 0x7fe4980a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.464+0000 7fe4ac960700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 0x7fe4980a3790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00b2e0 tx=0x7fe49c005e60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:00.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.466+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe494096330 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.605 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:00 smithi026 ceph-mon[95647]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:00.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.384+0000 7f2bd131a700 1 -- 172.21.15.26:0/1091332509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc101700 msgr2=0x7f2bcc109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.384+0000 7f2bd131a700 1 --2- 172.21.15.26:0/1091332509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc101700 0x7f2bcc109c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00098f0 tx=0x7f2bc0009c00 comp rx=0 tx=0).stop 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 -- 172.21.15.26:0/1091332509 shutdown_connections 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 --2- 172.21.15.26:0/1091332509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc10a140 0x7f2bcc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 --2- 172.21.15.26:0/1091332509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc100df0 0x7f2bcc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 --2- 172.21.15.26:0/1091332509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc101700 0x7f2bcc109c00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 -- 172.21.15.26:0/1091332509 >> 172.21.15.26:0/1091332509 conn(0x7f2bcc075b70 msgr2=0x7f2bcc075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 -- 172.21.15.26:0/1091332509 shutdown_connections 2023-07-13T22:13:00.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.385+0000 7f2bd131a700 1 -- 172.21.15.26:0/1091332509 wait complete. 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.387+0000 7f2bd131a700 1 Processor -- start 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.387+0000 7f2bd131a700 1 -- start start 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 0x7f2bcc0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc10a140 0x7f2bcc0ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc1a4130 con 0x7f2bcc10a140 2023-07-13T22:13:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc1a42a0 con 0x7f2bcc100df0 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bd131a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc1a4460 con 0x7f2bcc101700 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bca7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bca7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38214/0 (socket says 172.21.15.26:38214) 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bcaffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 0x7f2bcc0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bca7fc700 1 -- 172.21.15.26:0/3415248321 learned_addr learned my addr 172.21.15.26:0/3415248321 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.388+0000 7f2bcb7fe700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc10a140 0x7f2bcc0ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 -- 172.21.15.26:0/3415248321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 msgr2=0x7f2bcc0fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 0x7f2bcc0fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 -- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc10a140 msgr2=0x7f2bcc0ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc10a140 0x7f2bcc0ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 -- 172.21.15.26:0/3415248321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009520 con 0x7f2bcc101700 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bcaffd700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 0x7f2bcc0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bca7fc700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009810 tx=0x7f2bc000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.389+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc001c070 con 0x7f2bcc101700 2023-07-13T22:13:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.390+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc1a46e0 con 0x7f2bcc101700 2023-07-13T22:13:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.390+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc000bb20 con 0x7f2bcc101700 2023-07-13T22:13:00.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.390+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0004930 con 0x7f2bcc101700 2023-07-13T22:13:00.666 INFO:teuthology.orchestra.run.smithi026.stdout:107374182455 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.390+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc1a4bd0 con 0x7f2bcc101700 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.391+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bcc1049d0 con 0x7f2bcc101700 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.394+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2bc0020070 con 0x7f2bcc101700 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.395+0000 7f2bb3fff700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 0x7f2bb40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.395+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2bc00cd700 con 0x7f2bcc101700 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.395+0000 7f2bcaffd700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 0x7f2bb40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:00.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.396+0000 7f2bcaffd700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 0x7f2bb40a3710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011490 tx=0x7f2bbc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.397+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2bc0096d90 con 0x7f2bcc101700 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.624+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2bcc04e210 con 0x7f2bcc101700 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.625+0000 7f2bb3fff700 1 -- 172.21.15.26:0/3415248321 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bc0027020 con 0x7f2bcc101700 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 msgr2=0x7f2bb40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 0x7f2bb40a3710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc011490 tx=0x7f2bbc009260 comp rx=0 tx=0).stop 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 msgr2=0x7f2bcc0fef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009810 tx=0x7f2bc000fa00 comp rx=0 tx=0).stop 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 shutdown_connections 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2bb40a1260 0x7f2bb40a3710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bcc10a140 0x7f2bcc0ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bcc100df0 0x7f2bcc0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 --2- 172.21.15.26:0/3415248321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bcc101700 0x7f2bcc0fef90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.627+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 >> 172.21.15.26:0/3415248321 conn(0x7f2bcc075b70 msgr2=0x7f2bcc101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:00.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.628+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 shutdown_connections 2023-07-13T22:13:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.628+0000 7f2bd131a700 1 -- 172.21.15.26:0/3415248321 wait complete. 2023-07-13T22:13:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.687+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe4a819d5f0 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.687+0000 7fe4a5ffb700 1 -- 172.21.15.26:0/3012148638 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe494016020 con 0x7fe4a80ff4f0 2023-07-13T22:13:00.679 INFO:teuthology.orchestra.run.smithi026.stdout:141733920820 2023-07-13T22:13:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 msgr2=0x7fe4980a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 0x7fe4980a3790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00b2e0 tx=0x7fe49c005e60 comp rx=0 tx=0).stop 2023-07-13T22:13:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 msgr2=0x7fe4a819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a819c7d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d750 tx=0x7fe49400da60 comp rx=0 tx=0).stop 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 shutdown_connections 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4a8103f10 0x7fe4a81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4a80febe0 0x7fe4a819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe4980a12e0 0x7fe4980a3790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 --2- 172.21.15.26:0/3012148638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4a80ff4f0 0x7fe4a819c7d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 >> 172.21.15.26:0/3012148638 conn(0x7fe4a8075b70 msgr2=0x7fe4a8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 shutdown_connections 2023-07-13T22:13:00.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:00.690+0000 7fe4aebc4700 1 -- 172.21.15.26:0/3012148638 wait complete. 2023-07-13T22:13:00.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:00 smithi117 ceph-mon[129074]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:00.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:00 smithi121 ceph-mon[129187]: pgmap v289: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s; 188 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:01.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:01 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3415248321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-13T22:13:01.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:01 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3012148638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-13T22:13:01.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:01 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3415248321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-13T22:13:01.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:01 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3012148638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-13T22:13:01.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:01 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3415248321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-13T22:13:01.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:01 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3012148638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-13T22:13:02.631 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:02 smithi026 ceph-mon[95647]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:02.633 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182453 got 107374182455 for osd.3 2023-07-13T22:13:02.634 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:02.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920818 got 141733920820 for osd.4 2023-07-13T22:13:02.635 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 -- 172.21.15.26:0/3290775032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 msgr2=0x7fdc3c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 --2- 172.21.15.26:0/3290775032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c104770 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdc300099c0 tx=0x7fdc30009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 -- 172.21.15.26:0/3290775032 shutdown_connections 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 --2- 172.21.15.26:0/3290775032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c104ea0 0x7fdc3c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 --2- 172.21.15.26:0/3290775032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 --2- 172.21.15.26:0/3290775032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c104770 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 -- 172.21.15.26:0/3290775032 >> 172.21.15.26:0/3290775032 conn(0x7fdc3c0fb660 msgr2=0x7fdc3c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 -- 172.21.15.26:0/3290775032 shutdown_connections 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.710+0000 7fdc4205e700 1 -- 172.21.15.26:0/3290775032 wait complete. 2023-07-13T22:13:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.712+0000 7fdc4205e700 1 Processor -- start 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.712+0000 7fdc4205e700 1 -- start start 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104ea0 0x7fdc3c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a1320 con 0x7fdc3c1007a0 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc3b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38246/0 (socket says 172.21.15.26:38246) 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc3b7fe700 1 -- 172.21.15.26:0/2274002258 learned_addr learned my addr 172.21.15.26:0/2274002258 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a1490 con 0x7fdc3c104ea0 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a1650 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.713+0000 7fdc3affd700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3bfff700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104ea0 0x7fdc3c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 -- 172.21.15.26:0/2274002258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104ea0 msgr2=0x7fdc3c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104ea0 0x7fdc3c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 -- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 msgr2=0x7fdc3c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300095f0 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc3b7fe700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e9a0 tx=0x7fdc2c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.714+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c009790 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.715+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc2c00cbd0 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.715+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c018680 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.715+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c1a1930 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.715+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c193a80 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.716+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c04e210 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.719+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdc2c00cd40 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.720+0000 7fdc38ff9700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 0x7fdc240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.720+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc2c014070 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.720+0000 7fdc3affd700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 0x7fdc240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.721+0000 7fdc3affd700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 0x7fdc240a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3001a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.721+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc2c096d30 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.950 INFO:teuthology.orchestra.run.smithi026.stdout:304942678041 2023-07-13T22:13:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.941+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fdc3c1a1ac0 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.942+0000 7fdc38ff9700 1 -- 172.21.15.26:0/2274002258 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fdc2c09a350 con 0x7fdc3c0ffe00 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 msgr2=0x7fdc240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 0x7fdc240a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009fd0 tx=0x7fdc3001a040 comp rx=0 tx=0).stop 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 msgr2=0x7fdc3c19c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00e9a0 tx=0x7fdc2c00ed60 comp rx=0 tx=0).stop 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 shutdown_connections 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc3c104ea0 0x7fdc3c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc3c0ffe00 0x7fdc3c19c360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc240a1240 0x7fdc240a36f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 --2- 172.21.15.26:0/2274002258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc3c1007a0 0x7fdc3c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 >> 172.21.15.26:0/2274002258 conn(0x7fdc3c0fb660 msgr2=0x7fdc3c0fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 shutdown_connections 2023-07-13T22:13:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.945+0000 7fdc4205e700 1 -- 172.21.15.26:0/2274002258 wait complete. 2023-07-13T22:13:02.952 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:02 smithi117 ceph-mon[129074]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:02.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 -- 172.21.15.26:0/3070471476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c0ffe00 msgr2=0x7f645c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 --2- 172.21.15.26:0/3070471476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c0ffe00 0x7f645c1001d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f644c0099c0 tx=0x7f644c009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 -- 172.21.15.26:0/3070471476 shutdown_connections 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 --2- 172.21.15.26:0/3070471476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c0ffe00 0x7f645c1001d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 --2- 172.21.15.26:0/3070471476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c104ea0 0x7f645c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 --2- 172.21.15.26:0/3070471476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c1007a0 0x7f645c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 -- 172.21.15.26:0/3070471476 >> 172.21.15.26:0/3070471476 conn(0x7f645c0fb660 msgr2=0x7f645c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 -- 172.21.15.26:0/3070471476 shutdown_connections 2023-07-13T22:13:02.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.982+0000 7f6461640700 1 -- 172.21.15.26:0/3070471476 wait complete. 2023-07-13T22:13:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.983+0000 7f6461640700 1 Processor -- start 2023-07-13T22:13:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.983+0000 7f6461640700 1 -- start start 2023-07-13T22:13:02.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c1007a0 0x7f645c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 0x7f645c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19ce80 con 0x7f645c104ea0 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19cff0 con 0x7f645c0ffe00 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f6461640700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19d1b0 con 0x7f645c1007a0 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 0x7f645c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52954/0 (socket says 172.21.15.26:52954) 2023-07-13T22:13:02.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645affd700 1 -- 172.21.15.26:0/816059475 learned_addr learned my addr 172.21.15.26:0/816059475 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:02.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645a7fc700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c1007a0 0x7f645c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645affd700 1 -- 172.21.15.26:0/816059475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c1007a0 msgr2=0x7f645c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.984+0000 7f645affd700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c1007a0 0x7f645c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f645affd700 1 -- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 msgr2=0x7f645c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f645affd700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 0x7f645c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f645affd700 1 -- 172.21.15.26:0/816059475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0095f0 con 0x7f645c0ffe00 2023-07-13T22:13:02.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f645b7fe700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 0x7f645c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f645affd700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f644c009990 tx=0x7f644c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c01d070 con 0x7f645c0ffe00 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c19d430 con 0x7f645c0ffe00 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f644c005030 con 0x7f645c0ffe00 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c004a50 con 0x7f645c0ffe00 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.985+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c1a4cf0 con 0x7f645c0ffe00 2023-07-13T22:13:02.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.987+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c04e210 con 0x7f645c0ffe00 2023-07-13T22:13:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.988+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f644c005430 con 0x7f645c0ffe00 2023-07-13T22:13:02.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.990+0000 7f6443fff700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 0x7f64440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:02.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.990+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f644c0cd810 con 0x7f645c0ffe00 2023-07-13T22:13:02.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.990+0000 7f645a7fc700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 0x7f64440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:02.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.991+0000 7f645a7fc700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 0x7f64440a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f645c1994e0 tx=0x7f64500073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:02.994+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f644c096e40 con 0x7f645c0ffe00 2023-07-13T22:13:02.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:02 smithi121 ceph-mon[129187]: pgmap v290: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4 op/s; 152 B/s, 1 keys/s, 1 objects/s recovering 2023-07-13T22:13:03.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/497139027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 msgr2=0x7fb3ec1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/497139027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec1023f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/497139027 shutdown_connections 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/497139027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 0x7fb3ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/497139027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec1023f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/497139027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/497139027 >> 172.21.15.26:0/497139027 conn(0x7fb3ec0fd880 msgr2=0x7fb3ec0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/497139027 shutdown_connections 2023-07-13T22:13:03.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.197+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/497139027 wait complete. 2023-07-13T22:13:03.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.198+0000 7fb3f2e8a700 1 Processor -- start 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 -- start start 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 0x7fb3ec19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec19cec0 con 0x7fb3ec1070c0 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec19d030 con 0x7fb3ec102020 2023-07-13T22:13:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f2e8a700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec19d1f0 con 0x7fb3ec1029c0 2023-07-13T22:13:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f0c26700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3ebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33570/0 (socket says 172.21.15.26:33570) 2023-07-13T22:13:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3ebfff700 1 -- 172.21.15.26:0/200974002 learned_addr learned my addr 172.21.15.26:0/200974002 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.199+0000 7fb3f1427700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 0x7fb3ec19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 -- 172.21.15.26:0/200974002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 msgr2=0x7fb3ec198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 -- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 msgr2=0x7fb3ec19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 0x7fb3ec19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 -- 172.21.15.26:0/200974002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00095f0 con 0x7fb3ec102020 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3ebfff700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f0c26700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec197f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e001d070 con 0x7fb3ec102020 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ec19d470 con 0x7fb3ec102020 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.200+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e0021410 con 0x7fb3ec102020 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.201+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017650 con 0x7fb3ec102020 2023-07-13T22:13:03.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.201+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ec1a4d30 con 0x7fb3ec102020 2023-07-13T22:13:03.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.202+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ec04e210 con 0x7fb3ec102020 2023-07-13T22:13:03.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.204+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb3e0021810 con 0x7fb3ec102020 2023-07-13T22:13:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.205+0000 7fb3e9ffb700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 0x7fb3dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.205+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3e00ce3f0 con 0x7fb3ec102020 2023-07-13T22:13:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.205+0000 7fb3ebfff700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 0x7fb3dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.206+0000 7fb3ebfff700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 0x7fb3dc0a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec199520 tx=0x7fb3d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.207+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb3e0097a20 con 0x7fb3ec102020 2023-07-13T22:13:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.244+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f645c1a4e80 con 0x7f645c0ffe00 2023-07-13T22:13:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.245+0000 7f6443fff700 1 -- 172.21.15.26:0/816059475 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f644c026090 con 0x7f645c0ffe00 2023-07-13T22:13:03.236 INFO:teuthology.orchestra.run.smithi026.stdout:197568495657 2023-07-13T22:13:03.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 msgr2=0x7f64440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 0x7f64440a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f645c1994e0 tx=0x7f64500073d0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 msgr2=0x7f645c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f644c009990 tx=0x7f644c00f690 comp rx=0 tx=0).stop 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 shutdown_connections 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64440a11e0 0x7f64440a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f645c104ea0 0x7f645c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f645c0ffe00 0x7f645c197ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 --2- 172.21.15.26:0/816059475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c1007a0 0x7f645c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.247+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 >> 172.21.15.26:0/816059475 conn(0x7f645c0fb660 msgr2=0x7f645c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.248+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 shutdown_connections 2023-07-13T22:13:03.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.248+0000 7f6461640700 1 -- 172.21.15.26:0/816059475 wait complete. 2023-07-13T22:13:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 -- 172.21.15.26:0/326673164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4100df0 msgr2=0x7f09c41011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/326673164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4100df0 0x7f09c41011c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009990 tx=0x7f09b4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 -- 172.21.15.26:0/326673164 shutdown_connections 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/326673164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c410a140 0x7f09c410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/326673164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c4100df0 0x7f09c41011c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/326673164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c4101700 0x7f09c4109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 -- 172.21.15.26:0/326673164 >> 172.21.15.26:0/326673164 conn(0x7f09c4075b70 msgr2=0x7f09c4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 -- 172.21.15.26:0/326673164 shutdown_connections 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.448+0000 7f09c9fd9700 1 -- 172.21.15.26:0/326673164 wait complete. 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.449+0000 7f09c9fd9700 1 Processor -- start 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 -- start start 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 0x7f09c419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c4101700 0x7f09c419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 0x7f09c41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41a11c0 con 0x7f09c4100df0 2023-07-13T22:13:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41a1330 con 0x7f09c410a140 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.450+0000 7f09c9fd9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41a14f0 con 0x7f09c4101700 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 0x7f09c419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 0x7f09c41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 0x7f09c419c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43614/0 (socket says 172.21.15.26:43614) 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c37fe700 1 -- 172.21.15.26:0/2425505327 learned_addr learned my addr 172.21.15.26:0/2425505327 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 -- 172.21.15.26:0/2425505327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c4101700 msgr2=0x7f09c419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c4101700 0x7f09c419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 -- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 msgr2=0x7f09c419c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 0x7f09c419c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 -- 172.21.15.26:0/2425505327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b40095f0 con 0x7f09c410a140 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.451+0000 7f09c3fff700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 0x7f09c41a0bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00efc0 tx=0x7f09bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.452+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc018070 con 0x7f09c410a140 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.452+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09bc00f040 con 0x7f09c410a140 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.452+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c41a1770 con 0x7f09c410a140 2023-07-13T22:13:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.452+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc007760 con 0x7f09c410a140 2023-07-13T22:13:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.452+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c41a9030 con 0x7f09c410a140 2023-07-13T22:13:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.453+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c41049f0 con 0x7f09c410a140 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.455+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f09bc00c930 con 0x7f09c410a140 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.457+0000 7f09c0ff9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 0x7f09ac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.457+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f09bc0cc850 con 0x7f09c410a140 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.457+0000 7f09c37fe700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 0x7f09ac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.458+0000 7f09c37fe700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 0x7f09ac0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005b00 tx=0x7f09b4005a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.459+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f09bc099880 con 0x7f09c410a140 2023-07-13T22:13:03.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.469+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb3ec199260 con 0x7fb3ec102020 2023-07-13T22:13:03.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.470+0000 7fb3e9ffb700 1 -- 172.21.15.26:0/200974002 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb3e009b040 con 0x7fb3ec102020 2023-07-13T22:13:03.461 INFO:teuthology.orchestra.run.smithi026.stdout:249108103202 2023-07-13T22:13:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.474+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 msgr2=0x7fb3dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.474+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 0x7fb3dc0a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec199520 tx=0x7fb3d8009260 comp rx=0 tx=0).stop 2023-07-13T22:13:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.474+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 msgr2=0x7fb3ec197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.474+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec197f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009990 tx=0x7fb3e0009580 comp rx=0 tx=0).stop 2023-07-13T22:13:03.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 shutdown_connections 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3dc0a1260 0x7fb3dc0a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3ec1070c0 0x7fb3ec19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3ec102020 0x7fb3ec197f00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 --2- 172.21.15.26:0/200974002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3ec1029c0 0x7fb3ec198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 >> 172.21.15.26:0/200974002 conn(0x7fb3ec0fd880 msgr2=0x7fb3ec0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 shutdown_connections 2023-07-13T22:13:03.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.475+0000 7fb3f2e8a700 1 -- 172.21.15.26:0/200974002 wait complete. 2023-07-13T22:13:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:03.497 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:03.568 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:03 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2274002258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-13T22:13:03.568 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:03 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/816059475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-13T22:13:03.569 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:03 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/200974002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-13T22:13:03.594 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 -- 172.21.15.26:0/1819488245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f980ffe00 msgr2=0x7f0f981001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 --2- 172.21.15.26:0/1819488245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f980ffe00 0x7f0f981001d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0f940099c0 tx=0x7f0f94009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 -- 172.21.15.26:0/1819488245 shutdown_connections 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 --2- 172.21.15.26:0/1819488245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f980ffe00 0x7f0f981001d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 --2- 172.21.15.26:0/1819488245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 0x7f0f9810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 --2- 172.21.15.26:0/1819488245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f981007a0 0x7f0f98104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 -- 172.21.15.26:0/1819488245 >> 172.21.15.26:0/1819488245 conn(0x7f0f980fb660 msgr2=0x7f0f980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 -- 172.21.15.26:0/1819488245 shutdown_connections 2023-07-13T22:13:03.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.712+0000 7f0f9f59c700 1 -- 172.21.15.26:0/1819488245 wait complete. 2023-07-13T22:13:03.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.714+0000 7f0f9f59c700 1 Processor -- start 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.714+0000 7f0f9f59c700 1 -- start start 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.714+0000 7f0f9f59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 0x7f0f98197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.714+0000 7f0f9f59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 0x7f0f981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.714+0000 7f0f9f59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 0x7f0f9819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9f59c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819ce60 con 0x7f0f981007a0 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9f59c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819cfd0 con 0x7f0f98104ea0 2023-07-13T22:13:03.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9f59c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819d190 con 0x7f0f980ffe00 2023-07-13T22:13:03.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9cb37700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 0x7f0f981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9d338700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 0x7f0f98197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9cb37700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 0x7f0f981983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43632/0 (socket says 172.21.15.26:43632) 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9cb37700 1 -- 172.21.15.26:0/79988038 learned_addr learned my addr 172.21.15.26:0/79988038 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9db39700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 0x7f0f9819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9d338700 1 -- 172.21.15.26:0/79988038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 msgr2=0x7f0f9819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9d338700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 0x7f0f9819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9d338700 1 -- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 msgr2=0x7f0f981983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.715+0000 7f0f9d338700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 0x7f0f981983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f9d338700 1 -- 172.21.15.26:0/79988038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940095f0 con 0x7f0f980ffe00 2023-07-13T22:13:03.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f9d338700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 0x7f0f98197ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f940098e0 tx=0x7f0f9400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9401d070 con 0x7f0f980ffe00 2023-07-13T22:13:03.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f9f59c700 1 -- 172.21.15.26:0/79988038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9819d410 con 0x7f0f980ffe00 2023-07-13T22:13:03.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f94005050 con 0x7f0f980ffe00 2023-07-13T22:13:03.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94004970 con 0x7f0f980ffe00 2023-07-13T22:13:03.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.716+0000 7f0f9f59c700 1 -- 172.21.15.26:0/79988038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f981a4cd0 con 0x7f0f980ffe00 2023-07-13T22:13:03.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.718+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f780052f0 con 0x7f0f980ffe00 2023-07-13T22:13:03.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.721+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0f94005470 con 0x7f0f980ffe00 2023-07-13T22:13:03.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.724+0000 7f0f8e7fc700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:03.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.724+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f940cd6e0 con 0x7f0f980ffe00 2023-07-13T22:13:03.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.724+0000 7f0f9cb37700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.724+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0f940cdb20 con 0x7f0f980ffe00 2023-07-13T22:13:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.725+0000 7f0f9cb37700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 0x7f0f840a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f981994c0 tx=0x7f0f88009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:03.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.775+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09c402c850 con 0x7f09c410a140 2023-07-13T22:13:03.768 INFO:teuthology.orchestra.run.smithi026.stdout:34359738438 2023-07-13T22:13:03.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.777+0000 7f09c0ff9700 1 -- 172.21.15.26:0/2425505327 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09bc099410 con 0x7f09c410a140 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 msgr2=0x7f09ac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 0x7f09ac0a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005b00 tx=0x7f09b4005a90 comp rx=0 tx=0).stop 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 msgr2=0x7f09c41a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 0x7f09c41a0bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00efc0 tx=0x7f09bc00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 shutdown_connections 2023-07-13T22:13:03.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09ac0a1240 0x7f09ac0a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09c4100df0 0x7f09c419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09c410a140 0x7f09c41a0bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 --2- 172.21.15.26:0/2425505327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c4101700 0x7f09c419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.780+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 >> 172.21.15.26:0/2425505327 conn(0x7f09c4075b70 msgr2=0x7f09c4101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.781+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 shutdown_connections 2023-07-13T22:13:03.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.781+0000 7f09c9fd9700 1 -- 172.21.15.26:0/2425505327 wait complete. 2023-07-13T22:13:03.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:03 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2274002258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-13T22:13:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:03 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/816059475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-13T22:13:03.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:03 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/200974002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-13T22:13:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.976+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0f78005160 con 0x7f0f980ffe00 2023-07-13T22:13:03.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.976+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/79988038 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f94096d10 con 0x7f0f980ffe00 2023-07-13T22:13:03.968 INFO:teuthology.orchestra.run.smithi026.stdout:77309411388 2023-07-13T22:13:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 msgr2=0x7f0f840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 0x7f0f840a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f981994c0 tx=0x7f0f88009260 comp rx=0 tx=0).stop 2023-07-13T22:13:03.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 msgr2=0x7f0f98197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 0x7f0f98197ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f940098e0 tx=0x7f0f9400f690 comp rx=0 tx=0).stop 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 shutdown_connections 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f98104ea0 0x7f0f9819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f980ffe00 0x7f0f98197ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f840a11e0 0x7f0f840a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 --2- 172.21.15.26:0/79988038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f981007a0 0x7f0f981983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:03.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 >> 172.21.15.26:0/79988038 conn(0x7f0f980fb660 msgr2=0x7f0f980fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:03.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 shutdown_connections 2023-07-13T22:13:03.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:03.979+0000 7f0f83fff700 1 -- 172.21.15.26:0/79988038 wait complete. 2023-07-13T22:13:03.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:03 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2274002258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-13T22:13:03.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:03 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/816059475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-13T22:13:03.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:03 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/200974002' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-13T22:13:04.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:04 smithi117 ceph-mon[129074]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 138 B/s, 0 keys/s, 1 objects/s recovering 2023-07-13T22:13:04.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:04 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2425505327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-13T22:13:04.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:04 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/79988038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-13T22:13:04.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:04 smithi026 ceph-mon[95647]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 138 B/s, 0 keys/s, 1 objects/s recovering 2023-07-13T22:13:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2425505327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-13T22:13:04.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/79988038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-13T22:13:04.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:04 smithi121 ceph-mon[129187]: pgmap v291: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 7 op/s; 138 B/s, 0 keys/s, 1 objects/s recovering 2023-07-13T22:13:04.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:04 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2425505327' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-13T22:13:04.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:04 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/79988038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-13T22:13:05.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103202 for osd.8 2023-07-13T22:13:05.087 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:05.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 -- 172.21.15.26:0/630422271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29980ffe00 msgr2=0x7f29981001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 --2- 172.21.15.26:0/630422271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29980ffe00 0x7f29981001d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f29880099c0 tx=0x7f2988009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 -- 172.21.15.26:0/630422271 shutdown_connections 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 --2- 172.21.15.26:0/630422271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29980ffe00 0x7f29981001d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 --2- 172.21.15.26:0/630422271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2998104ea0 0x7f299810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 --2- 172.21.15.26:0/630422271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29981007a0 0x7f2998104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 -- 172.21.15.26:0/630422271 >> 172.21.15.26:0/630422271 conn(0x7f29980fb660 msgr2=0x7f29980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 -- 172.21.15.26:0/630422271 shutdown_connections 2023-07-13T22:13:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.251+0000 7f299d0d9700 1 -- 172.21.15.26:0/630422271 wait complete. 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.252+0000 7f299d0d9700 1 Processor -- start 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.252+0000 7f299d0d9700 1 -- start start 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980ffe00 0x7f2998197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 0x7f299819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819cd80 con 0x7f29981007a0 2023-07-13T22:13:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819cef0 con 0x7f2998104ea0 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299d0d9700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819d0b0 con 0x7f29980ffe00 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43650/0 (socket says 172.21.15.26:43650) 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 -- 172.21.15.26:0/3049110987 learned_addr learned my addr 172.21.15.26:0/3049110987 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f2996d9d700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980ffe00 0x7f2998197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299759e700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 0x7f299819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 -- 172.21.15.26:0/3049110987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980ffe00 msgr2=0x7f2998197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980ffe00 0x7f2998197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 -- 172.21.15.26:0/3049110987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 msgr2=0x7f299819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 0x7f299819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299659c700 1 -- 172.21.15.26:0/3049110987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880095f0 con 0x7f29981007a0 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.253+0000 7f299759e700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 0x7f299819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f299659c700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d980 tx=0x7f298c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c009930 con 0x7f29981007a0 2023-07-13T22:13:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299819d390 con 0x7f29981007a0 2023-07-13T22:13:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f298c010460 con 0x7f29981007a0 2023-07-13T22:13:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00f620 con 0x7f29981007a0 2023-07-13T22:13:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.254+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29981a4bf0 con 0x7f29981007a0 2023-07-13T22:13:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.256+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299804e210 con 0x7f29981007a0 2023-07-13T22:13:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.257+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f298c0108d0 con 0x7f29981007a0 2023-07-13T22:13:05.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.258+0000 7f297ffff700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 0x7f29800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.259+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f298c0cc5a0 con 0x7f29981007a0 2023-07-13T22:13:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.259+0000 7f2996d9d700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 0x7f29800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:05.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.259+0000 7f2996d9d700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 0x7f29800a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f2988009f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:05.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.263+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f298c095b40 con 0x7f29981007a0 2023-07-13T22:13:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.472+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29981a4d80 con 0x7f29981007a0 2023-07-13T22:13:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.472+0000 7f297ffff700 1 -- 172.21.15.26:0/3049110987 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f298c099160 con 0x7f29981007a0 2023-07-13T22:13:05.464 INFO:teuthology.orchestra.run.smithi026.stdout:55834574914 2023-07-13T22:13:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 msgr2=0x7f29800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 0x7f29800a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f2988009f10 comp rx=0 tx=0).stop 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 msgr2=0x7f2998198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d980 tx=0x7f298c00dc90 comp rx=0 tx=0).stop 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 shutdown_connections 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2998104ea0 0x7f299819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980ffe00 0x7f2998197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f29800a11e0 0x7f29800a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 --2- 172.21.15.26:0/3049110987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29981007a0 0x7f2998198300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.474+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 >> 172.21.15.26:0/3049110987 conn(0x7f29980fb660 msgr2=0x7f29980fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.475+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 shutdown_connections 2023-07-13T22:13:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:05.475+0000 7f299d0d9700 1 -- 172.21.15.26:0/3049110987 wait complete. 2023-07-13T22:13:05.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678039 got 304942678041 for osd.10 2023-07-13T22:13:05.555 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:05.617 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411386 got 77309411388 for osd.2 2023-07-13T22:13:05.617 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:05.762 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:05 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3049110987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-13T22:13:05.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:05 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3049110987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-13T22:13:05.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:05 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3049110987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-13T22:13:06.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738435 got 34359738438 for osd.0 2023-07-13T22:13:06.525 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:06.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495655 got 197568495657 for osd.6 2023-07-13T22:13:06.559 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:06.585 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574912 got 55834574914 for osd.1 2023-07-13T22:13:06.585 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:06.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:06 smithi026 ceph-mon[95647]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:06.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:06 smithi117 ceph-mon[129074]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:06.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:06 smithi121 ceph-mon[129187]: pgmap v292: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:08.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 -- 172.21.15.26:0/3627563654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 msgr2=0x7f9838104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3627563654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838104770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98240099c0 tx=0x7f9824009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 -- 172.21.15.26:0/3627563654 shutdown_connections 2023-07-13T22:13:08.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3627563654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9838104ea0 0x7f983810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3627563654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98380ffe00 0x7f98381001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3627563654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838104770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 -- 172.21.15.26:0/3627563654 >> 172.21.15.26:0/3627563654 conn(0x7f98380fb660 msgr2=0x7f98380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 -- 172.21.15.26:0/3627563654 shutdown_connections 2023-07-13T22:13:08.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.035+0000 7f983ebaf700 1 -- 172.21.15.26:0/3627563654 wait complete. 2023-07-13T22:13:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.037+0000 7f983ebaf700 1 Processor -- start 2023-07-13T22:13:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.037+0000 7f983ebaf700 1 -- start start 2023-07-13T22:13:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.037+0000 7f983ebaf700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98380ffe00 0x7f9838197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.037+0000 7f983ebaf700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983ebaf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9838104ea0 0x7f983819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983ebaf700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819cf10 con 0x7f98381007a0 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983ebaf700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819d080 con 0x7f9838104ea0 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983ebaf700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819d240 con 0x7f98380ffe00 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43668/0 (socket says 172.21.15.26:43668) 2023-07-13T22:13:08.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 -- 172.21.15.26:0/3261639954 learned_addr learned my addr 172.21.15.26:0/3261639954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983d14c700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9838104ea0 0x7f983819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f983c94b700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98380ffe00 0x7f9838197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 -- 172.21.15.26:0/3261639954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98380ffe00 msgr2=0x7f9838197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98380ffe00 0x7f9838197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 -- 172.21.15.26:0/3261639954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9838104ea0 msgr2=0x7f983819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9838104ea0 0x7f983819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.038+0000 7f9837fff700 1 -- 172.21.15.26:0/3261639954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240095f0 con 0x7f98381007a0 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f9837fff700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f982400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982401d070 con 0x7f98381007a0 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9824005050 con 0x7f98381007a0 2023-07-13T22:13:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983819d4c0 con 0x7f98381007a0 2023-07-13T22:13:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824004a90 con 0x7f98381007a0 2023-07-13T22:13:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.039+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a4d80 con 0x7f98381007a0 2023-07-13T22:13:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.041+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983804e210 con 0x7f98381007a0 2023-07-13T22:13:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.042+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9824005470 con 0x7f98381007a0 2023-07-13T22:13:08.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.044+0000 7f9835ffb700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 0x7f98280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.044+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98240cd850 con 0x7f98381007a0 2023-07-13T22:13:08.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.044+0000 7f983c94b700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 0x7f98280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.045+0000 7f983c94b700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 0x7f98280a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f982c009c60 tx=0x7f982c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.049+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9824096e80 con 0x7f98381007a0 2023-07-13T22:13:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.268+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f98381a4f10 con 0x7f98381007a0 2023-07-13T22:13:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.269+0000 7f9835ffb700 1 -- 172.21.15.26:0/3261639954 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f982409a4a0 con 0x7f98381007a0 2023-07-13T22:13:08.261 INFO:teuthology.orchestra.run.smithi026.stdout:163208757297 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 msgr2=0x7f98280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 0x7f98280a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f982c009c60 tx=0x7f982c009260 comp rx=0 tx=0).stop 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 msgr2=0x7f9838198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9824009fd0 tx=0x7f982400f690 comp rx=0 tx=0).stop 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 shutdown_connections 2023-07-13T22:13:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9838104ea0 0x7f983819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98380ffe00 0x7f9838197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98280a11e0 0x7f98280a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 --2- 172.21.15.26:0/3261639954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98381007a0 0x7f9838198490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 >> 172.21.15.26:0/3261639954 conn(0x7f98380fb660 msgr2=0x7f98380fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 shutdown_connections 2023-07-13T22:13:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.272+0000 7f983ebaf700 1 -- 172.21.15.26:0/3261639954 wait complete. 2023-07-13T22:13:08.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 -- 172.21.15.26:0/85933643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0731c0 msgr2=0x7fbf6c10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/85933643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c10bea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009990 tx=0x7fbf5c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 -- 172.21.15.26:0/85933643 shutdown_connections 2023-07-13T22:13:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/85933643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c10c3e0 0x7fbf6c10e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/85933643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/85933643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c10bea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 -- 172.21.15.26:0/85933643 >> 172.21.15.26:0/85933643 conn(0x7fbf6c0fb660 msgr2=0x7fbf6c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 -- 172.21.15.26:0/85933643 shutdown_connections 2023-07-13T22:13:08.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.316+0000 7fbf73d6f700 1 -- 172.21.15.26:0/85933643 wait complete. 2023-07-13T22:13:08.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.317+0000 7fbf73d6f700 1 Processor -- start 2023-07-13T22:13:08.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.317+0000 7fbf73d6f700 1 -- start start 2023-07-13T22:13:08.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c10c3e0 0x7fbf6c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19ce50 con 0x7fbf6c0731c0 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19cfc0 con 0x7fbf6c10c3e0 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf73d6f700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c19d180 con 0x7fbf6c0728b0 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33670/0 (socket says 172.21.15.26:33670) 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 -- 172.21.15.26:0/1015278797 learned_addr learned my addr 172.21.15.26:0/1015278797 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf7130a700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf7230c700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c10c3e0 0x7fbf6c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 -- 172.21.15.26:0/1015278797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c10c3e0 msgr2=0x7fbf6c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c10c3e0 0x7fbf6c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 -- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c0731c0 msgr2=0x7fbf6c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.318+0000 7fbf71b0b700 1 -- 172.21.15.26:0/1015278797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0095f0 con 0x7fbf6c0728b0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf71b0b700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e9a0 tx=0x7fbf6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68009790 con 0x7fbf6c0728b0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf68010430 con 0x7fbf6c0728b0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c19d460 con 0x7fbf6c0728b0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68018680 con 0x7fbf6c0728b0 2023-07-13T22:13:08.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.319+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c1a4c30 con 0x7fbf6c0728b0 2023-07-13T22:13:08.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.320+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c106d00 con 0x7fbf6c0728b0 2023-07-13T22:13:08.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.322+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbf680105a0 con 0x7fbf6c0728b0 2023-07-13T22:13:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.324+0000 7fbf62ffd700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 0x7fbf580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.324+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbf68014070 con 0x7fbf6c0728b0 2023-07-13T22:13:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.324+0000 7fbf7130a700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 0x7fbf580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:08.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.324+0000 7fbf7130a700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 0x7fbf580a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c199420 tx=0x7fbf5c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:08.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.327+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf68051790 con 0x7fbf6c0728b0 2023-07-13T22:13:08.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.550+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbf6c04e210 con 0x7fbf6c0728b0 2023-07-13T22:13:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.551+0000 7fbf62ffd700 1 -- 172.21.15.26:0/1015278797 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf68096250 con 0x7fbf6c0728b0 2023-07-13T22:13:08.542 INFO:teuthology.orchestra.run.smithi026.stdout:219043332135 2023-07-13T22:13:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 msgr2=0x7fbf580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 0x7fbf580a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c199420 tx=0x7fbf5c005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 msgr2=0x7fbf6c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800e9a0 tx=0x7fbf6800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 shutdown_connections 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf6c10c3e0 0x7fbf6c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf6c0728b0 0x7fbf6c197e00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbf580a11e0 0x7fbf580a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 --2- 172.21.15.26:0/1015278797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf6c0731c0 0x7fbf6c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 >> 172.21.15.26:0/1015278797 conn(0x7fbf6c0fb660 msgr2=0x7fbf6c103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 shutdown_connections 2023-07-13T22:13:08.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:08.554+0000 7fbf73d6f700 1 -- 172.21.15.26:0/1015278797 wait complete. 2023-07-13T22:13:08.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:08 smithi026 ceph-mon[95647]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:08 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3261639954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-13T22:13:08.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:08 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1015278797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-13T22:13:08.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:08 smithi117 ceph-mon[129074]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:08.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:08 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3261639954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-13T22:13:08.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:08 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1015278797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-13T22:13:08.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:08 smithi121 ceph-mon[129187]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-07-13T22:13:08.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:08 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3261639954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-13T22:13:08.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:08 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1015278797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-13T22:13:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 -- 172.21.15.26:0/3322946670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 msgr2=0x7f7440100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3322946670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440100200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f743c0099c0 tx=0x7f743c009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 -- 172.21.15.26:0/3322946670 shutdown_connections 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3322946670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 0x7f744010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3322946670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440100200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3322946670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f7440104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 -- 172.21.15.26:0/3322946670 >> 172.21.15.26:0/3322946670 conn(0x7f7440077b70 msgr2=0x7f7440077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 -- 172.21.15.26:0/3322946670 shutdown_connections 2023-07-13T22:13:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.010+0000 7f7447fc6700 1 -- 172.21.15.26:0/3322946670 wait complete. 2023-07-13T22:13:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.011+0000 7f7447fc6700 1 Processor -- start 2023-07-13T22:13:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.011+0000 7f7447fc6700 1 -- start start 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f74401961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 0x7f74401995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440199ba0 con 0x7f7440104ec0 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440199d10 con 0x7f74401007d0 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7447fc6700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440199ed0 con 0x7f74400ffe30 2023-07-13T22:13:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33686/0 (socket says 172.21.15.26:33686) 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 -- 172.21.15.26:0/3578157736 learned_addr learned my addr 172.21.15.26:0/3578157736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7446563700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 0x7f74401995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445561700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f74401961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 -- 172.21.15.26:0/3578157736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 msgr2=0x7f74401961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f74401961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 -- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 msgr2=0x7f74401995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 0x7f74401995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445d62700 1 -- 172.21.15.26:0/3578157736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c0095f0 con 0x7f74400ffe30 2023-07-13T22:13:09.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.012+0000 7f7445561700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f74401961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:09.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7445d62700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b580 tx=0x7f743c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c01d070 con 0x7f74400ffe30 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744019a150 con 0x7f74400ffe30 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f743c0054f0 con 0x7f74400ffe30 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c004b30 con 0x7f74400ffe30 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.013+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744019a420 con 0x7f74400ffe30 2023-07-13T22:13:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.014+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744004e210 con 0x7f74400ffe30 2023-07-13T22:13:09.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.017+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f743c00fb80 con 0x7f74400ffe30 2023-07-13T22:13:09.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.018+0000 7f7436ffd700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 0x7f742c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.019+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f743c0cd920 con 0x7f74400ffe30 2023-07-13T22:13:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.019+0000 7f7445561700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 0x7f742c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.019+0000 7f7445561700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 0x7f742c0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7430005950 tx=0x7f74300058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:09.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.020+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f743c096f50 con 0x7f74400ffe30 2023-07-13T22:13:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.232+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f7440196fc0 con 0x7f74400ffe30 2023-07-13T22:13:09.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.232+0000 7f7436ffd700 1 -- 172.21.15.26:0/3578157736 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f743c09a570 con 0x7f74400ffe30 2023-07-13T22:13:09.224 INFO:teuthology.orchestra.run.smithi026.stdout:283467841565 2023-07-13T22:13:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 msgr2=0x7f742c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 0x7f742c0a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7430005950 tx=0x7f74300058e0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 msgr2=0x7f7440195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f743c00b580 tx=0x7f743c00f690 comp rx=0 tx=0).stop 2023-07-13T22:13:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 shutdown_connections 2023-07-13T22:13:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f742c0a11e0 0x7f742c0a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7440104ec0 0x7f74401995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f74400ffe30 0x7f7440195ca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 --2- 172.21.15.26:0/3578157736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74401007d0 0x7f74401961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 >> 172.21.15.26:0/3578157736 conn(0x7f7440077b70 msgr2=0x7f74400fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 shutdown_connections 2023-07-13T22:13:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.235+0000 7f7447fc6700 1 -- 172.21.15.26:0/3578157736 wait complete. 2023-07-13T22:13:09.341 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332133 got 219043332135 for osd.7 2023-07-13T22:13:09.341 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:09.771 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841563 got 283467841565 for osd.9 2023-07-13T22:13:09.771 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:09.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757293 got 163208757297 for osd.5 2023-07-13T22:13:09.793 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 -- 172.21.15.26:0/2562152086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2410c3e0 msgr2=0x7fdc2410e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 --2- 172.21.15.26:0/2562152086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2410c3e0 0x7fdc2410e7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b0a0 tx=0x7fdc1c00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 -- 172.21.15.26:0/2562152086 shutdown_connections 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 --2- 172.21.15.26:0/2562152086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2410c3e0 0x7fdc2410e7c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 --2- 172.21.15.26:0/2562152086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc240728b0 0x7fdc24072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 --2- 172.21.15.26:0/2562152086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240731c0 0x7fdc2410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 -- 172.21.15.26:0/2562152086 >> 172.21.15.26:0/2562152086 conn(0x7fdc240fb660 msgr2=0x7fdc240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 -- 172.21.15.26:0/2562152086 shutdown_connections 2023-07-13T22:13:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.821+0000 7fdc296fe700 1 -- 172.21.15.26:0/2562152086 wait complete. 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 Processor -- start 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 -- start start 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240728b0 0x7fdc24197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc2410c3e0 0x7fdc2419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419cf60 con 0x7fdc240731c0 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419d0d0 con 0x7fdc240728b0 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.822+0000 7fdc296fe700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419d290 con 0x7fdc2410c3e0 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43724/0 (socket says 172.21.15.26:43724) 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 -- 172.21.15.26:0/1977992534 learned_addr learned my addr 172.21.15.26:0/1977992534 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc22ffd700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240728b0 0x7fdc24197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc237fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc2410c3e0 0x7fdc2419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 -- 172.21.15.26:0/1977992534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc2410c3e0 msgr2=0x7fdc2419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc2410c3e0 0x7fdc2419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 -- 172.21.15.26:0/1977992534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240728b0 msgr2=0x7fdc24197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240728b0 0x7fdc24197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 -- 172.21.15.26:0/1977992534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c009d00 con 0x7fdc240731c0 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.823+0000 7fdc227fc700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e9a0 tx=0x7fdc1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.824+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18009790 con 0x7fdc240731c0 2023-07-13T22:13:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.824+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2419d570 con 0x7fdc240731c0 2023-07-13T22:13:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.824+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc18010430 con 0x7fdc240731c0 2023-07-13T22:13:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.824+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18018680 con 0x7fdc240731c0 2023-07-13T22:13:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.824+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc241a4d90 con 0x7fdc240731c0 2023-07-13T22:13:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.825+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc24106d00 con 0x7fdc240731c0 2023-07-13T22:13:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.827+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdc180105a0 con 0x7fdc240731c0 2023-07-13T22:13:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.828+0000 7fdc0bfff700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 0x7fdc0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.828+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc18014070 con 0x7fdc240731c0 2023-07-13T22:13:09.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.828+0000 7fdc22ffd700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 0x7fdc0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:09.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.829+0000 7fdc22ffd700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 0x7fdc0c0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009c60 tx=0x7fdc14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:09.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:09.832+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdc18096080 con 0x7fdc240731c0 2023-07-13T22:13:09.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:09 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3578157736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-13T22:13:09.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:09 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3578157736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-13T22:13:09.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:09 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3578157736' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-13T22:13:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.042+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fdc2404e210 con 0x7fdc240731c0 2023-07-13T22:13:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.042+0000 7fdc0bfff700 1 -- 172.21.15.26:0/1977992534 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fdc180996a0 con 0x7fdc240731c0 2023-07-13T22:13:10.034 INFO:teuthology.orchestra.run.smithi026.stdout:335007449110 2023-07-13T22:13:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 msgr2=0x7fdc0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 0x7fdc0c0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009c60 tx=0x7fdc14009260 comp rx=0 tx=0).stop 2023-07-13T22:13:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 msgr2=0x7fdc24198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e9a0 tx=0x7fdc1800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:13:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 shutdown_connections 2023-07-13T22:13:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc2410c3e0 0x7fdc2419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc240728b0 0x7fdc24197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc0c0a11e0 0x7fdc0c0a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 --2- 172.21.15.26:0/1977992534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc240731c0 0x7fdc24198450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 >> 172.21.15.26:0/1977992534 conn(0x7fdc240fb660 msgr2=0x7fdc24103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 shutdown_connections 2023-07-13T22:13:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:10.045+0000 7fdc296fe700 1 -- 172.21.15.26:0/1977992534 wait complete. 2023-07-13T22:13:10.521 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449107 got 335007449110 for osd.11 2023-07-13T22:13:10.521 DEBUG:teuthology.parallel:result is None 2023-07-13T22:13:10.522 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-13T22:13:10.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph pg dump --format=json 2023-07-13T22:13:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:10 smithi117 ceph-mon[129074]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:10.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:10 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1977992534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-13T22:13:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:10 smithi026 ceph-mon[95647]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:10 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1977992534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-13T22:13:10.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:10 smithi121 ceph-mon[129187]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:10.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:10 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1977992534' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-13T22:13:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:12.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 -- 172.21.15.26:0/1468206415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 msgr2=0x7fc7d410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/1468206415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d410bea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80098f0 tx=0x7fc7c8009c00 comp rx=0 tx=0).stop 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 -- 172.21.15.26:0/1468206415 shutdown_connections 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/1468206415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d410e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/1468206415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/1468206415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d410bea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:12.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 -- 172.21.15.26:0/1468206415 >> 172.21.15.26:0/1468206415 conn(0x7fc7d40fb660 msgr2=0x7fc7d40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:12.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 -- 172.21.15.26:0/1468206415 shutdown_connections 2023-07-13T22:13:12.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.801+0000 7fc7d9b81700 1 -- 172.21.15.26:0/1468206415 wait complete. 2023-07-13T22:13:12.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.802+0000 7fc7d9b81700 1 Processor -- start 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.802+0000 7fc7d9b81700 1 -- start start 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d419cec0 con 0x7fc7d410c3e0 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d419d030 con 0x7fc7d40728b0 2023-07-13T22:13:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d9b81700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d419d1f0 con 0x7fc7d40731c0 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43744/0 (socket says 172.21.15.26:43744) 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 -- 172.21.15.26:0/2302259540 learned_addr learned my addr 172.21.15.26:0/2302259540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d2ffd700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 -- 172.21.15.26:0/2302259540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 msgr2=0x7fc7d41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 -- 172.21.15.26:0/2302259540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 msgr2=0x7fc7d4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d3fff700 1 -- 172.21.15.26:0/2302259540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc009d00 con 0x7fc7d410c3e0 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.803+0000 7fc7d37fe700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d3fff700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e3c0 tx=0x7fc7cc00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0090d0 con 0x7fc7d410c3e0 2023-07-13T22:13:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8009520 con 0x7fc7d410c3e0 2023-07-13T22:13:12.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7cc00f040 con 0x7fc7d410c3e0 2023-07-13T22:13:12.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc007c20 con 0x7fc7d410c3e0 2023-07-13T22:13:12.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.804+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d41a4ad0 con 0x7fc7d410c3e0 2023-07-13T22:13:12.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.805+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d404e210 con 0x7fc7d410c3e0 2023-07-13T22:13:12.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.807+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc7cc003a00 con 0x7fc7d410c3e0 2023-07-13T22:13:12.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.809+0000 7fc7d0ff9700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 0x7fc7bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:12.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.809+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7cc0cc890 con 0x7fc7d410c3e0 2023-07-13T22:13:12.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.809+0000 7fc7d37fe700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 0x7fc7bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:12.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.809+0000 7fc7d37fe700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 0x7fc7bc0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4011490 tx=0x7fc7c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:12.812+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc7cc095ec0 con 0x7fc7d410c3e0 2023-07-13T22:13:12.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:12 smithi117 ceph-mon[129074]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:12.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:12 smithi026 ceph-mon[95647]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:12.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:12 smithi121 ceph-mon[129187]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:13.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.016+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7d40fd2d0 con 0x7fc7bc0a11e0 2023-07-13T22:13:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.028+0000 7fc7d0ff9700 1 -- 172.21.15.26:0/2302259540 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341005 (secure 0 0 0) 0x7fc7d40fd2d0 con 0x7fc7bc0a11e0 2023-07-13T22:13:13.020 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:13.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 msgr2=0x7fc7bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:13.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 0x7fc7bc0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4011490 tx=0x7fc7c4009260 comp rx=0 tx=0).stop 2023-07-13T22:13:13.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 msgr2=0x7fc7d419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:13.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00e3c0 tx=0x7fc7cc00e6d0 comp rx=0 tx=0).stop 2023-07-13T22:13:13.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 shutdown_connections 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7bc0a11e0 0x7fc7bc0a3690 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7d410c3e0 0x7fc7d419c820 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d40728b0 0x7fc7d4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 --2- 172.21.15.26:0/2302259540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7d40731c0 0x7fc7d41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 >> 172.21.15.26:0/2302259540 conn(0x7fc7d40fb660 msgr2=0x7fc7d410e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 shutdown_connections 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:13.033+0000 7fc7d9b81700 1 -- 172.21.15.26:0/2302259540 wait complete. 2023-07-13T22:13:13.027 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:13:13.481 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2023-07-13T22:13:11.433237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654179,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":562,"num_read_kb":407,"num_write":414,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182248,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":13312000,"data_stored":7279457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005365"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538308+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:12:52.538308+0000","last_peered":"2023-07-13T22:12:52.538308+0000","last_clean":"2023-07-13T22:12:52.538308+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:12:52.538308+0000","last_undegraded":"2023-07-13T22:12:52.538308+0000","last_fullsized":"2023-07-13T22:12:52.538308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:47.548617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540988+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:12:52.540988+0000","last_peered":"2023-07-13T22:12:52.540988+0000","last_clean":"2023-07-13T22:12:52.540988+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:12:52.540988+0000","last_undegraded":"2023-07-13T22:12:52.540988+0000","last_fullsized":"2023-07-13T22:12:52.540988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:21.973731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540660+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:12:52.540660+0000","last_peered":"2023-07-13T22:12:52.540660+0000","last_clean":"2023-07-13T22:12:52.540660+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:12:52.540660+0000","last_undegraded":"2023-07-13T22:12:52.540660+0000","last_fullsized":"2023-07-13T22:12:52.540660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:51.412468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541316+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:12:52.541316+0000","last_peered":"2023-07-13T22:12:52.541316+0000","last_clean":"2023-07-13T22:12:52.541316+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:12:52.541316+0000","last_undegraded":"2023-07-13T22:12:52.541316+0000","last_fullsized":"2023-07-13T22:12:52.541316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:33.084972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542596+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:12:52.542596+0000","last_peered":"2023-07-13T22:12:52.542596+0000","last_clean":"2023-07-13T22:12:52.542596+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:12:52.542596+0000","last_undegraded":"2023-07-13T22:12:52.542596+0000","last_fullsized":"2023-07-13T22:12:52.542596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:14:33.505950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:04.653956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540811+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:12:52.540811+0000","last_peered":"2023-07-13T22:12:52.540811+0000","last_clean":"2023-07-13T22:12:52.540811+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:12:52.540811+0000","last_undegraded":"2023-07-13T22:12:52.540811+0000","last_fullsized":"2023-07-13T22:12:52.540811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:06.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791521+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:12:52.791521+0000","last_peered":"2023-07-13T22:12:52.791521+0000","last_clean":"2023-07-13T22:12:52.791521+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:12:52.791521+0000","last_undegraded":"2023-07-13T22:12:52.791521+0000","last_fullsized":"2023-07-13T22:12:52.791521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:49.982151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538742+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:12:52.538742+0000","last_peered":"2023-07-13T22:12:52.538742+0000","last_clean":"2023-07-13T22:12:52.538742+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:12:52.538742+0000","last_undegraded":"2023-07-13T22:12:52.538742+0000","last_fullsized":"2023-07-13T22:12:52.538742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.458063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.546759+0000","last_change":"2023-07-13T22:12:52.546759+0000","last_active":"2023-07-13T22:12:52.546759+0000","last_peered":"2023-07-13T22:12:52.546759+0000","last_clean":"2023-07-13T22:12:52.546759+0000","last_became_active":"2023-07-13T22:12:52.546378+0000","last_became_peered":"2023-07-13T22:12:52.546378+0000","last_unstale":"2023-07-13T22:12:52.546759+0000","last_undegraded":"2023-07-13T22:12:52.546759+0000","last_fullsized":"2023-07-13T22:12:52.546759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:38.494158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541557+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:12:52.541557+0000","last_peered":"2023-07-13T22:12:52.541557+0000","last_clean":"2023-07-13T22:12:52.541557+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:12:52.541557+0000","last_undegraded":"2023-07-13T22:12:52.541557+0000","last_fullsized":"2023-07-13T22:12:52.541557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:47.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540503+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:12:52.540503+0000","last_peered":"2023-07-13T22:12:52.540503+0000","last_clean":"2023-07-13T22:12:52.540503+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:12:52.540503+0000","last_undegraded":"2023-07-13T22:12:52.540503+0000","last_fullsized":"2023-07-13T22:12:52.540503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:55.880996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539581+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:12:52.539581+0000","last_peered":"2023-07-13T22:12:52.539581+0000","last_clean":"2023-07-13T22:12:52.539581+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:12:52.539581+0000","last_undegraded":"2023-07-13T22:12:52.539581+0000","last_fullsized":"2023-07-13T22:12:52.539581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:58.155432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540083+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:12:52.540083+0000","last_peered":"2023-07-13T22:12:52.540083+0000","last_clean":"2023-07-13T22:12:52.540083+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:12:52.540083+0000","last_undegraded":"2023-07-13T22:12:52.540083+0000","last_fullsized":"2023-07-13T22:12:52.540083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:57.803872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.549238+0000","last_change":"2023-07-13T22:12:52.549238+0000","last_active":"2023-07-13T22:12:52.549238+0000","last_peered":"2023-07-13T22:12:52.549238+0000","last_clean":"2023-07-13T22:12:52.549238+0000","last_became_active":"2023-07-13T22:12:52.548923+0000","last_became_peered":"2023-07-13T22:12:52.548923+0000","last_unstale":"2023-07-13T22:12:52.549238+0000","last_undegraded":"2023-07-13T22:12:52.549238+0000","last_fullsized":"2023-07-13T22:12:52.549238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:08.925620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538244+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:12:52.538244+0000","last_peered":"2023-07-13T22:12:52.538244+0000","last_clean":"2023-07-13T22:12:52.538244+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:12:52.538244+0000","last_undegraded":"2023-07-13T22:12:52.538244+0000","last_fullsized":"2023-07-13T22:12:52.538244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:23.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538454+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:12:52.538454+0000","last_peered":"2023-07-13T22:12:52.538454+0000","last_clean":"2023-07-13T22:12:52.538454+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:12:52.538454+0000","last_undegraded":"2023-07-13T22:12:52.538454+0000","last_fullsized":"2023-07-13T22:12:52.538454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:37.216203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540257+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:12:52.540257+0000","last_peered":"2023-07-13T22:12:52.540257+0000","last_clean":"2023-07-13T22:12:52.540257+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:12:52.540257+0000","last_undegraded":"2023-07-13T22:12:52.540257+0000","last_fullsized":"2023-07-13T22:12:52.540257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:18.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540881+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:12:52.540881+0000","last_peered":"2023-07-13T22:12:52.540881+0000","last_clean":"2023-07-13T22:12:52.540881+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:12:52.540881+0000","last_undegraded":"2023-07-13T22:12:52.540881+0000","last_fullsized":"2023-07-13T22:12:52.540881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:56.130000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:08.547651+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:13:08.547651+0000","last_peered":"2023-07-13T22:13:08.547651+0000","last_clean":"2023-07-13T22:13:08.547651+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:13:08.547651+0000","last_undegraded":"2023-07-13T22:13:08.547651+0000","last_fullsized":"2023-07-13T22:13:08.547651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:28:47.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791819+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:12:52.791819+0000","last_peered":"2023-07-13T22:12:52.791819+0000","last_clean":"2023-07-13T22:12:52.791819+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:12:52.791819+0000","last_undegraded":"2023-07-13T22:12:52.791819+0000","last_fullsized":"2023-07-13T22:12:52.791819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:42:35.256971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791902+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:52.791902+0000","last_peered":"2023-07-13T22:12:52.791902+0000","last_clean":"2023-07-13T22:12:52.791902+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:52.791902+0000","last_undegraded":"2023-07-13T22:12:52.791902+0000","last_fullsized":"2023-07-13T22:12:52.791902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:29.131604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541435+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:12:52.541435+0000","last_peered":"2023-07-13T22:12:52.541435+0000","last_clean":"2023-07-13T22:12:52.541435+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:12:52.541435+0000","last_undegraded":"2023-07-13T22:12:52.541435+0000","last_fullsized":"2023-07-13T22:12:52.541435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:53.951535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.554410+0000","last_change":"2023-07-13T22:12:52.554410+0000","last_active":"2023-07-13T22:12:52.554410+0000","last_peered":"2023-07-13T22:12:52.554410+0000","last_clean":"2023-07-13T22:12:52.554410+0000","last_became_active":"2023-07-13T22:12:52.541864+0000","last_became_peered":"2023-07-13T22:12:52.541864+0000","last_unstale":"2023-07-13T22:12:52.554410+0000","last_undegraded":"2023-07-13T22:12:52.554410+0000","last_fullsized":"2023-07-13T22:12:52.554410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:48:29.339493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539904+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:12:52.539904+0000","last_peered":"2023-07-13T22:12:52.539904+0000","last_clean":"2023-07-13T22:12:52.539904+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:12:52.539904+0000","last_undegraded":"2023-07-13T22:12:52.539904+0000","last_fullsized":"2023-07-13T22:12:52.539904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:58.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542998+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:52.542998+0000","last_peered":"2023-07-13T22:12:52.542998+0000","last_clean":"2023-07-13T22:12:52.542998+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:52.542998+0000","last_undegraded":"2023-07-13T22:12:52.542998+0000","last_fullsized":"2023-07-13T22:12:52.542998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:22:28.850188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535310+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:12:52.535310+0000","last_peered":"2023-07-13T22:12:52.535310+0000","last_clean":"2023-07-13T22:12:52.535310+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:12:52.535310+0000","last_undegraded":"2023-07-13T22:12:52.535310+0000","last_fullsized":"2023-07-13T22:12:52.535310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:21.578829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:05.496587+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:13:05.496587+0000","last_peered":"2023-07-13T22:13:05.496587+0000","last_clean":"2023-07-13T22:13:05.496587+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:13:05.496587+0000","last_undegraded":"2023-07-13T22:13:05.496587+0000","last_fullsized":"2023-07-13T22:13:05.496587+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:56:16.055459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542818+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:12:52.542818+0000","last_peered":"2023-07-13T22:12:52.542818+0000","last_clean":"2023-07-13T22:12:52.542818+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:12:52.542818+0000","last_undegraded":"2023-07-13T22:12:52.542818+0000","last_fullsized":"2023-07-13T22:12:52.542818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:28.566300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538988+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:52.538988+0000","last_peered":"2023-07-13T22:12:52.538988+0000","last_clean":"2023-07-13T22:12:52.538988+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:52.538988+0000","last_undegraded":"2023-07-13T22:12:52.538988+0000","last_fullsized":"2023-07-13T22:12:52.538988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:01:40.167436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535211+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:12:52.535211+0000","last_peered":"2023-07-13T22:12:52.535211+0000","last_clean":"2023-07-13T22:12:52.535211+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:12:52.535211+0000","last_undegraded":"2023-07-13T22:12:52.535211+0000","last_fullsized":"2023-07-13T22:12:52.535211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:03.123538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537975+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:52.537975+0000","last_peered":"2023-07-13T22:12:52.537975+0000","last_clean":"2023-07-13T22:12:52.537975+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:52.537975+0000","last_undegraded":"2023-07-13T22:12:52.537975+0000","last_fullsized":"2023-07-13T22:12:52.537975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.547995+0000","last_change":"2023-07-13T22:12:52.546725+0000","last_active":"2023-07-13T22:12:52.547995+0000","last_peered":"2023-07-13T22:12:52.547995+0000","last_clean":"2023-07-13T22:12:52.547995+0000","last_became_active":"2023-07-13T22:12:52.546385+0000","last_became_peered":"2023-07-13T22:12:52.546385+0000","last_unstale":"2023-07-13T22:12:52.547995+0000","last_undegraded":"2023-07-13T22:12:52.547995+0000","last_fullsized":"2023-07-13T22:12:52.547995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:18.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534851+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:52.534851+0000","last_peered":"2023-07-13T22:12:52.534851+0000","last_clean":"2023-07-13T22:12:52.534851+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:52.534851+0000","last_undegraded":"2023-07-13T22:12:52.534851+0000","last_fullsized":"2023-07-13T22:12:52.534851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:06.618468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539516+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:12:52.539516+0000","last_peered":"2023-07-13T22:12:52.539516+0000","last_clean":"2023-07-13T22:12:52.539516+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:12:52.539516+0000","last_undegraded":"2023-07-13T22:12:52.539516+0000","last_fullsized":"2023-07-13T22:12:52.539516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:26.423367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":106,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:06.497482+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:13:06.497482+0000","last_peered":"2023-07-13T22:13:06.497482+0000","last_clean":"2023-07-13T22:13:06.497482+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:13:06.497482+0000","last_undegraded":"2023-07-13T22:13:06.497482+0000","last_fullsized":"2023-07-13T22:13:06.497482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:37.488363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:47.438310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541598+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:52.541598+0000","last_peered":"2023-07-13T22:12:52.541598+0000","last_clean":"2023-07-13T22:12:52.541598+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:52.541598+0000","last_undegraded":"2023-07-13T22:12:52.541598+0000","last_fullsized":"2023-07-13T22:12:52.541598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:00.126919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541337+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:12:52.541337+0000","last_peered":"2023-07-13T22:12:52.541337+0000","last_clean":"2023-07-13T22:12:52.541337+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:12:52.541337+0000","last_undegraded":"2023-07-13T22:12:52.541337+0000","last_fullsized":"2023-07-13T22:12:52.541337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:50.338306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535374+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:52.535374+0000","last_peered":"2023-07-13T22:12:52.535374+0000","last_clean":"2023-07-13T22:12:52.535374+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:52.535374+0000","last_undegraded":"2023-07-13T22:12:52.535374+0000","last_fullsized":"2023-07-13T22:12:52.535374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:36.251918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540187+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:12:52.540187+0000","last_peered":"2023-07-13T22:12:52.540187+0000","last_clean":"2023-07-13T22:12:52.540187+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:12:52.540187+0000","last_undegraded":"2023-07-13T22:12:52.540187+0000","last_fullsized":"2023-07-13T22:12:52.540187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:23.829222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":140,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:06.497618+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:13:06.497618+0000","last_peered":"2023-07-13T22:13:06.497618+0000","last_clean":"2023-07-13T22:13:06.497618+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:13:06.497618+0000","last_undegraded":"2023-07-13T22:13:06.497618+0000","last_fullsized":"2023-07-13T22:13:06.497618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:54:07.569446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56945,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":100,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543338+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:12:52.543338+0000","last_peered":"2023-07-13T22:12:52.543338+0000","last_clean":"2023-07-13T22:12:52.543338+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:12:52.543338+0000","last_undegraded":"2023-07-13T22:12:52.543338+0000","last_fullsized":"2023-07-13T22:12:52.543338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:19.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540584+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:52.540584+0000","last_peered":"2023-07-13T22:12:52.540584+0000","last_clean":"2023-07-13T22:12:52.540584+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:52.540584+0000","last_undegraded":"2023-07-13T22:12:52.540584+0000","last_fullsized":"2023-07-13T22:12:52.540584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:23.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538351+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:12:52.538351+0000","last_peered":"2023-07-13T22:12:52.538351+0000","last_clean":"2023-07-13T22:12:52.538351+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:12:52.538351+0000","last_undegraded":"2023-07-13T22:12:52.538351+0000","last_fullsized":"2023-07-13T22:12:52.538351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:37.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534895+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:52.534895+0000","last_peered":"2023-07-13T22:12:52.534895+0000","last_clean":"2023-07-13T22:12:52.534895+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:52.534895+0000","last_undegraded":"2023-07-13T22:12:52.534895+0000","last_fullsized":"2023-07-13T22:12:52.534895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:03.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541398+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:12:52.541398+0000","last_peered":"2023-07-13T22:12:52.541398+0000","last_clean":"2023-07-13T22:12:52.541398+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:12:52.541398+0000","last_undegraded":"2023-07-13T22:12:52.541398+0000","last_fullsized":"2023-07-13T22:12:52.541398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:29.680386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":69,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:06.498393+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:13:06.498393+0000","last_peered":"2023-07-13T22:13:06.498393+0000","last_clean":"2023-07-13T22:13:06.498393+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:13:06.498393+0000","last_undegraded":"2023-07-13T22:13:06.498393+0000","last_fullsized":"2023-07-13T22:13:06.498393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:24.354790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540323+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:12:52.540323+0000","last_peered":"2023-07-13T22:12:52.540323+0000","last_clean":"2023-07-13T22:12:52.540323+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:12:52.540323+0000","last_undegraded":"2023-07-13T22:12:52.540323+0000","last_fullsized":"2023-07-13T22:12:52.540323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:39.397611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541703+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:52.541703+0000","last_peered":"2023-07-13T22:12:52.541703+0000","last_clean":"2023-07-13T22:12:52.541703+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:52.541703+0000","last_undegraded":"2023-07-13T22:12:52.541703+0000","last_fullsized":"2023-07-13T22:12:52.541703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:54.534152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534934+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:12:52.534934+0000","last_peered":"2023-07-13T22:12:52.534934+0000","last_clean":"2023-07-13T22:12:52.534934+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:12:52.534934+0000","last_undegraded":"2023-07-13T22:12:52.534934+0000","last_fullsized":"2023-07-13T22:12:52.534934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:18:37.092789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543636+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:52.543636+0000","last_peered":"2023-07-13T22:12:52.543636+0000","last_clean":"2023-07-13T22:12:52.543636+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:52.543636+0000","last_undegraded":"2023-07-13T22:12:52.543636+0000","last_fullsized":"2023-07-13T22:12:52.543636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:29:05.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538215+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:12:52.538215+0000","last_peered":"2023-07-13T22:12:52.538215+0000","last_clean":"2023-07-13T22:12:52.538215+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:12:52.538215+0000","last_undegraded":"2023-07-13T22:12:52.538215+0000","last_fullsized":"2023-07-13T22:12:52.538215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:53:20.274354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539898+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:52.539898+0000","last_peered":"2023-07-13T22:12:52.539898+0000","last_clean":"2023-07-13T22:12:52.539898+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:52.539898+0000","last_undegraded":"2023-07-13T22:12:52.539898+0000","last_fullsized":"2023-07-13T22:12:52.539898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:26.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538437+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:12:52.538437+0000","last_peered":"2023-07-13T22:12:52.538437+0000","last_clean":"2023-07-13T22:12:52.538437+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:12:52.538437+0000","last_undegraded":"2023-07-13T22:12:52.538437+0000","last_fullsized":"2023-07-13T22:12:52.538437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:02.628236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:06.545611+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:13:06.545611+0000","last_peered":"2023-07-13T22:13:06.545611+0000","last_clean":"2023-07-13T22:13:06.545611+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:13:06.545611+0000","last_undegraded":"2023-07-13T22:13:06.545611+0000","last_fullsized":"2023-07-13T22:13:06.545611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:22.807561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.546608+0000","last_change":"2023-07-13T22:12:52.546608+0000","last_active":"2023-07-13T22:12:52.546608+0000","last_peered":"2023-07-13T22:12:52.546608+0000","last_clean":"2023-07-13T22:12:52.546608+0000","last_became_active":"2023-07-13T22:12:52.546240+0000","last_became_peered":"2023-07-13T22:12:52.546240+0000","last_unstale":"2023-07-13T22:12:52.546608+0000","last_undegraded":"2023-07-13T22:12:52.546608+0000","last_fullsized":"2023-07-13T22:12:52.546608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:11.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535102+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:12:52.535102+0000","last_peered":"2023-07-13T22:12:52.535102+0000","last_clean":"2023-07-13T22:12:52.535102+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:12:52.535102+0000","last_undegraded":"2023-07-13T22:12:52.535102+0000","last_fullsized":"2023-07-13T22:12:52.535102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:05.891842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:05.496399+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:13:05.496399+0000","last_peered":"2023-07-13T22:13:05.496399+0000","last_clean":"2023-07-13T22:13:05.496399+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:13:05.496399+0000","last_undegraded":"2023-07-13T22:13:05.496399+0000","last_fullsized":"2023-07-13T22:13:05.496399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:09.930750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791509+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:12:52.791509+0000","last_peered":"2023-07-13T22:12:52.791509+0000","last_clean":"2023-07-13T22:12:52.791509+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:12:52.791509+0000","last_undegraded":"2023-07-13T22:12:52.791509+0000","last_fullsized":"2023-07-13T22:12:52.791509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:42:38.987941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539678+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:52.539678+0000","last_peered":"2023-07-13T22:12:52.539678+0000","last_clean":"2023-07-13T22:12:52.539678+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:52.539678+0000","last_undegraded":"2023-07-13T22:12:52.539678+0000","last_fullsized":"2023-07-13T22:12:52.539678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:49.259253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540813+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:52.540813+0000","last_peered":"2023-07-13T22:12:52.540813+0000","last_clean":"2023-07-13T22:12:52.540813+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:52.540813+0000","last_undegraded":"2023-07-13T22:12:52.540813+0000","last_fullsized":"2023-07-13T22:12:52.540813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:11:16.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537681+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:12:52.537681+0000","last_peered":"2023-07-13T22:12:52.537681+0000","last_clean":"2023-07-13T22:12:52.537681+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:12:52.537681+0000","last_undegraded":"2023-07-13T22:12:52.537681+0000","last_fullsized":"2023-07-13T22:12:52.537681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:45.725233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541086+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:52.541086+0000","last_peered":"2023-07-13T22:12:52.541086+0000","last_clean":"2023-07-13T22:12:52.541086+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:52.541086+0000","last_undegraded":"2023-07-13T22:12:52.541086+0000","last_fullsized":"2023-07-13T22:12:52.541086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:50.437984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543196+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:52.543196+0000","last_peered":"2023-07-13T22:12:52.543196+0000","last_clean":"2023-07-13T22:12:52.543196+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:52.543196+0000","last_undegraded":"2023-07-13T22:12:52.543196+0000","last_fullsized":"2023-07-13T22:12:52.543196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:37.229195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538554+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:12:52.538554+0000","last_peered":"2023-07-13T22:12:52.538554+0000","last_clean":"2023-07-13T22:12:52.538554+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:12:52.538554+0000","last_undegraded":"2023-07-13T22:12:52.538554+0000","last_fullsized":"2023-07-13T22:12:52.538554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:01.886536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540121+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:52.540121+0000","last_peered":"2023-07-13T22:12:52.540121+0000","last_clean":"2023-07-13T22:12:52.540121+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:52.540121+0000","last_undegraded":"2023-07-13T22:12:52.540121+0000","last_fullsized":"2023-07-13T22:12:52.540121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:27.067307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791660+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:12:52.791660+0000","last_peered":"2023-07-13T22:12:52.791660+0000","last_clean":"2023-07-13T22:12:52.791660+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:12:52.791660+0000","last_undegraded":"2023-07-13T22:12:52.791660+0000","last_fullsized":"2023-07-13T22:12:52.791660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:24.414406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:15.337161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541105+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:12:52.541105+0000","last_peered":"2023-07-13T22:12:52.541105+0000","last_clean":"2023-07-13T22:12:52.541105+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:12:52.541105+0000","last_undegraded":"2023-07-13T22:12:52.541105+0000","last_fullsized":"2023-07-13T22:12:52.541105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:39.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.551910+0000","last_change":"2023-07-13T22:12:52.551910+0000","last_active":"2023-07-13T22:12:52.551910+0000","last_peered":"2023-07-13T22:12:52.551910+0000","last_clean":"2023-07-13T22:12:52.551910+0000","last_became_active":"2023-07-13T22:12:52.548681+0000","last_became_peered":"2023-07-13T22:12:52.548681+0000","last_unstale":"2023-07-13T22:12:52.551910+0000","last_undegraded":"2023-07-13T22:12:52.551910+0000","last_fullsized":"2023-07-13T22:12:52.551910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:54:10.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541889+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:12:52.541889+0000","last_peered":"2023-07-13T22:12:52.541889+0000","last_clean":"2023-07-13T22:12:52.541889+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:12:52.541889+0000","last_undegraded":"2023-07-13T22:12:52.541889+0000","last_fullsized":"2023-07-13T22:12:52.541889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:53.309826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791341+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:52.791341+0000","last_peered":"2023-07-13T22:12:52.791341+0000","last_clean":"2023-07-13T22:12:52.791341+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:52.791341+0000","last_undegraded":"2023-07-13T22:12:52.791341+0000","last_fullsized":"2023-07-13T22:12:52.791341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:31.189715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537853+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:12:52.537853+0000","last_peered":"2023-07-13T22:12:52.537853+0000","last_clean":"2023-07-13T22:12:52.537853+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:12:52.537853+0000","last_undegraded":"2023-07-13T22:12:52.537853+0000","last_fullsized":"2023-07-13T22:12:52.537853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:22.391535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540832+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:52.540832+0000","last_peered":"2023-07-13T22:12:52.540832+0000","last_clean":"2023-07-13T22:12:52.540832+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:52.540832+0000","last_undegraded":"2023-07-13T22:12:52.540832+0000","last_fullsized":"2023-07-13T22:12:52.540832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:34.701191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791404+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:12:52.791404+0000","last_peered":"2023-07-13T22:12:52.791404+0000","last_clean":"2023-07-13T22:12:52.791404+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:12:52.791404+0000","last_undegraded":"2023-07-13T22:12:52.791404+0000","last_fullsized":"2023-07-13T22:12:52.791404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:26:29.278158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542783+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:52.542783+0000","last_peered":"2023-07-13T22:12:52.542783+0000","last_clean":"2023-07-13T22:12:52.542783+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:52.542783+0000","last_undegraded":"2023-07-13T22:12:52.542783+0000","last_fullsized":"2023-07-13T22:12:52.542783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:04:20.180883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541374+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:12:52.541374+0000","last_peered":"2023-07-13T22:12:52.541374+0000","last_clean":"2023-07-13T22:12:52.541374+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:12:52.541374+0000","last_undegraded":"2023-07-13T22:12:52.541374+0000","last_fullsized":"2023-07-13T22:12:52.541374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:53.859029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:06.572069+0000","last_change":"2023-07-13T22:12:52.555796+0000","last_active":"2023-07-13T22:13:06.572069+0000","last_peered":"2023-07-13T22:13:06.572069+0000","last_clean":"2023-07-13T22:13:06.572069+0000","last_became_active":"2023-07-13T22:12:52.548664+0000","last_became_peered":"2023-07-13T22:12:52.548664+0000","last_unstale":"2023-07-13T22:13:06.572069+0000","last_undegraded":"2023-07-13T22:13:06.572069+0000","last_fullsized":"2023-07-13T22:13:06.572069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:37.846269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541095+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:12:52.541095+0000","last_peered":"2023-07-13T22:12:52.541095+0000","last_clean":"2023-07-13T22:12:52.541095+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:12:52.541095+0000","last_undegraded":"2023-07-13T22:12:52.541095+0000","last_fullsized":"2023-07-13T22:12:52.541095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:58.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535107+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:52.535107+0000","last_peered":"2023-07-13T22:12:52.535107+0000","last_clean":"2023-07-13T22:12:52.535107+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:52.535107+0000","last_undegraded":"2023-07-13T22:12:52.535107+0000","last_fullsized":"2023-07-13T22:12:52.535107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:29.092920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541961+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:12:52.541961+0000","last_peered":"2023-07-13T22:12:52.541961+0000","last_clean":"2023-07-13T22:12:52.541961+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:12:52.541961+0000","last_undegraded":"2023-07-13T22:12:52.541961+0000","last_fullsized":"2023-07-13T22:12:52.541961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:10.233101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538994+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:52.538994+0000","last_peered":"2023-07-13T22:12:52.538994+0000","last_clean":"2023-07-13T22:12:52.538994+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:52.538994+0000","last_undegraded":"2023-07-13T22:12:52.538994+0000","last_fullsized":"2023-07-13T22:12:52.538994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:31.553022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541758+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:12:52.541758+0000","last_peered":"2023-07-13T22:12:52.541758+0000","last_clean":"2023-07-13T22:12:52.541758+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:12:52.541758+0000","last_undegraded":"2023-07-13T22:12:52.541758+0000","last_fullsized":"2023-07-13T22:12:52.541758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:27.141696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538003+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:12:52.538003+0000","last_peered":"2023-07-13T22:12:52.538003+0000","last_clean":"2023-07-13T22:12:52.538003+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:12:52.538003+0000","last_undegraded":"2023-07-13T22:12:52.538003+0000","last_fullsized":"2023-07-13T22:12:52.538003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:12.051633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540360+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:12:52.540360+0000","last_peered":"2023-07-13T22:12:52.540360+0000","last_clean":"2023-07-13T22:12:52.540360+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:12:52.540360+0000","last_undegraded":"2023-07-13T22:12:52.540360+0000","last_fullsized":"2023-07-13T22:12:52.540360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:04:24.267514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540660+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:12:52.540660+0000","last_peered":"2023-07-13T22:12:52.540660+0000","last_clean":"2023-07-13T22:12:52.540660+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:12:52.540660+0000","last_undegraded":"2023-07-13T22:12:52.540660+0000","last_fullsized":"2023-07-13T22:12:52.540660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:21.951262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540451+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:12:52.540451+0000","last_peered":"2023-07-13T22:12:52.540451+0000","last_clean":"2023-07-13T22:12:52.540451+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:12:52.540451+0000","last_undegraded":"2023-07-13T22:12:52.540451+0000","last_fullsized":"2023-07-13T22:12:52.540451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:45.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538676+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:12:52.538676+0000","last_peered":"2023-07-13T22:12:52.538676+0000","last_clean":"2023-07-13T22:12:52.538676+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:12:52.538676+0000","last_undegraded":"2023-07-13T22:12:52.538676+0000","last_fullsized":"2023-07-13T22:12:52.538676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:36.995817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538119+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:12:52.538119+0000","last_peered":"2023-07-13T22:12:52.538119+0000","last_clean":"2023-07-13T22:12:52.538119+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:12:52.538119+0000","last_undegraded":"2023-07-13T22:12:52.538119+0000","last_fullsized":"2023-07-13T22:12:52.538119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:14.395567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541076+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:12:52.541076+0000","last_peered":"2023-07-13T22:12:52.541076+0000","last_clean":"2023-07-13T22:12:52.541076+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:12:52.541076+0000","last_undegraded":"2023-07-13T22:12:52.541076+0000","last_fullsized":"2023-07-13T22:12:52.541076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:32.074362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791573+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:12:52.791573+0000","last_peered":"2023-07-13T22:12:52.791573+0000","last_clean":"2023-07-13T22:12:52.791573+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:12:52.791573+0000","last_undegraded":"2023-07-13T22:12:52.791573+0000","last_fullsized":"2023-07-13T22:12:52.791573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:16:17.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.793042+0000","last_change":"2023-07-13T22:12:52.793042+0000","last_active":"2023-07-13T22:12:52.793042+0000","last_peered":"2023-07-13T22:12:52.793042+0000","last_clean":"2023-07-13T22:12:52.793042+0000","last_became_active":"2023-07-13T22:12:52.792625+0000","last_became_peered":"2023-07-13T22:12:52.792625+0000","last_unstale":"2023-07-13T22:12:52.793042+0000","last_undegraded":"2023-07-13T22:12:52.793042+0000","last_fullsized":"2023-07-13T22:12:52.793042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:37.356733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791400+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:12:52.791400+0000","last_peered":"2023-07-13T22:12:52.791400+0000","last_clean":"2023-07-13T22:12:52.791400+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:12:52.791400+0000","last_undegraded":"2023-07-13T22:12:52.791400+0000","last_fullsized":"2023-07-13T22:12:52.791400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:38.425408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539647+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:12:52.539647+0000","last_peered":"2023-07-13T22:12:52.539647+0000","last_clean":"2023-07-13T22:12:52.539647+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:12:52.539647+0000","last_undegraded":"2023-07-13T22:12:52.539647+0000","last_fullsized":"2023-07-13T22:12:52.539647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:38.511996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539462+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:12:52.539462+0000","last_peered":"2023-07-13T22:12:52.539462+0000","last_clean":"2023-07-13T22:12:52.539462+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:12:52.539462+0000","last_undegraded":"2023-07-13T22:12:52.539462+0000","last_fullsized":"2023-07-13T22:12:52.539462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:16.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537971+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:12:52.537971+0000","last_peered":"2023-07-13T22:12:52.537971+0000","last_clean":"2023-07-13T22:12:52.537971+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:12:52.537971+0000","last_undegraded":"2023-07-13T22:12:52.537971+0000","last_fullsized":"2023-07-13T22:12:52.537971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:02.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63565,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":436,"num_read_kb":298,"num_write":231,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1512,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449110,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68700000000000006}]},{"osd":1,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83999999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82199999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73999999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.84999999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678043,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1092284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.438}]},{"osd":2,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58899999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.3220000000000001,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.746},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63}]},{"osd":5,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.2800000000000002,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.234,"5min":5.234,"15min":5.234},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.81299999999999994}]},{"osd":11,"last update":"Thu Jul 13 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82899999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841566,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.749},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":1.109,"15min":1.109},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":1.101,"15min":1.101},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.0649999999999999,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.018,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89000000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.94199999999999995,"15min":0.95199999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.95599999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.581,"15min":1.581},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":0.999},"min":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.024},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.998}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.98299999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103204,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":444660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71099999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.748},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73599999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.82699999999999996,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.623}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332136,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1036649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.57999999999999996,"5min":1.48,"15min":1.48},"last":0.442},{"interface":"front","average":{"1min":0.432,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.317,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.68200000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89900000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77300000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.36299999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.72499999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.82799999999999996,"5min":0.871,"15min":0.871},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92800000000000005,"5min":1.056,"15min":1.056},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.58899999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94099999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59199999999999997}]},{"osd":8,"last update":"Thu Jul 13 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.625},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.51400000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69399999999999995}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":0.90300000000000002,"15min":0.85099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":10.394,"5min":10.394,"15min":10.394},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.228,"5min":0.874,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":10.452999999999999,"5min":10.452999999999999,"15min":10.452999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83999999999999997,"5min":0.877,"15min":0.877},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80100000000000005,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.49299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94099999999999995,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.085,"15min":1.085},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2010000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.726,"15min":1.726},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.52300000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0620000000000001,"5min":1.369,"15min":1.369},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81899999999999995,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.238,"5min":1.573,"15min":1.573},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.83499999999999996},"min":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0780000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.78000000000000003},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.84699999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.85899999999999999,"15min":0.872},"min":{"1min":0.57799999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0680000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86399999999999999,"15min":0.86799999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0329999999999999,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.91000000000000003,"15min":0.92100000000000004},"min":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93600000000000005,"15min":0.95699999999999996},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1499999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.91300000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.94099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":501809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.36299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.499,"15min":0.46400000000000002},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57599999999999996,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83399999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59699999999999998,"15min":0.54800000000000004},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.79500000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:12:07 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.28799999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.496},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.436,"15min":0.40799999999999997},"min":{"1min":0.31,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.75,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80200000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.94099999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.872}]},{"osd":7,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.53200000000000003,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.42099999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.78700000000000003,"15min":0.74299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6470000000000002,"5min":6.6470000000000002,"15min":6.6470000000000002},"last":0.752}]},{"osd":9,"last update":"Thu Jul 13 22:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82999999999999996},"min":{"1min":0.64800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.042,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.92900000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":1}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.629},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.69799999999999995,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.79300000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.52400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86699999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.70099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.139,"5min":1.331,"15min":1.331},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81499999999999995,"15min":0.79300000000000004},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.95599999999999996,"15min":0.98599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0900000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88500000000000001,"15min":0.90900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0549999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.97999999999999998}]},{"osd":9,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jul 13 22:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.026},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.93600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411390,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78200000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.628},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89500000000000002,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.751},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.88600000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.72499999999999998,"15min":0.67300000000000004},"min":{"1min":0.44600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.4219999999999997,"5min":6.4219999999999997,"15min":6.4219999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60099999999999998,"15min":0.58799999999999997},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Thu Jul 13 22:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.496,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":4.21,"15min":4.21},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.56200000000000006,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84499999999999997}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.8940000000000001,"5min":7.8940000000000001,"15min":7.8940000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":0.60299999999999998,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":13.968999999999999,"5min":13.968999999999999,"15min":13.968999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.74199999999999999,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.0460000000000003,"5min":7.0460000000000003,"15min":7.0460000000000003},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.425,"5min":0.79000000000000004,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.127,"5min":0.755,"15min":0.65600000000000003},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":7.1319999999999997,"5min":7.1319999999999997,"15min":7.1319999999999997},"last":0.91400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.78100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.157,"5min":0.80700000000000005,"15min":0.69799999999999995},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.9950000000000001,"5min":6.9950000000000001,"15min":6.9950000000000001},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.006},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65000000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.62},{"interface":"front","average":{"1min":1.4730000000000001,"5min":0.86899999999999999,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":13.891,"5min":13.891,"15min":13.891},"last":0.75800000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.93700000000000006,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":13.797000000000001,"5min":13.797000000000001,"15min":13.797000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1699999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.93899999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":0.98699999999999999,"15min":0.94399999999999995},"min":{"1min":0.56000000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.2709999999999999,"5min":7.2709999999999999,"15min":7.2709999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.615,"5min":1.0529999999999999,"15min":0.95899999999999996},"min":{"1min":0.57499999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":13.930999999999999,"5min":13.930999999999999,"15min":13.930999999999999},"last":0.80600000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.98199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":445324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48399999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":1.004,"15min":1.004},"last":3.5550000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.49099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":3.5750000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.495,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55300000000000005,"15min":0.53700000000000003},"min":{"1min":0.36399999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.626},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.78300000000000003}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.56499999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4119999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.57499999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.74299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1556480,"data_stored":1036423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.57299999999999995}]},{"osd":1,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81799999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.437}]},{"osd":3,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.288,"15min":1.288},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.69699999999999995,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jul 13 22:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.625,"15min":0.628},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.754}]},{"osd":8,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88100000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67800000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.80200000000000005,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.68899999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:13:13.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph pg dump --format=json 2023-07-13T22:13:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:14.777 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:14 smithi026 ceph-mon[95647]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:14.778 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:14 smithi026 ceph-mon[95647]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:14.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:14 smithi117 ceph-mon[129074]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:14.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:14 smithi117 ceph-mon[129074]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:14.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:14 smithi121 ceph-mon[129187]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:14.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:14 smithi121 ceph-mon[129187]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:15.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 -- 172.21.15.26:0/3976554906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 msgr2=0x7f956c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 --2- 172.21.15.26:0/3976554906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c1001d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f95680099c0 tx=0x7f9568009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 -- 172.21.15.26:0/3976554906 shutdown_connections 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 --2- 172.21.15.26:0/3976554906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c1001d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 --2- 172.21.15.26:0/3976554906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 0x7f956c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 --2- 172.21.15.26:0/3976554906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 0x7f956c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 -- 172.21.15.26:0/3976554906 >> 172.21.15.26:0/3976554906 conn(0x7f956c0fb660 msgr2=0x7f956c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:15.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 -- 172.21.15.26:0/3976554906 shutdown_connections 2023-07-13T22:13:15.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.625+0000 7f9574327700 1 -- 172.21.15.26:0/3976554906 wait complete. 2023-07-13T22:13:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.626+0000 7f9574327700 1 Processor -- start 2023-07-13T22:13:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.626+0000 7f9574327700 1 -- start start 2023-07-13T22:13:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:15.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 0x7f956c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 0x7f956c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c19cec0 con 0x7f956c0ffe00 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c19d030 con 0x7f956c1007a0 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f9574327700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956c19d1f0 con 0x7f956c104ea0 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44348/0 (socket says 172.21.15.26:44348) 2023-07-13T22:13:15.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 -- 172.21.15.26:0/4047330035 learned_addr learned my addr 172.21.15.26:0/4047330035 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95718c2700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 0x7f956c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95728c4700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 0x7f956c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 -- 172.21.15.26:0/4047330035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 msgr2=0x7f956c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 0x7f956c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 -- 172.21.15.26:0/4047330035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 msgr2=0x7f956c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 0x7f956c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.627+0000 7f95720c3700 1 -- 172.21.15.26:0/4047330035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680095f0 con 0x7f956c0ffe00 2023-07-13T22:13:15.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f95720c3700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801d070 con 0x7f956c0ffe00 2023-07-13T22:13:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956c19d470 con 0x7f956c0ffe00 2023-07-13T22:13:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9568003f90 con 0x7f956c0ffe00 2023-07-13T22:13:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568022ba0 con 0x7f956c0ffe00 2023-07-13T22:13:15.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.628+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956c1a4d30 con 0x7f956c0ffe00 2023-07-13T22:13:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.629+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956c04e210 con 0x7f956c0ffe00 2023-07-13T22:13:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.631+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9568003ad0 con 0x7f956c0ffe00 2023-07-13T22:13:15.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.633+0000 7f95637fe700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 0x7f95580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.633+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f95680cdfc0 con 0x7f956c0ffe00 2023-07-13T22:13:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.633+0000 7f95718c2700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 0x7f95580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:15.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.633+0000 7f95718c2700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 0x7f95580a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f956c199520 tx=0x7f955c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:15.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.637+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f95680975f0 con 0x7f956c0ffe00 2023-07-13T22:13:15.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.838+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f956c0fc8a0 con 0x7f95580a11e0 2023-07-13T22:13:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.849+0000 7f95637fe700 1 -- 172.21.15.26:0/4047330035 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+341325 (secure 0 0 0) 0x7f956c0fc8a0 con 0x7f95580a11e0 2023-07-13T22:13:15.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 msgr2=0x7f95580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 0x7f95580a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f956c199520 tx=0x7f955c0073d0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 msgr2=0x7f956c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9568009990 tx=0x7f9568005740 comp rx=0 tx=0).stop 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 shutdown_connections 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f95580a11e0 0x7f95580a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956c0ffe00 0x7f956c197f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f956c104ea0 0x7f956c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 --2- 172.21.15.26:0/4047330035 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f956c1007a0 0x7f956c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 >> 172.21.15.26:0/4047330035 conn(0x7f956c0fb660 msgr2=0x7f956c0fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 shutdown_connections 2023-07-13T22:13:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:15.855+0000 7f9574327700 1 -- 172.21.15.26:0/4047330035 wait complete. 2023-07-13T22:13:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:13:16.331 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-07-13T22:13:15.434312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654179,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":612,"num_read_kb":432,"num_write":414,"num_write_kb":3067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182248,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":341,"ondisk_log_size":341,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":13312000,"data_stored":7279457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003045"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538308+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:12:52.538308+0000","last_peered":"2023-07-13T22:12:52.538308+0000","last_clean":"2023-07-13T22:12:52.538308+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:12:52.538308+0000","last_undegraded":"2023-07-13T22:12:52.538308+0000","last_fullsized":"2023-07-13T22:12:52.538308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:47.548617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540988+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:12:52.540988+0000","last_peered":"2023-07-13T22:12:52.540988+0000","last_clean":"2023-07-13T22:12:52.540988+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:12:52.540988+0000","last_undegraded":"2023-07-13T22:12:52.540988+0000","last_fullsized":"2023-07-13T22:12:52.540988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:21.973731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540660+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:12:52.540660+0000","last_peered":"2023-07-13T22:12:52.540660+0000","last_clean":"2023-07-13T22:12:52.540660+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:12:52.540660+0000","last_undegraded":"2023-07-13T22:12:52.540660+0000","last_fullsized":"2023-07-13T22:12:52.540660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:51.412468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541316+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:12:52.541316+0000","last_peered":"2023-07-13T22:12:52.541316+0000","last_clean":"2023-07-13T22:12:52.541316+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:12:52.541316+0000","last_undegraded":"2023-07-13T22:12:52.541316+0000","last_fullsized":"2023-07-13T22:12:52.541316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:33.084972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542596+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:12:52.542596+0000","last_peered":"2023-07-13T22:12:52.542596+0000","last_clean":"2023-07-13T22:12:52.542596+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:12:52.542596+0000","last_undegraded":"2023-07-13T22:12:52.542596+0000","last_fullsized":"2023-07-13T22:12:52.542596+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:14:33.505950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:04.653956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540811+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:12:52.540811+0000","last_peered":"2023-07-13T22:12:52.540811+0000","last_clean":"2023-07-13T22:12:52.540811+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:12:52.540811+0000","last_undegraded":"2023-07-13T22:12:52.540811+0000","last_fullsized":"2023-07-13T22:12:52.540811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:06.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791521+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:12:52.791521+0000","last_peered":"2023-07-13T22:12:52.791521+0000","last_clean":"2023-07-13T22:12:52.791521+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:12:52.791521+0000","last_undegraded":"2023-07-13T22:12:52.791521+0000","last_fullsized":"2023-07-13T22:12:52.791521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:49.982151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538742+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:12:52.538742+0000","last_peered":"2023-07-13T22:12:52.538742+0000","last_clean":"2023-07-13T22:12:52.538742+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:12:52.538742+0000","last_undegraded":"2023-07-13T22:12:52.538742+0000","last_fullsized":"2023-07-13T22:12:52.538742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.458063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.546759+0000","last_change":"2023-07-13T22:12:52.546759+0000","last_active":"2023-07-13T22:12:52.546759+0000","last_peered":"2023-07-13T22:12:52.546759+0000","last_clean":"2023-07-13T22:12:52.546759+0000","last_became_active":"2023-07-13T22:12:52.546378+0000","last_became_peered":"2023-07-13T22:12:52.546378+0000","last_unstale":"2023-07-13T22:12:52.546759+0000","last_undegraded":"2023-07-13T22:12:52.546759+0000","last_fullsized":"2023-07-13T22:12:52.546759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:38.494158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541557+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:12:52.541557+0000","last_peered":"2023-07-13T22:12:52.541557+0000","last_clean":"2023-07-13T22:12:52.541557+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:12:52.541557+0000","last_undegraded":"2023-07-13T22:12:52.541557+0000","last_fullsized":"2023-07-13T22:12:52.541557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:47.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540503+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:12:52.540503+0000","last_peered":"2023-07-13T22:12:52.540503+0000","last_clean":"2023-07-13T22:12:52.540503+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:12:52.540503+0000","last_undegraded":"2023-07-13T22:12:52.540503+0000","last_fullsized":"2023-07-13T22:12:52.540503+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:55.880996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539581+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:12:52.539581+0000","last_peered":"2023-07-13T22:12:52.539581+0000","last_clean":"2023-07-13T22:12:52.539581+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:12:52.539581+0000","last_undegraded":"2023-07-13T22:12:52.539581+0000","last_fullsized":"2023-07-13T22:12:52.539581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:58.155432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540083+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:12:52.540083+0000","last_peered":"2023-07-13T22:12:52.540083+0000","last_clean":"2023-07-13T22:12:52.540083+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:12:52.540083+0000","last_undegraded":"2023-07-13T22:12:52.540083+0000","last_fullsized":"2023-07-13T22:12:52.540083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:57.803872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.549238+0000","last_change":"2023-07-13T22:12:52.549238+0000","last_active":"2023-07-13T22:12:52.549238+0000","last_peered":"2023-07-13T22:12:52.549238+0000","last_clean":"2023-07-13T22:12:52.549238+0000","last_became_active":"2023-07-13T22:12:52.548923+0000","last_became_peered":"2023-07-13T22:12:52.548923+0000","last_unstale":"2023-07-13T22:12:52.549238+0000","last_undegraded":"2023-07-13T22:12:52.549238+0000","last_fullsized":"2023-07-13T22:12:52.549238+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:08.925620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538244+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:12:52.538244+0000","last_peered":"2023-07-13T22:12:52.538244+0000","last_clean":"2023-07-13T22:12:52.538244+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:12:52.538244+0000","last_undegraded":"2023-07-13T22:12:52.538244+0000","last_fullsized":"2023-07-13T22:12:52.538244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:23.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538454+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:12:52.538454+0000","last_peered":"2023-07-13T22:12:52.538454+0000","last_clean":"2023-07-13T22:12:52.538454+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:12:52.538454+0000","last_undegraded":"2023-07-13T22:12:52.538454+0000","last_fullsized":"2023-07-13T22:12:52.538454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:37.216203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540257+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:12:52.540257+0000","last_peered":"2023-07-13T22:12:52.540257+0000","last_clean":"2023-07-13T22:12:52.540257+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:12:52.540257+0000","last_undegraded":"2023-07-13T22:12:52.540257+0000","last_fullsized":"2023-07-13T22:12:52.540257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:18.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540881+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:12:52.540881+0000","last_peered":"2023-07-13T22:12:52.540881+0000","last_clean":"2023-07-13T22:12:52.540881+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:12:52.540881+0000","last_undegraded":"2023-07-13T22:12:52.540881+0000","last_fullsized":"2023-07-13T22:12:52.540881+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:56.130000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":81,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:13.555164+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:13:13.555164+0000","last_peered":"2023-07-13T22:13:13.555164+0000","last_clean":"2023-07-13T22:13:13.555164+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:13:13.555164+0000","last_undegraded":"2023-07-13T22:13:13.555164+0000","last_fullsized":"2023-07-13T22:13:13.555164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:28:47.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791819+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:12:52.791819+0000","last_peered":"2023-07-13T22:12:52.791819+0000","last_clean":"2023-07-13T22:12:52.791819+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:12:52.791819+0000","last_undegraded":"2023-07-13T22:12:52.791819+0000","last_fullsized":"2023-07-13T22:12:52.791819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:42:35.256971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791902+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:52.791902+0000","last_peered":"2023-07-13T22:12:52.791902+0000","last_clean":"2023-07-13T22:12:52.791902+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:52.791902+0000","last_undegraded":"2023-07-13T22:12:52.791902+0000","last_fullsized":"2023-07-13T22:12:52.791902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:29.131604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541435+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:12:52.541435+0000","last_peered":"2023-07-13T22:12:52.541435+0000","last_clean":"2023-07-13T22:12:52.541435+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:12:52.541435+0000","last_undegraded":"2023-07-13T22:12:52.541435+0000","last_fullsized":"2023-07-13T22:12:52.541435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:53.951535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.554410+0000","last_change":"2023-07-13T22:12:52.554410+0000","last_active":"2023-07-13T22:12:52.554410+0000","last_peered":"2023-07-13T22:12:52.554410+0000","last_clean":"2023-07-13T22:12:52.554410+0000","last_became_active":"2023-07-13T22:12:52.541864+0000","last_became_peered":"2023-07-13T22:12:52.541864+0000","last_unstale":"2023-07-13T22:12:52.554410+0000","last_undegraded":"2023-07-13T22:12:52.554410+0000","last_fullsized":"2023-07-13T22:12:52.554410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:48:29.339493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539904+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:12:52.539904+0000","last_peered":"2023-07-13T22:12:52.539904+0000","last_clean":"2023-07-13T22:12:52.539904+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:12:52.539904+0000","last_undegraded":"2023-07-13T22:12:52.539904+0000","last_fullsized":"2023-07-13T22:12:52.539904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:58.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542998+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:52.542998+0000","last_peered":"2023-07-13T22:12:52.542998+0000","last_clean":"2023-07-13T22:12:52.542998+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:52.542998+0000","last_undegraded":"2023-07-13T22:12:52.542998+0000","last_fullsized":"2023-07-13T22:12:52.542998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:22:28.850188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535310+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:12:52.535310+0000","last_peered":"2023-07-13T22:12:52.535310+0000","last_clean":"2023-07-13T22:12:52.535310+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:12:52.535310+0000","last_undegraded":"2023-07-13T22:12:52.535310+0000","last_fullsized":"2023-07-13T22:12:52.535310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:21.578829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:10.504157+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:13:10.504157+0000","last_peered":"2023-07-13T22:13:10.504157+0000","last_clean":"2023-07-13T22:13:10.504157+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:13:10.504157+0000","last_undegraded":"2023-07-13T22:13:10.504157+0000","last_fullsized":"2023-07-13T22:13:10.504157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:56:16.055459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542818+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:12:52.542818+0000","last_peered":"2023-07-13T22:12:52.542818+0000","last_clean":"2023-07-13T22:12:52.542818+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:12:52.542818+0000","last_undegraded":"2023-07-13T22:12:52.542818+0000","last_fullsized":"2023-07-13T22:12:52.542818+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:28.566300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538988+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:52.538988+0000","last_peered":"2023-07-13T22:12:52.538988+0000","last_clean":"2023-07-13T22:12:52.538988+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:52.538988+0000","last_undegraded":"2023-07-13T22:12:52.538988+0000","last_fullsized":"2023-07-13T22:12:52.538988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:01:40.167436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535211+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:12:52.535211+0000","last_peered":"2023-07-13T22:12:52.535211+0000","last_clean":"2023-07-13T22:12:52.535211+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:12:52.535211+0000","last_undegraded":"2023-07-13T22:12:52.535211+0000","last_fullsized":"2023-07-13T22:12:52.535211+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:03.123538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537975+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:52.537975+0000","last_peered":"2023-07-13T22:12:52.537975+0000","last_clean":"2023-07-13T22:12:52.537975+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:52.537975+0000","last_undegraded":"2023-07-13T22:12:52.537975+0000","last_fullsized":"2023-07-13T22:12:52.537975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.547995+0000","last_change":"2023-07-13T22:12:52.546725+0000","last_active":"2023-07-13T22:12:52.547995+0000","last_peered":"2023-07-13T22:12:52.547995+0000","last_clean":"2023-07-13T22:12:52.547995+0000","last_became_active":"2023-07-13T22:12:52.546385+0000","last_became_peered":"2023-07-13T22:12:52.546385+0000","last_unstale":"2023-07-13T22:12:52.547995+0000","last_undegraded":"2023-07-13T22:12:52.547995+0000","last_fullsized":"2023-07-13T22:12:52.547995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:18.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534851+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:52.534851+0000","last_peered":"2023-07-13T22:12:52.534851+0000","last_clean":"2023-07-13T22:12:52.534851+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:52.534851+0000","last_undegraded":"2023-07-13T22:12:52.534851+0000","last_fullsized":"2023-07-13T22:12:52.534851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:06.618468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539516+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:12:52.539516+0000","last_peered":"2023-07-13T22:12:52.539516+0000","last_clean":"2023-07-13T22:12:52.539516+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:12:52.539516+0000","last_undegraded":"2023-07-13T22:12:52.539516+0000","last_fullsized":"2023-07-13T22:12:52.539516+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:26.423367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:11.505078+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:13:11.505078+0000","last_peered":"2023-07-13T22:13:11.505078+0000","last_clean":"2023-07-13T22:13:11.505078+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:13:11.505078+0000","last_undegraded":"2023-07-13T22:13:11.505078+0000","last_fullsized":"2023-07-13T22:13:11.505078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:37.488363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:47.438310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541598+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:52.541598+0000","last_peered":"2023-07-13T22:12:52.541598+0000","last_clean":"2023-07-13T22:12:52.541598+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:52.541598+0000","last_undegraded":"2023-07-13T22:12:52.541598+0000","last_fullsized":"2023-07-13T22:12:52.541598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:00.126919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541337+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:12:52.541337+0000","last_peered":"2023-07-13T22:12:52.541337+0000","last_clean":"2023-07-13T22:12:52.541337+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:12:52.541337+0000","last_undegraded":"2023-07-13T22:12:52.541337+0000","last_fullsized":"2023-07-13T22:12:52.541337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:50.338306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535374+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:52.535374+0000","last_peered":"2023-07-13T22:12:52.535374+0000","last_clean":"2023-07-13T22:12:52.535374+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:52.535374+0000","last_undegraded":"2023-07-13T22:12:52.535374+0000","last_fullsized":"2023-07-13T22:12:52.535374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:36.251918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540187+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:12:52.540187+0000","last_peered":"2023-07-13T22:12:52.540187+0000","last_clean":"2023-07-13T22:12:52.540187+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:12:52.540187+0000","last_undegraded":"2023-07-13T22:12:52.540187+0000","last_fullsized":"2023-07-13T22:12:52.540187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:23.829222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'29","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:11.504999+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:13:11.504999+0000","last_peered":"2023-07-13T22:13:11.504999+0000","last_clean":"2023-07-13T22:13:11.504999+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:13:11.504999+0000","last_undegraded":"2023-07-13T22:13:11.504999+0000","last_fullsized":"2023-07-13T22:13:11.504999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:54:07.569446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56945,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":105,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543338+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:12:52.543338+0000","last_peered":"2023-07-13T22:12:52.543338+0000","last_clean":"2023-07-13T22:12:52.543338+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:12:52.543338+0000","last_undegraded":"2023-07-13T22:12:52.543338+0000","last_fullsized":"2023-07-13T22:12:52.543338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:19.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540584+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:52.540584+0000","last_peered":"2023-07-13T22:12:52.540584+0000","last_clean":"2023-07-13T22:12:52.540584+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:52.540584+0000","last_undegraded":"2023-07-13T22:12:52.540584+0000","last_fullsized":"2023-07-13T22:12:52.540584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:23.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538351+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:12:52.538351+0000","last_peered":"2023-07-13T22:12:52.538351+0000","last_clean":"2023-07-13T22:12:52.538351+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:12:52.538351+0000","last_undegraded":"2023-07-13T22:12:52.538351+0000","last_fullsized":"2023-07-13T22:12:52.538351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:37.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534895+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:52.534895+0000","last_peered":"2023-07-13T22:12:52.534895+0000","last_clean":"2023-07-13T22:12:52.534895+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:52.534895+0000","last_undegraded":"2023-07-13T22:12:52.534895+0000","last_fullsized":"2023-07-13T22:12:52.534895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:03.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541398+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:12:52.541398+0000","last_peered":"2023-07-13T22:12:52.541398+0000","last_clean":"2023-07-13T22:12:52.541398+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:12:52.541398+0000","last_undegraded":"2023-07-13T22:12:52.541398+0000","last_fullsized":"2023-07-13T22:12:52.541398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:29.680386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:11.505110+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:13:11.505110+0000","last_peered":"2023-07-13T22:13:11.505110+0000","last_clean":"2023-07-13T22:13:11.505110+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:13:11.505110+0000","last_undegraded":"2023-07-13T22:13:11.505110+0000","last_fullsized":"2023-07-13T22:13:11.505110+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:24.354790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540323+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:12:52.540323+0000","last_peered":"2023-07-13T22:12:52.540323+0000","last_clean":"2023-07-13T22:12:52.540323+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:12:52.540323+0000","last_undegraded":"2023-07-13T22:12:52.540323+0000","last_fullsized":"2023-07-13T22:12:52.540323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:39.397611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541703+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:52.541703+0000","last_peered":"2023-07-13T22:12:52.541703+0000","last_clean":"2023-07-13T22:12:52.541703+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:52.541703+0000","last_undegraded":"2023-07-13T22:12:52.541703+0000","last_fullsized":"2023-07-13T22:12:52.541703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:54.534152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534934+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:12:52.534934+0000","last_peered":"2023-07-13T22:12:52.534934+0000","last_clean":"2023-07-13T22:12:52.534934+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:12:52.534934+0000","last_undegraded":"2023-07-13T22:12:52.534934+0000","last_fullsized":"2023-07-13T22:12:52.534934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:18:37.092789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543636+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:52.543636+0000","last_peered":"2023-07-13T22:12:52.543636+0000","last_clean":"2023-07-13T22:12:52.543636+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:52.543636+0000","last_undegraded":"2023-07-13T22:12:52.543636+0000","last_fullsized":"2023-07-13T22:12:52.543636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:29:05.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538215+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:12:52.538215+0000","last_peered":"2023-07-13T22:12:52.538215+0000","last_clean":"2023-07-13T22:12:52.538215+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:12:52.538215+0000","last_undegraded":"2023-07-13T22:12:52.538215+0000","last_fullsized":"2023-07-13T22:12:52.538215+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:53:20.274354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539898+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:52.539898+0000","last_peered":"2023-07-13T22:12:52.539898+0000","last_clean":"2023-07-13T22:12:52.539898+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:52.539898+0000","last_undegraded":"2023-07-13T22:12:52.539898+0000","last_fullsized":"2023-07-13T22:12:52.539898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:26.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538437+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:12:52.538437+0000","last_peered":"2023-07-13T22:12:52.538437+0000","last_clean":"2023-07-13T22:12:52.538437+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:12:52.538437+0000","last_undegraded":"2023-07-13T22:12:52.538437+0000","last_fullsized":"2023-07-13T22:12:52.538437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:02.628236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:11.553034+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:13:11.553034+0000","last_peered":"2023-07-13T22:13:11.553034+0000","last_clean":"2023-07-13T22:13:11.553034+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:13:11.553034+0000","last_undegraded":"2023-07-13T22:13:11.553034+0000","last_fullsized":"2023-07-13T22:13:11.553034+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:22.807561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.546608+0000","last_change":"2023-07-13T22:12:52.546608+0000","last_active":"2023-07-13T22:12:52.546608+0000","last_peered":"2023-07-13T22:12:52.546608+0000","last_clean":"2023-07-13T22:12:52.546608+0000","last_became_active":"2023-07-13T22:12:52.546240+0000","last_became_peered":"2023-07-13T22:12:52.546240+0000","last_unstale":"2023-07-13T22:12:52.546608+0000","last_undegraded":"2023-07-13T22:12:52.546608+0000","last_fullsized":"2023-07-13T22:12:52.546608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:11.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535102+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:12:52.535102+0000","last_peered":"2023-07-13T22:12:52.535102+0000","last_clean":"2023-07-13T22:12:52.535102+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:12:52.535102+0000","last_undegraded":"2023-07-13T22:12:52.535102+0000","last_fullsized":"2023-07-13T22:12:52.535102+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:05.891842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:10.503397+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:13:10.503397+0000","last_peered":"2023-07-13T22:13:10.503397+0000","last_clean":"2023-07-13T22:13:10.503397+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:13:10.503397+0000","last_undegraded":"2023-07-13T22:13:10.503397+0000","last_fullsized":"2023-07-13T22:13:10.503397+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:09.930750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791509+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:12:52.791509+0000","last_peered":"2023-07-13T22:12:52.791509+0000","last_clean":"2023-07-13T22:12:52.791509+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:12:52.791509+0000","last_undegraded":"2023-07-13T22:12:52.791509+0000","last_fullsized":"2023-07-13T22:12:52.791509+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:42:38.987941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539678+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:52.539678+0000","last_peered":"2023-07-13T22:12:52.539678+0000","last_clean":"2023-07-13T22:12:52.539678+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:52.539678+0000","last_undegraded":"2023-07-13T22:12:52.539678+0000","last_fullsized":"2023-07-13T22:12:52.539678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:49.259253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540813+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:52.540813+0000","last_peered":"2023-07-13T22:12:52.540813+0000","last_clean":"2023-07-13T22:12:52.540813+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:52.540813+0000","last_undegraded":"2023-07-13T22:12:52.540813+0000","last_fullsized":"2023-07-13T22:12:52.540813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:11:16.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537681+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:12:52.537681+0000","last_peered":"2023-07-13T22:12:52.537681+0000","last_clean":"2023-07-13T22:12:52.537681+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:12:52.537681+0000","last_undegraded":"2023-07-13T22:12:52.537681+0000","last_fullsized":"2023-07-13T22:12:52.537681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:45.725233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541086+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:52.541086+0000","last_peered":"2023-07-13T22:12:52.541086+0000","last_clean":"2023-07-13T22:12:52.541086+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:52.541086+0000","last_undegraded":"2023-07-13T22:12:52.541086+0000","last_fullsized":"2023-07-13T22:12:52.541086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:50.437984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543196+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:52.543196+0000","last_peered":"2023-07-13T22:12:52.543196+0000","last_clean":"2023-07-13T22:12:52.543196+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:52.543196+0000","last_undegraded":"2023-07-13T22:12:52.543196+0000","last_fullsized":"2023-07-13T22:12:52.543196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:37.229195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538554+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:12:52.538554+0000","last_peered":"2023-07-13T22:12:52.538554+0000","last_clean":"2023-07-13T22:12:52.538554+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:12:52.538554+0000","last_undegraded":"2023-07-13T22:12:52.538554+0000","last_fullsized":"2023-07-13T22:12:52.538554+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:01.886536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'10","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540121+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:52.540121+0000","last_peered":"2023-07-13T22:12:52.540121+0000","last_clean":"2023-07-13T22:12:52.540121+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:52.540121+0000","last_undegraded":"2023-07-13T22:12:52.540121+0000","last_fullsized":"2023-07-13T22:12:52.540121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:27.067307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791660+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:12:52.791660+0000","last_peered":"2023-07-13T22:12:52.791660+0000","last_clean":"2023-07-13T22:12:52.791660+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:12:52.791660+0000","last_undegraded":"2023-07-13T22:12:52.791660+0000","last_fullsized":"2023-07-13T22:12:52.791660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:24.414406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:15.337161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541105+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:12:52.541105+0000","last_peered":"2023-07-13T22:12:52.541105+0000","last_clean":"2023-07-13T22:12:52.541105+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:12:52.541105+0000","last_undegraded":"2023-07-13T22:12:52.541105+0000","last_fullsized":"2023-07-13T22:12:52.541105+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:39.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.551910+0000","last_change":"2023-07-13T22:12:52.551910+0000","last_active":"2023-07-13T22:12:52.551910+0000","last_peered":"2023-07-13T22:12:52.551910+0000","last_clean":"2023-07-13T22:12:52.551910+0000","last_became_active":"2023-07-13T22:12:52.548681+0000","last_became_peered":"2023-07-13T22:12:52.548681+0000","last_unstale":"2023-07-13T22:12:52.551910+0000","last_undegraded":"2023-07-13T22:12:52.551910+0000","last_fullsized":"2023-07-13T22:12:52.551910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:54:10.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541889+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:12:52.541889+0000","last_peered":"2023-07-13T22:12:52.541889+0000","last_clean":"2023-07-13T22:12:52.541889+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:12:52.541889+0000","last_undegraded":"2023-07-13T22:12:52.541889+0000","last_fullsized":"2023-07-13T22:12:52.541889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:53.309826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791341+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:52.791341+0000","last_peered":"2023-07-13T22:12:52.791341+0000","last_clean":"2023-07-13T22:12:52.791341+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:52.791341+0000","last_undegraded":"2023-07-13T22:12:52.791341+0000","last_fullsized":"2023-07-13T22:12:52.791341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:31.189715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537853+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:12:52.537853+0000","last_peered":"2023-07-13T22:12:52.537853+0000","last_clean":"2023-07-13T22:12:52.537853+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:12:52.537853+0000","last_undegraded":"2023-07-13T22:12:52.537853+0000","last_fullsized":"2023-07-13T22:12:52.537853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:22.391535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540832+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:52.540832+0000","last_peered":"2023-07-13T22:12:52.540832+0000","last_clean":"2023-07-13T22:12:52.540832+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:52.540832+0000","last_undegraded":"2023-07-13T22:12:52.540832+0000","last_fullsized":"2023-07-13T22:12:52.540832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:34.701191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791404+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:12:52.791404+0000","last_peered":"2023-07-13T22:12:52.791404+0000","last_clean":"2023-07-13T22:12:52.791404+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:12:52.791404+0000","last_undegraded":"2023-07-13T22:12:52.791404+0000","last_fullsized":"2023-07-13T22:12:52.791404+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:26:29.278158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542783+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:52.542783+0000","last_peered":"2023-07-13T22:12:52.542783+0000","last_clean":"2023-07-13T22:12:52.542783+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:52.542783+0000","last_undegraded":"2023-07-13T22:12:52.542783+0000","last_fullsized":"2023-07-13T22:12:52.542783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:04:20.180883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541374+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:12:52.541374+0000","last_peered":"2023-07-13T22:12:52.541374+0000","last_clean":"2023-07-13T22:12:52.541374+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:12:52.541374+0000","last_undegraded":"2023-07-13T22:12:52.541374+0000","last_fullsized":"2023-07-13T22:12:52.541374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:53.859029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:11.580566+0000","last_change":"2023-07-13T22:12:52.555796+0000","last_active":"2023-07-13T22:13:11.580566+0000","last_peered":"2023-07-13T22:13:11.580566+0000","last_clean":"2023-07-13T22:13:11.580566+0000","last_became_active":"2023-07-13T22:12:52.548664+0000","last_became_peered":"2023-07-13T22:12:52.548664+0000","last_unstale":"2023-07-13T22:13:11.580566+0000","last_undegraded":"2023-07-13T22:13:11.580566+0000","last_fullsized":"2023-07-13T22:13:11.580566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:37.846269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1512,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541095+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:12:52.541095+0000","last_peered":"2023-07-13T22:12:52.541095+0000","last_clean":"2023-07-13T22:12:52.541095+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:12:52.541095+0000","last_undegraded":"2023-07-13T22:12:52.541095+0000","last_fullsized":"2023-07-13T22:12:52.541095+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:58.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535107+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:52.535107+0000","last_peered":"2023-07-13T22:12:52.535107+0000","last_clean":"2023-07-13T22:12:52.535107+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:52.535107+0000","last_undegraded":"2023-07-13T22:12:52.535107+0000","last_fullsized":"2023-07-13T22:12:52.535107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:29.092920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541961+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:12:52.541961+0000","last_peered":"2023-07-13T22:12:52.541961+0000","last_clean":"2023-07-13T22:12:52.541961+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:12:52.541961+0000","last_undegraded":"2023-07-13T22:12:52.541961+0000","last_fullsized":"2023-07-13T22:12:52.541961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:10.233101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538994+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:52.538994+0000","last_peered":"2023-07-13T22:12:52.538994+0000","last_clean":"2023-07-13T22:12:52.538994+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:52.538994+0000","last_undegraded":"2023-07-13T22:12:52.538994+0000","last_fullsized":"2023-07-13T22:12:52.538994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:31.553022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541758+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:12:52.541758+0000","last_peered":"2023-07-13T22:12:52.541758+0000","last_clean":"2023-07-13T22:12:52.541758+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:12:52.541758+0000","last_undegraded":"2023-07-13T22:12:52.541758+0000","last_fullsized":"2023-07-13T22:12:52.541758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:27.141696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538003+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:12:52.538003+0000","last_peered":"2023-07-13T22:12:52.538003+0000","last_clean":"2023-07-13T22:12:52.538003+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:12:52.538003+0000","last_undegraded":"2023-07-13T22:12:52.538003+0000","last_fullsized":"2023-07-13T22:12:52.538003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:12.051633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540360+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:12:52.540360+0000","last_peered":"2023-07-13T22:12:52.540360+0000","last_clean":"2023-07-13T22:12:52.540360+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:12:52.540360+0000","last_undegraded":"2023-07-13T22:12:52.540360+0000","last_fullsized":"2023-07-13T22:12:52.540360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:04:24.267514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540660+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:12:52.540660+0000","last_peered":"2023-07-13T22:12:52.540660+0000","last_clean":"2023-07-13T22:12:52.540660+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:12:52.540660+0000","last_undegraded":"2023-07-13T22:12:52.540660+0000","last_fullsized":"2023-07-13T22:12:52.540660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:21.951262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540451+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:12:52.540451+0000","last_peered":"2023-07-13T22:12:52.540451+0000","last_clean":"2023-07-13T22:12:52.540451+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:12:52.540451+0000","last_undegraded":"2023-07-13T22:12:52.540451+0000","last_fullsized":"2023-07-13T22:12:52.540451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:45.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538676+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:12:52.538676+0000","last_peered":"2023-07-13T22:12:52.538676+0000","last_clean":"2023-07-13T22:12:52.538676+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:12:52.538676+0000","last_undegraded":"2023-07-13T22:12:52.538676+0000","last_fullsized":"2023-07-13T22:12:52.538676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:36.995817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538119+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:12:52.538119+0000","last_peered":"2023-07-13T22:12:52.538119+0000","last_clean":"2023-07-13T22:12:52.538119+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:12:52.538119+0000","last_undegraded":"2023-07-13T22:12:52.538119+0000","last_fullsized":"2023-07-13T22:12:52.538119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:14.395567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541076+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:12:52.541076+0000","last_peered":"2023-07-13T22:12:52.541076+0000","last_clean":"2023-07-13T22:12:52.541076+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:12:52.541076+0000","last_undegraded":"2023-07-13T22:12:52.541076+0000","last_fullsized":"2023-07-13T22:12:52.541076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:32.074362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791573+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:12:52.791573+0000","last_peered":"2023-07-13T22:12:52.791573+0000","last_clean":"2023-07-13T22:12:52.791573+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:12:52.791573+0000","last_undegraded":"2023-07-13T22:12:52.791573+0000","last_fullsized":"2023-07-13T22:12:52.791573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:16:17.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.793042+0000","last_change":"2023-07-13T22:12:52.793042+0000","last_active":"2023-07-13T22:12:52.793042+0000","last_peered":"2023-07-13T22:12:52.793042+0000","last_clean":"2023-07-13T22:12:52.793042+0000","last_became_active":"2023-07-13T22:12:52.792625+0000","last_became_peered":"2023-07-13T22:12:52.792625+0000","last_unstale":"2023-07-13T22:12:52.793042+0000","last_undegraded":"2023-07-13T22:12:52.793042+0000","last_fullsized":"2023-07-13T22:12:52.793042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:37.356733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791400+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:12:52.791400+0000","last_peered":"2023-07-13T22:12:52.791400+0000","last_clean":"2023-07-13T22:12:52.791400+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:12:52.791400+0000","last_undegraded":"2023-07-13T22:12:52.791400+0000","last_fullsized":"2023-07-13T22:12:52.791400+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:38.425408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539647+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:12:52.539647+0000","last_peered":"2023-07-13T22:12:52.539647+0000","last_clean":"2023-07-13T22:12:52.539647+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:12:52.539647+0000","last_undegraded":"2023-07-13T22:12:52.539647+0000","last_fullsized":"2023-07-13T22:12:52.539647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:38.511996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539462+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:12:52.539462+0000","last_peered":"2023-07-13T22:12:52.539462+0000","last_clean":"2023-07-13T22:12:52.539462+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:12:52.539462+0000","last_undegraded":"2023-07-13T22:12:52.539462+0000","last_fullsized":"2023-07-13T22:12:52.539462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:16.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537971+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:12:52.537971+0000","last_peered":"2023-07-13T22:12:52.537971+0000","last_clean":"2023-07-13T22:12:52.537971+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:12:52.537971+0000","last_undegraded":"2023-07-13T22:12:52.537971+0000","last_fullsized":"2023-07-13T22:12:52.537971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:02.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63565,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":486,"num_read_kb":323,"num_write":231,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1512,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":190695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":202,"ondisk_log_size":202,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449111,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":444694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.747}]},{"osd":6,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58599999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":9,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83199999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:12:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.70699999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678044,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1092284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.3220000000000001,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.621}]},{"osd":5,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":5.2800000000000002,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":5.234,"5min":5.234,"15min":5.234},"last":0.79900000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.72699999999999998}]},{"osd":9,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Jul 13 22:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.92000000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841567,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.373},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.5}]},{"osd":1,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":1.109,"15min":1.109},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":1.101,"15min":1.101},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.0649999999999999,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.018,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89000000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.59699999999999998}]},{"osd":7,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.94199999999999995,"15min":0.95199999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.95599999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.581,"15min":1.581},"last":0.626}]},{"osd":8,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":0.999},"min":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.66800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":444660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.752}]},{"osd":1,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63800000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.44900000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88200000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.748},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57499999999999996,"15min":0.57799999999999996},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.93400000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.82699999999999996,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.89800000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.91200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":7,"up_from":51,"seq":219043332137,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1036649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28299999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":1.48,"15min":1.48},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74199999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.72799999999999998,"15min":0.67300000000000004},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":10.412000000000001,"5min":10.412000000000001,"15min":10.412000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.70399999999999996,"15min":0.63500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":7.4909999999999997,"5min":7.4909999999999997,"15min":7.4909999999999997},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86099999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":4.8090000000000002,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.81699999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74099999999999999,"15min":0.72899999999999998},"min":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.747},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.72399999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92300000000000004,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.747,"15min":0.745},"min":{"1min":0.51400000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.97799999999999998}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":0.90300000000000002,"15min":0.85099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":10.394,"5min":10.394,"15min":10.394},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.228,"5min":0.874,"15min":0.81499999999999995},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":10.452999999999999,"5min":10.452999999999999,"15min":10.452999999999999},"last":0.94999999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.998,"5min":1.016,"15min":1.016},"last":0.80000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.51100000000000001,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65800000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.83999999999999997,"5min":0.877,"15min":0.877},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67700000000000005,"15min":0.69499999999999995},"min":{"1min":0.47599999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80100000000000005,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89400000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.49299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94099999999999995,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84399999999999997,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.67300000000000004,"15min":0.68500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.085,"15min":1.085},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2010000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.79400000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.093,"5min":1.726,"15min":1.726},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.52300000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0620000000000001,"5min":1.369,"15min":1.369},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81899999999999995,"15min":0.80800000000000005},"min":{"1min":0.57799999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.238,"5min":1.573,"15min":1.573},"last":0.73799999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.044,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79700000000000004,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.85899999999999999,"15min":0.872},"min":{"1min":0.57799999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0680000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.86399999999999999,"15min":0.86799999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0329999999999999,"5min":1.238,"15min":1.238},"last":0.755}]},{"osd":9,"last update":"Thu Jul 13 22:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.91000000000000003,"15min":0.92100000000000004},"min":{"1min":0.65800000000000003,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93600000000000005,"15min":0.95699999999999996},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1499999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.78000000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.78900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":501809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:12:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.36299999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.499,"15min":0.46400000000000002},"min":{"1min":0.38400000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55300000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:12:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57599999999999996,"15min":0.52200000000000002},"min":{"1min":0.46800000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83399999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59699999999999998,"15min":0.54800000000000004},"min":{"1min":0.44900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.42099999999999999},"min":{"1min":0.41999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55200000000000005,"15min":0.53300000000000003},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56999999999999995,"15min":0.54900000000000004},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80200000000000005,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.94099999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72799999999999998,"15min":0.71499999999999997},"min":{"1min":0.51300000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83699999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.52500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.749},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72299999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.78700000000000003,"15min":0.74299999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":6.6470000000000002,"5min":6.6470000000000002,"15min":6.6470000000000002},"last":0.873}]},{"osd":9,"last update":"Thu Jul 13 22:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.83599999999999997,"15min":0.82999999999999996},"min":{"1min":0.64800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.042,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81799999999999995,"15min":0.81499999999999995},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.90000000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.623,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.85499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":445680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"min":{"1min":0.38200000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.624,"15min":0.55900000000000005},"min":{"1min":0.40000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.58699999999999997},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.47199999999999998},"min":{"1min":0.377,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.58099999999999996,"15min":0.55200000000000005},"min":{"1min":0.432,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78600000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.69799999999999995,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:12:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.749,"15min":0.73599999999999999},"min":{"1min":0.52400000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86699999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72899999999999998,"15min":0.70099999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.139,"5min":1.331,"15min":1.331},"last":0.748},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81499999999999995,"15min":0.79300000000000004},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:12:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.95599999999999996,"15min":0.98599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.0900000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.88500000000000001,"15min":0.90900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.0549999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.58199999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":1.0089999999999999,"15min":1.036},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.94799999999999995,"15min":0.96699999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.76900000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:12:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.625},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.92800000000000005}]},{"osd":11,"last update":"Thu Jul 13 22:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.86399999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82299999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54000000000000004,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.371,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.78200000000000003,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jul 13 22:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65900000000000003,"15min":0.628},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89500000000000002,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.625},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57799999999999996,"15min":0.56000000000000005},"min":{"1min":0.41599999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.83599999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.439}]},{"osd":6,"last update":"Thu Jul 13 22:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.72499999999999998,"15min":0.67300000000000004},"min":{"1min":0.44600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.4219999999999997,"5min":6.4219999999999997,"15min":6.4219999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67700000000000005,"15min":0.66200000000000003},"min":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.60099999999999998,"15min":0.58799999999999997},"min":{"1min":0.35899999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55100000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.83899999999999997,"15min":0.84999999999999998},"min":{"1min":0.496,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.028,"5min":4.21,"15min":4.21},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.56200000000000006,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.64100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.52500000000000002,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":7.8940000000000001,"5min":7.8940000000000001,"15min":7.8940000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.2330000000000001,"5min":0.60299999999999998,"15min":0.48099999999999998},"min":{"1min":0.36899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":13.968999999999999,"5min":13.968999999999999,"15min":13.968999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.74199999999999999,"15min":0.63200000000000001},"min":{"1min":0.48699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":7.0460000000000003,"5min":7.0460000000000003,"15min":7.0460000000000003},"last":0.622},{"interface":"front","average":{"1min":1.425,"5min":0.79000000000000004,"15min":0.64300000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":13.851000000000001,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.621},"min":{"1min":0.52200000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.127,"5min":0.755,"15min":0.65600000000000003},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":7.1319999999999997,"5min":7.1319999999999997,"15min":7.1319999999999997},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.78100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.157,"5min":0.80700000000000005,"15min":0.69799999999999995},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.9950000000000001,"5min":6.9950000000000001,"15min":6.9950000000000001},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:12:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.64300000000000002},"min":{"1min":0.52900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.006},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65000000000000002},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.752},{"interface":"front","average":{"1min":1.4730000000000001,"5min":0.86899999999999999,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":13.891,"5min":13.891,"15min":13.891},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.93700000000000006,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":13.797000000000001,"5min":13.797000000000001,"15min":13.797000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1699999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.85199999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":0.98699999999999999,"15min":0.94399999999999995},"min":{"1min":0.56000000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":7.2709999999999999,"5min":7.2709999999999999,"15min":7.2709999999999999},"last":0.876},{"interface":"front","average":{"1min":1.615,"5min":1.0529999999999999,"15min":0.95899999999999996},"min":{"1min":0.57499999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":13.930999999999999,"5min":13.930999999999999,"15min":13.930999999999999},"last":0.82699999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":445324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.878,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.41399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":1.004,"15min":1.004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.873}]},{"osd":5,"last update":"Thu Jul 13 22:12:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.56399999999999995,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.58499999999999996,"15min":0.56000000000000005},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92000000000000004,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.67200000000000004,"15min":0.65800000000000003},"min":{"1min":0.49099999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.96799999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.57099999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.48099999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.751},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.90900000000000003}]},{"osd":9,"last update":"Thu Jul 13 22:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.82699999999999996,"15min":0.83499999999999996},"min":{"1min":0.56499999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4119999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.77500000000000002,"15min":0.77100000000000002},"min":{"1min":0.57499999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.94499999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757298,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1556480,"data_stored":1036423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.59899999999999998,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.58499999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":3.0009999999999999,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55400000000000005,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.81799999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.41899999999999998,"15min":0.39200000000000002},"min":{"1min":0.28599999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40699999999999997,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.249,"15min":0.249},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":2.9590000000000001,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59299999999999997,"15min":0.58899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:12:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61599999999999999,"15min":0.59199999999999997},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.9159999999999999,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2589999999999999,"5min":1.288,"15min":1.288},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.69699999999999995,"15min":0.65500000000000003},"min":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:12:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58199999999999996,"15min":0.56799999999999995},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.41499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88100000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:12:50 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.80200000000000005,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.77300000000000002,"15min":0.75700000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:12:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:13:16.340 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-13T22:13:16.340 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-13T22:13:16.345 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-13T22:13:16.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph health --format=json 2023-07-13T22:13:16.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:16 smithi117 ceph-mon[129074]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:16.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:16 smithi026 ceph-mon[95647]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:16.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:16 smithi121 ceph-mon[129187]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:17.618 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:17 smithi026 ceph-mon[95647]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:17.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:17 smithi117 ceph-mon[129074]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:17.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:17 smithi121 ceph-mon[129187]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:13:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 -- 172.21.15.26:0/629747678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac0ffe00 msgr2=0x7fb1ac1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:18.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/629747678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac1001d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0099c0 tx=0x7fb19c009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 -- 172.21.15.26:0/629747678 shutdown_connections 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/629747678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac1001d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/629747678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/629747678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 0x7fb1ac104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 -- 172.21.15.26:0/629747678 >> 172.21.15.26:0/629747678 conn(0x7fb1ac0fb660 msgr2=0x7fb1ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 -- 172.21.15.26:0/629747678 shutdown_connections 2023-07-13T22:13:18.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.509+0000 7fb1b1da1700 1 -- 172.21.15.26:0/629747678 wait complete. 2023-07-13T22:13:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.510+0000 7fb1b1da1700 1 Processor -- start 2023-07-13T22:13:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.510+0000 7fb1b1da1700 1 -- start start 2023-07-13T22:13:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 0x7fb1ac072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:18.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1a4140 con 0x7fb1ac104ea0 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1a42b0 con 0x7fb1ac1007a0 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1b1da1700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac1a4470 con 0x7fb1ac0ffe00 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44362/0 (socket says 172.21.15.26:44362) 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 -- 172.21.15.26:0/3490849341 learned_addr learned my addr 172.21.15.26:0/3490849341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:18.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1ab7fe700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1aaffd700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 0x7fb1ac072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 -- 172.21.15.26:0/3490849341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac0ffe00 msgr2=0x7fb1ac072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 -- 172.21.15.26:0/3490849341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 msgr2=0x7fb1ac072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 0x7fb1ac072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.511+0000 7fb1abfff700 1 -- 172.21.15.26:0/3490849341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0095f0 con 0x7fb1ac104ea0 2023-07-13T22:13:18.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1abfff700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400ef30 tx=0x7fb1a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4018070 con 0x7fb1ac104ea0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1a400f040 con 0x7fb1ac104ea0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac1a46f0 con 0x7fb1ac104ea0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a400cd00 con 0x7fb1ac104ea0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.512+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac1a4bf0 con 0x7fb1ac104ea0 2023-07-13T22:13:18.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.513+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac04e210 con 0x7fb1ac104ea0 2023-07-13T22:13:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.515+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb1a4003a00 con 0x7fb1ac104ea0 2023-07-13T22:13:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.516+0000 7fb1a8ff9700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.517+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1a40cc560 con 0x7fb1ac104ea0 2023-07-13T22:13:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.517+0000 7fb1ab7fe700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:18.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.517+0000 7fb1ab7fe700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 0x7fb1940a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009990 tx=0x7fb19c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:18.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.520+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb1a4095b90 con 0x7fb1ac104ea0 2023-07-13T22:13:18.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.777+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb1ac073d40 con 0x7fb1ac104ea0 2023-07-13T22:13:18.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.777+0000 7fb1a8ff9700 1 -- 172.21.15.26:0/3490849341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb1a40991b0 con 0x7fb1ac104ea0 2023-07-13T22:13:18.768 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:18.768 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-13T22:13:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 msgr2=0x7fb1940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 0x7fb1940a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb19c009990 tx=0x7fb19c005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 msgr2=0x7fb1ac0732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400ef30 tx=0x7fb1a400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 shutdown_connections 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1940a11e0 0x7fb1940a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1ac104ea0 0x7fb1ac0732d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb1ac0ffe00 0x7fb1ac072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 --2- 172.21.15.26:0/3490849341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1ac1007a0 0x7fb1ac072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 >> 172.21.15.26:0/3490849341 conn(0x7fb1ac0fb660 msgr2=0x7fb1ac0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 shutdown_connections 2023-07-13T22:13:18.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:18.780+0000 7fb1b1da1700 1 -- 172.21.15.26:0/3490849341 wait complete. 2023-07-13T22:13:18.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:18 smithi117 ceph-mon[129074]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:18.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:18 smithi026 ceph-mon[95647]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:18.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:18 smithi121 ceph-mon[129187]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:19.281 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-13T22:13:19.282 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 -- 172.21.15.26:0/1907753960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 msgr2=0x7f38440fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 --2- 172.21.15.26:0/1907753960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f38440fff10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3840009990 tx=0x7f3840009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 -- 172.21.15.26:0/1907753960 shutdown_connections 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 --2- 172.21.15.26:0/1907753960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f38440fff10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 --2- 172.21.15.26:0/1907753960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844101680 0x7f3844107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 --2- 172.21.15.26:0/1907753960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844100cf0 0x7f3844101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 -- 172.21.15.26:0/1907753960 >> 172.21.15.26:0/1907753960 conn(0x7f38440fb0e0 msgr2=0x7f38440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.501+0000 7f384b7ff700 1 -- 172.21.15.26:0/1907753960 shutdown_connections 2023-07-13T22:13:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.502+0000 7f384b7ff700 1 -- 172.21.15.26:0/1907753960 wait complete. 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.503+0000 7f384b7ff700 1 Processor -- start 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.503+0000 7f384b7ff700 1 -- start start 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100cf0 0x7f3844195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844101680 0x7f384419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384419d3e0 con 0x7f38440ffb00 2023-07-13T22:13:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384419d520 con 0x7f3844100cf0 2023-07-13T22:13:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384b7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384419d6b0 con 0x7f3844101680 2023-07-13T22:13:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44372/0 (socket says 172.21.15.26:44372) 2023-07-13T22:13:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 -- 172.21.15.26:0/2750004644 learned_addr learned my addr 172.21.15.26:0/2750004644 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f3848d9a700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100cf0 0x7f3844195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f3849d9c700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844101680 0x7f384419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 -- 172.21.15.26:0/2750004644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844101680 msgr2=0x7f384419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844101680 0x7f384419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 -- 172.21.15.26:0/2750004644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100cf0 msgr2=0x7f3844195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100cf0 0x7f3844195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.504+0000 7f384959b700 1 -- 172.21.15.26:0/2750004644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c009c40 con 0x7f38440ffb00 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f384959b700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3840004730 tx=0x7f3840009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384001d070 con 0x7f38440ffb00 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3840004cd0 con 0x7f38440ffb00 2023-07-13T22:13:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38400095f0 con 0x7f38440ffb00 2023-07-13T22:13:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840005530 con 0x7f38440ffb00 2023-07-13T22:13:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.505+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384419dbc0 con 0x7f38440ffb00 2023-07-13T22:13:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.506+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384410b340 con 0x7f38440ffb00 2023-07-13T22:13:19.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.508+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f384000fbc0 con 0x7f38440ffb00 2023-07-13T22:13:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.510+0000 7f383a7fc700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 0x7f38300a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.510+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f38400cdd00 con 0x7f38440ffb00 2023-07-13T22:13:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.510+0000 7f3848d9a700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 0x7f38300a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.510+0000 7f3848d9a700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 0x7f38300a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3834009bf0 tx=0x7f3834009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.512+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3840097330 con 0x7f38440ffb00 2023-07-13T22:13:19.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.725+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f384410b530 con 0x7f38440ffb00 2023-07-13T22:13:19.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.727+0000 7f383a7fc700 1 -- 172.21.15.26:0/2750004644 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f384009a950 con 0x7f38440ffb00 2023-07-13T22:13:19.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:19.717 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 msgr2=0x7f38300a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 0x7f38300a3750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3834009bf0 tx=0x7f3834009260 comp rx=0 tx=0).stop 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 msgr2=0x7f3844195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3840004730 tx=0x7f3840009510 comp rx=0 tx=0).stop 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 shutdown_connections 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f38300a12a0 0x7f38300a3750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38440ffb00 0x7f3844195860 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3844101680 0x7f384419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 --2- 172.21.15.26:0/2750004644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3844100cf0 0x7f3844195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 >> 172.21.15.26:0/2750004644 conn(0x7f38440fb0e0 msgr2=0x7f38440fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 shutdown_connections 2023-07-13T22:13:19.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.729+0000 7f384b7ff700 1 -- 172.21.15.26:0/2750004644 wait complete. 2023-07-13T22:13:19.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:13:19.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:13:19.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:19 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3490849341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-13T22:13:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 -- 172.21.15.26:0/920770932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0072f70 msgr2=0x7f6cb00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/920770932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0072f70 0x7f6cb00733e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009990 tx=0x7f6ca0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 -- 172.21.15.26:0/920770932 shutdown_connections 2023-07-13T22:13:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/920770932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb01079a0 0x7f6cb0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/920770932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0074550 0x7f6cb00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/920770932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb0072f70 0x7f6cb00733e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 -- 172.21.15.26:0/920770932 >> 172.21.15.26:0/920770932 conn(0x7f6cb00fb0e0 msgr2=0x7f6cb00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 -- 172.21.15.26:0/920770932 shutdown_connections 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.961+0000 7f6cb81ff700 1 -- 172.21.15.26:0/920770932 wait complete. 2023-07-13T22:13:19.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.962+0000 7f6cb81ff700 1 Processor -- start 2023-07-13T22:13:19.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:19 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3490849341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-13T22:13:19.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.962+0000 7f6cb81ff700 1 -- start start 2023-07-13T22:13:19.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 0x7f6cb0103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 0x7f6cb00ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0100130 con 0x7f6cb0074550 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb01002a0 con 0x7f6cb01079a0 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb81ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0100460 con 0x7f6cb0072f70 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb5f9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb5f9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:58296/0 (socket says 172.21.15.26:58296) 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb679c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 0x7f6cb00ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb5f9b700 1 -- 172.21.15.26:0/4041932784 learned_addr learned my addr 172.21.15.26:0/4041932784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.963+0000 7f6cb579a700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 0x7f6cb0103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 -- 172.21.15.26:0/4041932784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 msgr2=0x7f6cb00ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 0x7f6cb00ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 -- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 msgr2=0x7f6cb0103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:19.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 0x7f6cb0103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 -- 172.21.15.26:0/4041932784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00095f0 con 0x7f6cb0072f70 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb679c700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 0x7f6cb00ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb579a700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 0x7f6cb0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb5f9b700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00e970 tx=0x7f6cac00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac009790 con 0x7f6cb0072f70 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cac00c7e0 con 0x7f6cb0072f70 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0054e0 con 0x7f6cb0072f70 2023-07-13T22:13:19.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb01a3fd0 con 0x7f6cb0072f70 2023-07-13T22:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.964+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb01a4450 con 0x7f6cb0072f70 2023-07-13T22:13:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.965+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb0106050 con 0x7f6cb0072f70 2023-07-13T22:13:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.969+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6cac00c950 con 0x7f6cb0072f70 2023-07-13T22:13:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.970+0000 7f6ca6ffd700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 0x7f6c9c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.971+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cac014070 con 0x7f6cb0072f70 2023-07-13T22:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.971+0000 7f6cb579a700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 0x7f6c9c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.971+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6cac0968f0 con 0x7f6cb0072f70 2023-07-13T22:13:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:19.972+0000 7f6cb579a700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 0x7f6c9c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000b2e0 tx=0x7f6ca0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:19.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:19 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3490849341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-13T22:13:20.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.184+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cb002cb00 con 0x7f6cb0072f70 2023-07-13T22:13:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.185+0000 7f6ca6ffd700 1 -- 172.21.15.26:0/4041932784 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6cac099f10 con 0x7f6cb0072f70 2023-07-13T22:13:20.175 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.187+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 msgr2=0x7f6c9c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.187+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 0x7f6c9c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000b2e0 tx=0x7f6ca0005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.187+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 msgr2=0x7f6cb0102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.187+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00e970 tx=0x7f6cac00ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 shutdown_connections 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cb01079a0 0x7f6cb00ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6cb0072f70 0x7f6cb0102e00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c9c0a1260 0x7f6c9c0a3710 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 --2- 172.21.15.26:0/4041932784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cb0074550 0x7f6cb0103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 >> 172.21.15.26:0/4041932784 conn(0x7f6cb00fb0e0 msgr2=0x7f6cb00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 shutdown_connections 2023-07-13T22:13:20.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.188+0000 7f6cb81ff700 1 -- 172.21.15.26:0/4041932784 wait complete. 2023-07-13T22:13:20.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.425+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3793976280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c01005c0 msgr2=0x7fa0c01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.425+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3793976280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c01005c0 0x7fa0c01029a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc0098f0 tx=0x7fa0bc009c00 comp rx=0 tx=0).stop 2023-07-13T22:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3793976280 shutdown_connections 2023-07-13T22:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3793976280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c0105800 0x7fa0c0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3793976280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c01005c0 0x7fa0c01029a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3793976280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0102ee0 0x7fa0c01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3793976280 >> 172.21.15.26:0/3793976280 conn(0x7fa0c00fa000 msgr2=0x7fa0c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3793976280 shutdown_connections 2023-07-13T22:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.426+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3793976280 wait complete. 2023-07-13T22:13:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 Processor -- start 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 -- start start 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 0x7fa0c0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c0102ee0 0x7fa0c0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 0x7fa0c019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c019f610 con 0x7fa0c01005c0 2023-07-13T22:13:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c019f750 con 0x7fa0c0105800 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.427+0000 7fa0c81ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c019f8e0 con 0x7fa0c0102ee0 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c679c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 0x7fa0c019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 0x7fa0c0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c679c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 0x7fa0c019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40676/0 (socket says 172.21.15.26:40676) 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c679c700 1 -- 172.21.15.26:0/3016937549 learned_addr learned my addr 172.21.15.26:0/3016937549 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c579a700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c0102ee0 0x7fa0c0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 -- 172.21.15.26:0/3016937549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c0102ee0 msgr2=0x7fa0c0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c0102ee0 0x7fa0c0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 -- 172.21.15.26:0/3016937549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 msgr2=0x7fa0c019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 0x7fa0c019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 -- 172.21.15.26:0/3016937549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009520 con 0x7fa0c01005c0 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.428+0000 7fa0c5f9b700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 0x7fa0c0197a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009230 tx=0x7fa0bc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.429+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc01c070 con 0x7fa0c01005c0 2023-07-13T22:13:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.429+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0bc00f7d0 con 0x7fa0c01005c0 2023-07-13T22:13:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.429+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c019fb60 con 0x7fa0c01005c0 2023-07-13T22:13:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.429+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc003a20 con 0x7fa0c01005c0 2023-07-13T22:13:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.429+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c019ffe0 con 0x7fa0c01005c0 2023-07-13T22:13:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.430+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c010b2e0 con 0x7fa0c01005c0 2023-07-13T22:13:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.434+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa0bc020030 con 0x7fa0c01005c0 2023-07-13T22:13:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.435+0000 7fa0b6ffd700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 0x7fa0ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.435+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa0bc0cd860 con 0x7fa0c01005c0 2023-07-13T22:13:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.435+0000 7fa0c579a700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 0x7fa0ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.436+0000 7fa0c579a700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 0x7fa0ac0a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0005fd0 tx=0x7fa0b0009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.443+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa0bc096ef0 con 0x7fa0c01005c0 2023-07-13T22:13:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.667+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0c01a0590 con 0x7fa0c01005c0 2023-07-13T22:13:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.667+0000 7fa0b6ffd700 1 -- 172.21.15.26:0/3016937549 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0bc09a510 con 0x7fa0c01005c0 2023-07-13T22:13:20.657 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 msgr2=0x7fa0ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 0x7fa0ac0a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0005fd0 tx=0x7fa0b0009310 comp rx=0 tx=0).stop 2023-07-13T22:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 msgr2=0x7fa0c0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 0x7fa0c0197a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009230 tx=0x7fa0bc00b790 comp rx=0 tx=0).stop 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 shutdown_connections 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0ac0a1260 0x7fa0ac0a3710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0c01005c0 0x7fa0c0197a90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0c0105800 0x7fa0c019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 --2- 172.21.15.26:0/3016937549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c0102ee0 0x7fa0c0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 >> 172.21.15.26:0/3016937549 conn(0x7fa0c00fa000 msgr2=0x7fa0c0101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 shutdown_connections 2023-07-13T22:13:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.670+0000 7fa0c81ff700 1 -- 172.21.15.26:0/3016937549 wait complete. 2023-07-13T22:13:20.686 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:13:20.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 -- 172.21.15.26:0/2146516575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50840ffae0 msgr2=0x7f50840ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 --2- 172.21.15.26:0/2146516575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50840ffae0 0x7f50840ffef0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5074009990 tx=0x7f5074009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 -- 172.21.15.26:0/2146516575 shutdown_connections 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 --2- 172.21.15.26:0/2146516575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5084101660 0x7f5084107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 --2- 172.21.15.26:0/2146516575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50840ffae0 0x7f50840ffef0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 --2- 172.21.15.26:0/2146516575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084100cd0 0x7f5084101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 -- 172.21.15.26:0/2146516575 >> 172.21.15.26:0/2146516575 conn(0x7f50840fb0e0 msgr2=0x7f50840fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 -- 172.21.15.26:0/2146516575 shutdown_connections 2023-07-13T22:13:20.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.914+0000 7f5088bff700 1 -- 172.21.15.26:0/2146516575 wait complete. 2023-07-13T22:13:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.915+0000 7f5088bff700 1 Processor -- start 2023-07-13T22:13:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.915+0000 7f5088bff700 1 -- start start 2023-07-13T22:13:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5084100cd0 0x7f5084072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084101660 0x7f5084072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840735e0 con 0x7f50840ffae0 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5084073750 con 0x7f5084101660 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5088bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508419f890 con 0x7f5084100cd0 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44440/0 (socket says 172.21.15.26:44440) 2023-07-13T22:13:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 -- 172.21.15.26:0/4258542196 learned_addr learned my addr 172.21.15.26:0/4258542196 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5081d9b700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5084100cd0 0x7f5084072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f5082d9d700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084101660 0x7f5084072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 -- 172.21.15.26:0/4258542196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5084100cd0 msgr2=0x7f5084072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5084100cd0 0x7f5084072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 -- 172.21.15.26:0/4258542196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084101660 msgr2=0x7f5084072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084101660 0x7f5084072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 -- 172.21.15.26:0/4258542196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740095f0 con 0x7f50840ffae0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.916+0000 7f508259c700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5074004710 tx=0x7f5074009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.917+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507401d070 con 0x7f50840ffae0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.917+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5074021410 con 0x7f50840ffae0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.917+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508419fb10 con 0x7f50840ffae0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.917+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074017650 con 0x7f50840ffae0 2023-07-13T22:13:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.917+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508419ff90 con 0x7f50840ffae0 2023-07-13T22:13:20.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.918+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084106a60 con 0x7f50840ffae0 2023-07-13T22:13:20.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.920+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5074021580 con 0x7f50840ffae0 2023-07-13T22:13:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.922+0000 7f50737fe700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 0x7f506c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.922+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50740cdf20 con 0x7f50840ffae0 2023-07-13T22:13:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.922+0000 7f5081d9b700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 0x7f506c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.923+0000 7f5081d9b700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 0x7f506c0a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5078009c60 tx=0x7f5078009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:20.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:20.926+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5074097550 con 0x7f50840ffae0 2023-07-13T22:13:20.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:20 smithi117 ceph-mon[129074]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:20.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:20 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2750004644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:20 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:20 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4041932784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:20 smithi026 ceph-mon[95647]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:20 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2750004644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:20 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:20 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4041932784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:20.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:20 smithi121 ceph-mon[129187]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:20.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:20 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2750004644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:20 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:13:20.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:20 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4041932784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.143+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f508402caa0 con 0x7f50840ffae0 2023-07-13T22:13:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.144+0000 7f50737fe700 1 -- 172.21.15.26:0/4258542196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f507409ab70 con 0x7f50840ffae0 2023-07-13T22:13:21.134 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:13:21.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 msgr2=0x7f506c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 0x7f506c0a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5078009c60 tx=0x7f5078009260 comp rx=0 tx=0).stop 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 msgr2=0x7f50840742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5074004710 tx=0x7f5074009580 comp rx=0 tx=0).stop 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 shutdown_connections 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f506c0a11e0 0x7f506c0a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50840ffae0 0x7f50840742c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5084101660 0x7f5084072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 --2- 172.21.15.26:0/4258542196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5084100cd0 0x7f5084072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.146+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 >> 172.21.15.26:0/4258542196 conn(0x7f50840fb0e0 msgr2=0x7f50840fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.147+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 shutdown_connections 2023-07-13T22:13:21.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.147+0000 7f5088bff700 1 -- 172.21.15.26:0/4258542196 wait complete. 2023-07-13T22:13:21.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:13:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 -- 172.21.15.26:0/3728094216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 msgr2=0x7ff8d40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/3728094216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d40733e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009990 tx=0x7ff8c4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 -- 172.21.15.26:0/3728094216 shutdown_connections 2023-07-13T22:13:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/3728094216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 0x7ff8d40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/3728094216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/3728094216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d40733e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 -- 172.21.15.26:0/3728094216 >> 172.21.15.26:0/3728094216 conn(0x7ff8d40fb0e0 msgr2=0x7ff8d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 -- 172.21.15.26:0/3728094216 shutdown_connections 2023-07-13T22:13:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.388+0000 7ff8dbfff700 1 -- 172.21.15.26:0/3728094216 wait complete. 2023-07-13T22:13:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.389+0000 7ff8dbfff700 1 Processor -- start 2023-07-13T22:13:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 -- start start 2023-07-13T22:13:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d419be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 0x7ff8d419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a3a00 con 0x7ff8d4074550 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a3b40 con 0x7ff8d410c0d0 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8dbfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41a3cd0 con 0x7ff8d4072f70 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8d959a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 0x7ff8d419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8da59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8da59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40726/0 (socket says 172.21.15.26:40726) 2023-07-13T22:13:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.390+0000 7ff8da59c700 1 -- 172.21.15.26:0/2072069492 learned_addr learned my addr 172.21.15.26:0/2072069492 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8d9d9b700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d419be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 -- 172.21.15.26:0/2072069492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 msgr2=0x7ff8d419be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d419be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 -- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 msgr2=0x7ff8d419c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 0x7ff8d419c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 -- 172.21.15.26:0/2072069492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40095f0 con 0x7ff8d410c0d0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8da59c700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00ef30 tx=0x7ff8cc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc018070 con 0x7ff8d410c0d0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8cc00f040 con 0x7ff8d410c0d0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00cd00 con 0x7ff8d410c0d0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.391+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d41a3f50 con 0x7ff8d410c0d0 2023-07-13T22:13:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.392+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d41a4380 con 0x7ff8d410c0d0 2023-07-13T22:13:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.393+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d4101c20 con 0x7ff8d410c0d0 2023-07-13T22:13:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.394+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff8cc003a00 con 0x7ff8d410c0d0 2023-07-13T22:13:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.396+0000 7ff8caffd700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 0x7ff8c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.396+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8cc0cc630 con 0x7ff8d410c0d0 2023-07-13T22:13:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.396+0000 7ff8d9d9b700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 0x7ff8c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.397+0000 7ff8d9d9b700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 0x7ff8c00a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4074920 tx=0x7ff8d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.400+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff8cc095c60 con 0x7ff8d410c0d0 2023-07-13T22:13:21.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.681+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8d402c850 con 0x7ff8d410c0d0 2023-07-13T22:13:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.681+0000 7ff8caffd700 1 -- 172.21.15.26:0/2072069492 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7ff8cc099280 con 0x7ff8d410c0d0 2023-07-13T22:13:21.671 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:21.672 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:13:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 msgr2=0x7ff8c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 0x7ff8c00a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4074920 tx=0x7ff8d0009260 comp rx=0 tx=0).stop 2023-07-13T22:13:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 msgr2=0x7ff8d41a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00ef30 tx=0x7ff8cc00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 shutdown_connections 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d410c0d0 0x7ff8d41a3430 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff8d4072f70 0x7ff8d419be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff8c00a1220 0x7ff8c00a36d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 --2- 172.21.15.26:0/2072069492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d4074550 0x7ff8d419c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 >> 172.21.15.26:0/2072069492 conn(0x7ff8d40fb0e0 msgr2=0x7ff8d40fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 shutdown_connections 2023-07-13T22:13:21.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.684+0000 7ff8dbfff700 1 -- 172.21.15.26:0/2072069492 wait complete. 2023-07-13T22:13:21.676 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:21.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:21.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 -- 172.21.15.26:0/1095251811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90072f70 msgr2=0x7feb900733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 --2- 172.21.15.26:0/1095251811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90072f70 0x7feb900733e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb84009990 tx=0x7feb84009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 -- 172.21.15.26:0/1095251811 shutdown_connections 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 --2- 172.21.15.26:0/1095251811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb9010c0d0 0x7feb9010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 --2- 172.21.15.26:0/1095251811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb90074550 0x7feb900729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 --2- 172.21.15.26:0/1095251811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90072f70 0x7feb900733e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 -- 172.21.15.26:0/1095251811 >> 172.21.15.26:0/1095251811 conn(0x7feb900fb0e0 msgr2=0x7feb900fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:21.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.917+0000 7feb965ff700 1 -- 172.21.15.26:0/1095251811 shutdown_connections 2023-07-13T22:13:21.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.918+0000 7feb965ff700 1 -- 172.21.15.26:0/1095251811 wait complete. 2023-07-13T22:13:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.918+0000 7feb965ff700 1 Processor -- start 2023-07-13T22:13:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 -- start start 2023-07-13T22:13:21.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90074550 0x7feb9019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 0x7feb901a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb901a39f0 con 0x7feb90074550 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb901a3b30 con 0x7feb9010c0d0 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb965ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb901a3cc0 con 0x7feb90072f70 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb8ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:58366/0 (socket says 172.21.15.26:58366) 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb8ffff700 1 -- 172.21.15.26:0/2199319608 learned_addr learned my addr 172.21.15.26:0/2199319608 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb8f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90074550 0x7feb9019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.919+0000 7feb94b9c700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 0x7feb901a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 -- 172.21.15.26:0/2199319608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 msgr2=0x7feb901a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 0x7feb901a3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 -- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90074550 msgr2=0x7feb9019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90074550 0x7feb9019c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:21.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 -- 172.21.15.26:0/2199319608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840095f0 con 0x7feb90072f70 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb94b9c700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 0x7feb901a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8ffff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb8000d950 tx=0x7feb8000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80004220 con 0x7feb90072f70 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb80007620 con 0x7feb90072f70 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb901a3fa0 con 0x7feb90072f70 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80005690 con 0x7feb90072f70 2023-07-13T22:13:21.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.920+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb901a43c0 con 0x7feb90072f70 2023-07-13T22:13:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.921+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90101c20 con 0x7feb90072f70 2023-07-13T22:13:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.923+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feb80004750 con 0x7feb90072f70 2023-07-13T22:13:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.925+0000 7feb8d7fa700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 0x7feb780a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.925+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feb800cc920 con 0x7feb90072f70 2023-07-13T22:13:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.925+0000 7feb8f7fe700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 0x7feb780a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.925+0000 7feb8f7fe700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 0x7feb780a36d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b2e0 tx=0x7feb84005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:21.928+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feb80095ec0 con 0x7feb90072f70 2023-07-13T22:13:21.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:21 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3016937549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:21 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4258542196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:21 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3016937549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:21 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4258542196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:21 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3016937549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:21.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:21 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4258542196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:22.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.154+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb9002cab0 con 0x7feb90072f70 2023-07-13T22:13:22.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.157+0000 7feb8d7fa700 1 -- 172.21.15.26:0/2199319608 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7feb80016020 con 0x7feb90072f70 2023-07-13T22:13:22.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:22.147 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 msgr2=0x7feb780a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 0x7feb780a36d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b2e0 tx=0x7feb84005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 msgr2=0x7feb9019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb8000d950 tx=0x7feb8000dc60 comp rx=0 tx=0).stop 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 shutdown_connections 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb9010c0d0 0x7feb901a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb90072f70 0x7feb9019be70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feb780a1220 0x7feb780a36d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 --2- 172.21.15.26:0/2199319608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb90074550 0x7feb9019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 >> 172.21.15.26:0/2199319608 conn(0x7feb900fb0e0 msgr2=0x7feb900fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 shutdown_connections 2023-07-13T22:13:22.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.159+0000 7feb965ff700 1 -- 172.21.15.26:0/2199319608 wait complete. 2023-07-13T22:13:22.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:13:22.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:13:22.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 -- 172.21.15.26:0/1737219170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc100ce0 msgr2=0x7f5fbc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/1737219170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc101130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009990 tx=0x7f5fb0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 -- 172.21.15.26:0/1737219170 shutdown_connections 2023-07-13T22:13:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/1737219170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc101670 0x7f5fbc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/1737219170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/1737219170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc101130 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 -- 172.21.15.26:0/1737219170 >> 172.21.15.26:0/1737219170 conn(0x7f5fbc0fb130 msgr2=0x7f5fbc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 -- 172.21.15.26:0/1737219170 shutdown_connections 2023-07-13T22:13:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.401+0000 7f5fc11ff700 1 -- 172.21.15.26:0/1737219170 wait complete. 2023-07-13T22:13:22.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.402+0000 7f5fc11ff700 1 Processor -- start 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 -- start start 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc1996e0 con 0x7f5fbc101670 2023-07-13T22:13:22.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc073380 con 0x7f5fbc100ce0 2023-07-13T22:13:22.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fc11ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc073540 con 0x7f5fbc0ffaf0 2023-07-13T22:13:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fba59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fbad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fbb59e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44492/0 (socket says 172.21.15.26:44492) 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.404+0000 7f5fbb59e700 1 -- 172.21.15.26:0/2871525744 learned_addr learned my addr 172.21.15.26:0/2871525744 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 -- 172.21.15.26:0/2871525744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 msgr2=0x7f5fbc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 -- 172.21.15.26:0/2871525744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 msgr2=0x7f5fbc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 -- 172.21.15.26:0/2871525744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00095f0 con 0x7f5fbc101670 2023-07-13T22:13:22.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbad9d700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fba59c700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fbb59e700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400ef30 tx=0x7f5fb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4018070 con 0x7f5fbc101670 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fb400f040 con 0x7f5fbc101670 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.405+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fbc1a4090 con 0x7f5fbc101670 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.406+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb400cdb0 con 0x7f5fbc101670 2023-07-13T22:13:22.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.406+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1a4450 con 0x7f5fbc101670 2023-07-13T22:13:22.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.407+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc10b310 con 0x7f5fbc101670 2023-07-13T22:13:22.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.410+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5fb4003a00 con 0x7f5fbc101670 2023-07-13T22:13:22.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.412+0000 7f5fa3fff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 0x7f5fa40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.412+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fb40cc6a0 con 0x7f5fbc101670 2023-07-13T22:13:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.412+0000 7f5fbad9d700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 0x7f5fa40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.413+0000 7f5fbad9d700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 0x7f5fa40a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc100b40 tx=0x7f5fac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.416+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5fb4095cd0 con 0x7f5fbc101670 2023-07-13T22:13:22.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.667+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fbc1069f0 con 0x7f5fbc101670 2023-07-13T22:13:22.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.667+0000 7f5fa3fff700 1 -- 172.21.15.26:0/2871525744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fb40992f0 con 0x7f5fbc101670 2023-07-13T22:13:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 msgr2=0x7f5fa40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 0x7f5fa40a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc100b40 tx=0x7f5fac009260 comp rx=0 tx=0).stop 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 msgr2=0x7f5fbc072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400ef30 tx=0x7f5fb400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 shutdown_connections 2023-07-13T22:13:22.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fa40a11e0 0x7f5fa40a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fbc101670 0x7f5fbc072e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fbc0ffaf0 0x7f5fbc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 --2- 172.21.15.26:0/2871525744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fbc100ce0 0x7f5fbc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 >> 172.21.15.26:0/2871525744 conn(0x7f5fbc0fb130 msgr2=0x7f5fbc0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 shutdown_connections 2023-07-13T22:13:22.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.670+0000 7f5fc11ff700 1 -- 172.21.15.26:0/2871525744 wait complete. 2023-07-13T22:13:22.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:13:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 -- 172.21.15.26:0/4056093764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 msgr2=0x7f8f980ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/4056093764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f980ffef0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009990 tx=0x7f8f94009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 -- 172.21.15.26:0/4056093764 shutdown_connections 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/4056093764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f98107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/4056093764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f980ffef0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/4056093764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 -- 172.21.15.26:0/4056093764 >> 172.21.15.26:0/4056093764 conn(0x7f8f980fb0e0 msgr2=0x7f8f980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 -- 172.21.15.26:0/4056093764 shutdown_connections 2023-07-13T22:13:22.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.901+0000 7f8f9fdff700 1 -- 172.21.15.26:0/4056093764 wait complete. 2023-07-13T22:13:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.902+0000 7f8f9fdff700 1 Processor -- start 2023-07-13T22:13:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.902+0000 7f8f9fdff700 1 -- start start 2023-07-13T22:13:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f9819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819f620 con 0x7f8f98100cd0 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819f760 con 0x7f8f980ffae0 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9fdff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9819f8f0 con 0x7f8f98101660 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9db9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9db9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59798/0 (socket says 172.21.15.26:59798) 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9db9b700 1 -- 172.21.15.26:0/2338237149 learned_addr learned my addr 172.21.15.26:0/2338237149 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9e39c700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f9819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 -- 172.21.15.26:0/2338237149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 msgr2=0x7f8f9819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f9819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 -- 172.21.15.26:0/2338237149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 msgr2=0x7f8f98197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:22.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9d39a700 1 -- 172.21.15.26:0/2338237149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940095f0 con 0x7f8f98100cd0 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9e39c700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f9819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.903+0000 7f8f9db9b700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f9d39a700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98197fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9f0 tx=0x7f8f8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800c9c0 con 0x7f8f98100cd0 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9819fbd0 con 0x7f8f98100cd0 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f88010430 con 0x7f8f98100cd0 2023-07-13T22:13:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88018680 con 0x7f8f98100cd0 2023-07-13T22:13:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.904+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f981a00e0 con 0x7f8f98100cd0 2023-07-13T22:13:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.905+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f981921c0 con 0x7f8f98100cd0 2023-07-13T22:13:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.907+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8f88010860 con 0x7f8f98100cd0 2023-07-13T22:13:22.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.909+0000 7f8f8effd700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 0x7f8f840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:22.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.909+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8f88014070 con 0x7f8f98100cd0 2023-07-13T22:13:22.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.910+0000 7f8f9db9b700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 0x7f8f840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:22.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.910+0000 7f8f9db9b700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 0x7f8f840a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b2e0 tx=0x7f8f94005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:22.913+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8f88051790 con 0x7f8f98100cd0 2023-07-13T22:13:22.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:22.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2072069492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2199319608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:22 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2072069492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2199319608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:22 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:22.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2072069492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2199319608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:22.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:22.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:22 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.144+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8f9802caa0 con 0x7f8f98100cd0 2023-07-13T22:13:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.144+0000 7f8f8effd700 1 -- 172.21.15.26:0/2338237149 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f88096190 con 0x7f8f98100cd0 2023-07-13T22:13:23.134 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:13:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 msgr2=0x7f8f840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 0x7f8f840a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b2e0 tx=0x7f8f94005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 msgr2=0x7f8f98197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98197fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800e9f0 tx=0x7f8f8800ed00 comp rx=0 tx=0).stop 2023-07-13T22:13:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 shutdown_connections 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f98101660 0x7f8f9819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f980ffae0 0x7f8f98197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8f840a11e0 0x7f8f840a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 --2- 172.21.15.26:0/2338237149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f98100cd0 0x7f8f98197fe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 >> 172.21.15.26:0/2338237149 conn(0x7f8f980fb0e0 msgr2=0x7f8f98106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 shutdown_connections 2023-07-13T22:13:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.147+0000 7f8f9fdff700 1 -- 172.21.15.26:0/2338237149 wait complete. 2023-07-13T22:13:23.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:13:23.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 -- 172.21.15.26:0/2171032833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 msgr2=0x7f23c80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/2171032833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c80729a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009990 tx=0x7f23b8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 -- 172.21.15.26:0/2171032833 shutdown_connections 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/2171032833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c810c0b0 0x7f23c810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/2171032833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c80729a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/2171032833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c8072f70 0x7f23c80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 -- 172.21.15.26:0/2171032833 >> 172.21.15.26:0/2171032833 conn(0x7f23c80fb0e0 msgr2=0x7f23c80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 -- 172.21.15.26:0/2171032833 shutdown_connections 2023-07-13T22:13:23.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.381+0000 7f23cd9ff700 1 -- 172.21.15.26:0/2171032833 wait complete. 2023-07-13T22:13:23.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.382+0000 7f23cd9ff700 1 Processor -- start 2023-07-13T22:13:23.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 -- start start 2023-07-13T22:13:23.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 0x7f23c819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c810c0b0 0x7f23c81a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c81a3a00 con 0x7f23c8072f70 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c81a3b40 con 0x7f23c810c0b0 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23cd9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c81a3cd0 con 0x7f23c8074550 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 0x7f23c819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23c67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23c6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 0x7f23c819be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59024/0 (socket says 172.21.15.26:59024) 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23c67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:59374/0 (socket says 172.21.15.26:59374) 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.383+0000 7f23c6ffd700 1 -- 172.21.15.26:0/3739475195 learned_addr learned my addr 172.21.15.26:0/3739475195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c77fe700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c810c0b0 0x7f23c81a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 -- 172.21.15.26:0/3739475195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c810c0b0 msgr2=0x7f23c81a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c810c0b0 0x7f23c81a3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 -- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 msgr2=0x7f23c819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 0x7f23c819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 -- 172.21.15.26:0/3739475195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80095f0 con 0x7f23c8074550 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23c67fc700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e9c0 tx=0x7f23bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc004220 con 0x7f23c8074550 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23bc004380 con 0x7f23c8074550 2023-07-13T22:13:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c81a3fb0 con 0x7f23c8074550 2023-07-13T22:13:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.384+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003be0 con 0x7f23c8074550 2023-07-13T22:13:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.385+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c81a4430 con 0x7f23c8074550 2023-07-13T22:13:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.386+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c8101c00 con 0x7f23c8074550 2023-07-13T22:13:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.388+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f23bc003d40 con 0x7f23c8074550 2023-07-13T22:13:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.389+0000 7f23cc9fd700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.389+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23bc014070 con 0x7f23c8074550 2023-07-13T22:13:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.389+0000 7f23c6ffd700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.390+0000 7f23c6ffd700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 0x7f23b00a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b2e0 tx=0x7f23b8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.391+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f23bc096970 con 0x7f23c8074550 2023-07-13T22:13:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.625+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23c802c9e0 con 0x7f23c8074550 2023-07-13T22:13:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.629+0000 7f23cc9fd700 1 -- 172.21.15.26:0/3739475195 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23bc099f90 con 0x7f23c8074550 2023-07-13T22:13:23.619 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:13:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 msgr2=0x7f23b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 0x7f23b00a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b2e0 tx=0x7f23b8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 msgr2=0x7f23c819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00e9c0 tx=0x7f23bc00ecd0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 shutdown_connections 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23b00a11e0 0x7f23b00a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23c8072f70 0x7f23c819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23c810c0b0 0x7f23c81a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 --2- 172.21.15.26:0/3739475195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23c8074550 0x7f23c819c3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 >> 172.21.15.26:0/3739475195 conn(0x7f23c80fb0e0 msgr2=0x7f23c80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 shutdown_connections 2023-07-13T22:13:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.632+0000 7f23cd9ff700 1 -- 172.21.15.26:0/3739475195 wait complete. 2023-07-13T22:13:23.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:13:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 -- 172.21.15.26:0/322760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc102ef0 msgr2=0x7f8fbc103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/322760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc103340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009990 tx=0x7f8fb0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 -- 172.21.15.26:0/322760564 shutdown_connections 2023-07-13T22:13:23.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/322760564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc103880 0x7f8fbc10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/322760564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc101d00 0x7f8fbc102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/322760564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc103340 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 -- 172.21.15.26:0/322760564 >> 172.21.15.26:0/322760564 conn(0x7f8fbc0fd320 msgr2=0x7f8fbc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 -- 172.21.15.26:0/322760564 shutdown_connections 2023-07-13T22:13:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.861+0000 7f8fc29ff700 1 -- 172.21.15.26:0/322760564 wait complete. 2023-07-13T22:13:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.862+0000 7f8fc29ff700 1 Processor -- start 2023-07-13T22:13:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.862+0000 7f8fc29ff700 1 -- start start 2023-07-13T22:13:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 0x7f8fbc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc19f590 con 0x7f8fbc101d00 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc19f6d0 con 0x7f8fbc103880 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc29ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc19f860 con 0x7f8fbc102ef0 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59054/0 (socket says 172.21.15.26:59054) 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 -- 172.21.15.26:0/804439078 learned_addr learned my addr 172.21.15.26:0/804439078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbb7fe700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc0f9c700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 0x7f8fbc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 -- 172.21.15.26:0/804439078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 msgr2=0x7f8fbc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 -- 172.21.15.26:0/804439078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 msgr2=0x7f8fbc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 0x7f8fbc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbbfff700 1 -- 172.21.15.26:0/804439078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009520 con 0x7f8fbc101d00 2023-07-13T22:13:23.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fc0f9c700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 0x7f8fbc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.863+0000 7f8fbb7fe700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fbbfff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ea50 tx=0x7f8fac00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00cc90 con 0x7f8fbc101d00 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fac00cdf0 con 0x7f8fbc101d00 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb00095f0 con 0x7f8fbc101d00 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0187a0 con 0x7f8fbc101d00 2023-07-13T22:13:23.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.864+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc19ff10 con 0x7f8fbc101d00 2023-07-13T22:13:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.865+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc109270 con 0x7f8fbc101d00 2023-07-13T22:13:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.867+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8fac018900 con 0x7f8fbc101d00 2023-07-13T22:13:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.868+0000 7f8fb97fa700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 0x7f8fa40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.869+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fac014070 con 0x7f8fbc101d00 2023-07-13T22:13:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.869+0000 7f8fbb7fe700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 0x7f8fa40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:23.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.869+0000 7f8fbb7fe700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 0x7f8fa40a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b2e0 tx=0x7f8fb0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:23.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:23.872+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8fac0966f0 con 0x7f8fbc101d00 2023-07-13T22:13:23.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2871525744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2338237149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:23 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2871525744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2338237149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:23 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2871525744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2338237149' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:23.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:23.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:23 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:24.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.132+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fbc02c850 con 0x7f8fbc101d00 2023-07-13T22:13:24.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.133+0000 7f8fb97fa700 1 -- 172.21.15.26:0/804439078 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f8fac099d10 con 0x7f8fbc101d00 2023-07-13T22:13:24.123 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:24.123 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 msgr2=0x7f8fa40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 0x7f8fa40a3750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b2e0 tx=0x7f8fb0005fd0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 msgr2=0x7f8fbc197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ea50 tx=0x7f8fac00ee10 comp rx=0 tx=0).stop 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 shutdown_connections 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fa40a12a0 0x7f8fa40a3750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fbc101d00 0x7f8fbc197a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc103880 0x7f8fbc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 --2- 172.21.15.26:0/804439078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102ef0 0x7f8fbc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 >> 172.21.15.26:0/804439078 conn(0x7f8fbc0fd320 msgr2=0x7f8fbc1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 shutdown_connections 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.136+0000 7f8fc29ff700 1 -- 172.21.15.26:0/804439078 wait complete. 2023-07-13T22:13:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:24.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:24.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.375+0000 7f903d1ff700 1 -- 172.21.15.26:0/696704731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 msgr2=0x7f9038101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.375+0000 7f903d1ff700 1 --2- 172.21.15.26:0/696704731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038101140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f902c009990 tx=0x7f902c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.375+0000 7f903d1ff700 1 -- 172.21.15.26:0/696704731 shutdown_connections 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.375+0000 7f903d1ff700 1 --2- 172.21.15.26:0/696704731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9038101680 0x7f9038107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.376+0000 7f903d1ff700 1 --2- 172.21.15.26:0/696704731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90380ffb00 0x7f90380fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.376+0000 7f903d1ff700 1 --2- 172.21.15.26:0/696704731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038101140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.376+0000 7f903d1ff700 1 -- 172.21.15.26:0/696704731 >> 172.21.15.26:0/696704731 conn(0x7f90380fb0e0 msgr2=0x7f90380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:24.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.376+0000 7f903d1ff700 1 -- 172.21.15.26:0/696704731 shutdown_connections 2023-07-13T22:13:24.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.376+0000 7f903d1ff700 1 -- 172.21.15.26:0/696704731 wait complete. 2023-07-13T22:13:24.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 Processor -- start 2023-07-13T22:13:24.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 -- start start 2023-07-13T22:13:24.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90380ffb00 0x7f90380742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9038101680 0x7f9038072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380735e0 con 0x7f9038100cf0 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038073750 con 0x7f90380ffb00 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903d1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903819f850 con 0x7f9038101680 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.377+0000 7f903659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59078/0 (socket says 172.21.15.26:59078) 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 -- 172.21.15.26:0/972427195 learned_addr learned my addr 172.21.15.26:0/972427195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f9036d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90380ffb00 0x7f90380742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903759e700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9038101680 0x7f9038072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 -- 172.21.15.26:0/972427195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9038101680 msgr2=0x7f9038072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9038101680 0x7f9038072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 -- 172.21.15.26:0/972427195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90380ffb00 msgr2=0x7f90380742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90380ffb00 0x7f90380742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 -- 172.21.15.26:0/972427195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0095f0 con 0x7f9038100cf0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903659c700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f902c005770 tx=0x7f902c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c01d070 con 0x7f9038100cf0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f903d1ff700 1 -- 172.21.15.26:0/972427195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903819fad0 con 0x7f9038100cf0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902c003f60 con 0x7f9038100cf0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.378+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c022c30 con 0x7f9038100cf0 2023-07-13T22:13:24.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.379+0000 7f903d1ff700 1 -- 172.21.15.26:0/972427195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903819fef0 con 0x7f9038100cf0 2023-07-13T22:13:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.380+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90180052f0 con 0x7f9038100cf0 2023-07-13T22:13:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.382+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f902c0040d0 con 0x7f9038100cf0 2023-07-13T22:13:24.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.383+0000 7f901ffff700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 0x7f90200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.383+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f902c0cdfe0 con 0x7f9038100cf0 2023-07-13T22:13:24.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.383+0000 7f9036d9d700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 0x7f90200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.384+0000 7f9036d9d700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 0x7f90200a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9038100b50 tx=0x7f9028007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:24.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.387+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f902c097670 con 0x7f9038100cf0 2023-07-13T22:13:24.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.602+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9018005160 con 0x7f9038100cf0 2023-07-13T22:13:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.604+0000 7f901ffff700 1 -- 172.21.15.26:0/972427195 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f902c09ac90 con 0x7f9038100cf0 2023-07-13T22:13:24.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:24.595 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 msgr2=0x7f90200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 0x7f90200a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9038100b50 tx=0x7f9028007400 comp rx=0 tx=0).stop 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 msgr2=0x7f9038072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f902c005770 tx=0x7f902c0057a0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 shutdown_connections 2023-07-13T22:13:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9038101680 0x7f9038072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90380ffb00 0x7f90380742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f90200a1220 0x7f90200a36d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 --2- 172.21.15.26:0/972427195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9038100cf0 0x7f9038072900 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 >> 172.21.15.26:0/972427195 conn(0x7f90380fb0e0 msgr2=0x7f90381041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 shutdown_connections 2023-07-13T22:13:24.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.607+0000 7f901dffb700 1 -- 172.21.15.26:0/972427195 wait complete. 2023-07-13T22:13:24.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 -- 172.21.15.26:0/2307963253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71140ffb00 msgr2=0x7f71140fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 --2- 172.21.15.26:0/2307963253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71140ffb00 0x7f71140fff10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7104009990 tx=0x7f7104009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 -- 172.21.15.26:0/2307963253 shutdown_connections 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 --2- 172.21.15.26:0/2307963253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71140ffb00 0x7f71140fff10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 --2- 172.21.15.26:0/2307963253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114101680 0x7f7114107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 --2- 172.21.15.26:0/2307963253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7114100cf0 0x7f7114101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 -- 172.21.15.26:0/2307963253 >> 172.21.15.26:0/2307963253 conn(0x7f71140fb0e0 msgr2=0x7f71140fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 -- 172.21.15.26:0/2307963253 shutdown_connections 2023-07-13T22:13:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.833+0000 7f71197ff700 1 -- 172.21.15.26:0/2307963253 wait complete. 2023-07-13T22:13:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.834+0000 7f71197ff700 1 Processor -- start 2023-07-13T22:13:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.834+0000 7f71197ff700 1 -- start start 2023-07-13T22:13:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.834+0000 7f71197ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71140ffb00 0x7f7114197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.834+0000 7f71197ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114100cf0 0x7f7114197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.834+0000 7f71197ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71197ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711419f5f0 con 0x7f7114101680 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71197ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711419f730 con 0x7f7114100cf0 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71197ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711419f8c0 con 0x7f71140ffb00 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59096/0 (socket says 172.21.15.26:59096) 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 -- 172.21.15.26:0/2427829103 learned_addr learned my addr 172.21.15.26:0/2427829103 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:24.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f7112ffd700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71140ffb00 0x7f7114197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71127fc700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114100cf0 0x7f7114197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 -- 172.21.15.26:0/2427829103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71140ffb00 msgr2=0x7f7114197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71140ffb00 0x7f7114197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 -- 172.21.15.26:0/2427829103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114100cf0 msgr2=0x7f7114197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114100cf0 0x7f7114197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 -- 172.21.15.26:0/2427829103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040095f0 con 0x7f7114101680 2023-07-13T22:13:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.835+0000 7f71137fe700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f710c00ef30 tx=0x7f710c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.836+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c018070 con 0x7f7114101680 2023-07-13T22:13:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.836+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711419fb40 con 0x7f7114101680 2023-07-13T22:13:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.836+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f710c00f040 con 0x7f7114101680 2023-07-13T22:13:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.836+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c00cd00 con 0x7f7114101680 2023-07-13T22:13:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.836+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71141a0050 con 0x7f7114101680 2023-07-13T22:13:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.837+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711410b320 con 0x7f7114101680 2023-07-13T22:13:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.839+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f710c003a00 con 0x7f7114101680 2023-07-13T22:13:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.840+0000 7f70fbfff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 0x7f70fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.841+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f710c0cc5d0 con 0x7f7114101680 2023-07-13T22:13:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.841+0000 7f7112ffd700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 0x7f70fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.841+0000 7f7112ffd700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 0x7f70fc0a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f710400b620 tx=0x7f7104005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:24.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:24.844+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f710c095c00 con 0x7f7114101680 2023-07-13T22:13:24.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:24.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3739475195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/804439078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:13:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:13:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:24.945 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/972427195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3739475195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/804439078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:13:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:13:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:24.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/972427195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3739475195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/804439078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:13:24.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:13:24.996 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/972427195' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:25.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.100+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f711410b510 con 0x7f7114101680 2023-07-13T22:13:25.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.100+0000 7f70fbfff700 1 -- 172.21.15.26:0/2427829103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f710c0b7e60 con 0x7f7114101680 2023-07-13T22:13:25.090 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:25.091 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 msgr2=0x7f70fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 0x7f70fc0a36d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f710400b620 tx=0x7f7104005fd0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 msgr2=0x7f711419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f710c00ef30 tx=0x7f710c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 shutdown_connections 2023-07-13T22:13:25.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70fc0a1220 0x7f70fc0a36d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7114101680 0x7f711419f020 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71140ffb00 0x7f7114197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 --2- 172.21.15.26:0/2427829103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7114100cf0 0x7f7114197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 >> 172.21.15.26:0/2427829103 conn(0x7f71140fb0e0 msgr2=0x7f71140fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 shutdown_connections 2023-07-13T22:13:25.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.103+0000 7f71197ff700 1 -- 172.21.15.26:0/2427829103 wait complete. 2023-07-13T22:13:25.095 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:25.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-13T22:12:18.516293+0000', 'modified': '2023-07-13T22:12:26.449971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24575, 'mds_1': 24563, 'mds_2': 24530, 'mds_3': 24493, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14520': {'gid': 14520, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6839/1019342380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 1019342380}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 1019342380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6833/930793918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 930793918}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 930793918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.117:6833/2196716047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 2196716047}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 2196716047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.121:6835/59384395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 59384395}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 59384395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/1505729647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1505729647}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1505729647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/1862924316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1862924316}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1862924316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6839/3259819874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 3259819874}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 3259819874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.117:6837/1128080760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 1128080760}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1128080760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6841/2542800089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2542800089}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2542800089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6839/1698058160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1698058160}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1698058160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:13:25.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-13T22:13:25.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-07-13T22:13:25.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 -- 172.21.15.26:0/303080648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef0100cd0 msgr2=0x7faef0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 --2- 172.21.15.26:0/303080648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef0100cd0 0x7faef0101120 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faed8009990 tx=0x7faed8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 -- 172.21.15.26:0/303080648 shutdown_connections 2023-07-13T22:13:25.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 --2- 172.21.15.26:0/303080648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 0x7faef0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 --2- 172.21.15.26:0/303080648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef00ffae0 0x7faef00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 --2- 172.21.15.26:0/303080648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef0100cd0 0x7faef0101120 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 -- 172.21.15.26:0/303080648 >> 172.21.15.26:0/303080648 conn(0x7faef00fb0e0 msgr2=0x7faef00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 -- 172.21.15.26:0/303080648 shutdown_connections 2023-07-13T22:13:25.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.327+0000 7faef49ff700 1 -- 172.21.15.26:0/303080648 wait complete. 2023-07-13T22:13:25.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.328+0000 7faef49ff700 1 Processor -- start 2023-07-13T22:13:25.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.328+0000 7faef49ff700 1 -- start start 2023-07-13T22:13:25.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 0x7faef0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 0x7faef019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019f610 con 0x7faef00ffae0 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019f750 con 0x7faef0101660 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faef49ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef019f8e0 con 0x7faef0100cd0 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59106/0 (socket says 172.21.15.26:59106) 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 -- 172.21.15.26:0/1895522728 learned_addr learned my addr 172.21.15.26:0/1895522728 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faee7fff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 0x7faef0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeeed9d700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 0x7faef019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 -- 172.21.15.26:0/1895522728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 msgr2=0x7faef0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 0x7faef0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 -- 172.21.15.26:0/1895522728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 msgr2=0x7faef019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 0x7faef019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 -- 172.21.15.26:0/1895522728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8009c40 con 0x7faef00ffae0 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faee7fff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 0x7faef0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:25.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.329+0000 7faeee59c700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faee000d980 tx=0x7faee000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.330+0000 7faee77fe700 1 -- 172.21.15.26:0/1895522728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000de60 con 0x7faef00ffae0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.330+0000 7faee77fe700 1 -- 172.21.15.26:0/1895522728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee00044e0 con 0x7faef00ffae0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.330+0000 7faee77fe700 1 -- 172.21.15.26:0/1895522728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0003be0 con 0x7faef00ffae0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.330+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed80095f0 con 0x7faef00ffae0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.330+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef019ff40 con 0x7faef00ffae0 2023-07-13T22:13:25.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.334+0000 7faee77fe700 1 -- 172.21.15.26:0/1895522728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faee0009bb0 con 0x7faef00ffae0 2023-07-13T22:13:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.335+0000 7faf08321b80 1 Processor -- start 2023-07-13T22:13:25.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.335+0000 7faf08321b80 1 -- start start 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.335+0000 7faee77fe700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 0x7faedc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee77fe700 1 -- 172.21.15.26:0/1895522728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faee00cc820 con 0x7faef00ffae0 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e28690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d2cc60 con 0x5637d1e27dd0 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d7f9f0 con 0x5637d1e27480 2023-07-13T22:13:25.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faf08321b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d78e80 con 0x5637d1e28c20 2023-07-13T22:13:25.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeeed9d700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 0x7faedc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e28690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59882/0 (socket says 172.21.15.26:59882) 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeee59c700 1 -- 172.21.15.26:0/3210238067 learned_addr learned my addr 172.21.15.26:0/3210238067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeeed9d700 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 -- 172.21.15.26:0/3210238067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 msgr2=0x5637d1e29090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e29090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 -- 172.21.15.26:0/3210238067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 msgr2=0x5637d1e27890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faee7fff700 1 -- 172.21.15.26:0/3210238067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d1ba6d90 con 0x5637d1e27dd0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.336+0000 7faeee59c700 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.337+0000 7faeeed9d700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 0x7faedc0a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faee8009550 tx=0x7faee8007bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.337+0000 7faee7fff700 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e28690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5060296f710161c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.337+0000 7faee4ff9700 1 -- 172.21.15.26:0/3210238067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800baf0 con 0x5637d1e27dd0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.337+0000 7faee4ff9700 1 -- 172.21.15.26:0/3210238067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed801e040 con 0x5637d1e27dd0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.337+0000 7faee4ff9700 1 -- 172.21.15.26:0/3210238067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80053b0 con 0x5637d1e27dd0 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- 172.21.15.26:0/3210238067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 msgr2=0x5637d1e28690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e28690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faed8000c00 tx=0x7faed8005c00 comp rx=0 tx=0).stop 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- 172.21.15.26:0/3210238067 shutdown_connections 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e29090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 --2- 172.21.15.26:0/3210238067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e28690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- 172.21.15.26:0/3210238067 >> 172.21.15.26:0/3210238067 conn(0x5637d1e1cd50 msgr2=0x5637d1e22180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- 172.21.15.26:0/3210238067 shutdown_connections 2023-07-13T22:13:25.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- 172.21.15.26:0/3210238067 wait complete. 2023-07-13T22:13:25.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.338+0000 7faf08321b80 1 -- start start 2023-07-13T22:13:25.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 Processor -- start 2023-07-13T22:13:25.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e31060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d18a70 con 0x5637d1e27dd0 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d18770 con 0x5637d1e27480 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1bfc8d0 con 0x5637d1e28c20 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faf08321b80 10 client.0 _ll_register_callbacks cb 0x5637d1d16710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faee7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faee7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59112/0 (socket says 172.21.15.26:59112) 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.339+0000 7faee7fff700 1 -- 172.21.15.26:0/1966825251 learned_addr learned my addr 172.21.15.26:0/1966825251 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faeee59c700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e31060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faeeed9d700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 -- 172.21.15.26:0/1966825251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 msgr2=0x5637d1e385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 -- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 msgr2=0x5637d1e31060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e31060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 -- 172.21.15.26:0/1966825251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef019ff40 con 0x5637d1e27dd0 2023-07-13T22:13:25.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faee7fff700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faed80098e0 tx=0x7faed8005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af60a419260d584 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80047e0 con 0x5637d1e27dd0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637d1cd24a0 con 0x5637d1e27dd0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.340+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faed8004940 con 0x5637d1e27dd0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed8004b00 con 0x5637d1e27dd0 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 10 client.14946 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7faef0108850 con 0x5637d1e27dd0 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637d1cd24a0 con 0x5637d1e27dd0 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faed802b430 con 0x5637d1e27dd0 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 10 client.14946 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 10 client.14946 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 20 client.14946 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e3bdd0 0x5637d1e3b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 4 client.14946 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:25.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.341+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637d1d71e60 con 0x5637d1e3bdd0 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.342+0000 7faeee59c700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e3bdd0 0x5637d1e3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.342+0000 7faeee59c700 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e3bdd0 0x5637d1e3b510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cea331b13d9d549c server_cookie=b5870f2995cf65e3 in_seq=0 out_seq=0 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.342+0000 7faed5ffb700 10 client.14946 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faed5ffb700 1 -- 172.21.15.26:0/1966825251 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637d1d71e60 con 0x5637d1e3bdd0 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faed5ffb700 10 client.14946 handle_command_reply: tid=0 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 client.14946 shutdown 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 msgr2=0x5637d1e315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faed80098e0 tx=0x7faed8005c70 comp rx=0 tx=0).stop 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 shutdown_connections 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e3bdd0 0x5637d1e3b510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e28c20 0x5637d1e385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27480 0x5637d1e31060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faed5ffb700 0 client.14946 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 --2- 172.21.15.26:0/1966825251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27dd0 0x5637d1e315a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 >> 172.21.15.26:0/1966825251 conn(0x5637d1e1cd50 msgr2=0x5637d1e21860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 shutdown_connections 2023-07-13T22:13:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.343+0000 7faf08321b80 1 -- 172.21.15.26:0/1966825251 wait complete. 2023-07-13T22:13:25.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.344+0000 7faf08321b80 20 client.14946 trim_cache size 0 max 16384 2023-07-13T22:13:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 Processor -- start 2023-07-13T22:13:25.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 -- start start 2023-07-13T22:13:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27480 0x5637d1e1e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 0x5637d1e44b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d22470 con 0x5637d1e28c20 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1d47790 con 0x5637d1e27dd0 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.362+0000 7faf08321b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1c08de0 con 0x5637d1e27480 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faeeed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 0x5637d1e44b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faeeed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 0x5637d1e44b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59122/0 (socket says 172.21.15.26:59122) 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faeeed9d700 1 -- 172.21.15.26:0/416493384 learned_addr learned my addr 172.21.15.26:0/416493384 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faeee59c700 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27480 0x5637d1e1e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 -- 172.21.15.26:0/416493384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27480 msgr2=0x5637d1e1e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27480 0x5637d1e1e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 -- 172.21.15.26:0/416493384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 msgr2=0x5637d1e44b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 0x5637d1e44b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 -- 172.21.15.26:0/416493384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637d1b8fc10 con 0x5637d1e27dd0 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee7fff700 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faed8009990 tx=0x7faed8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d020b322e52d5fa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.363+0000 7faee57fa700 1 -- 172.21.15.26:0/416493384 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800bcb0 con 0x5637d1e27dd0 2023-07-13T22:13:25.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faee57fa700 1 -- 172.21.15.26:0/416493384 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faed802b430 con 0x5637d1e27dd0 2023-07-13T22:13:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faee57fa700 1 -- 172.21.15.26:0/416493384 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed80377f0 con 0x5637d1e27dd0 2023-07-13T22:13:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 -- 172.21.15.26:0/416493384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 msgr2=0x5637d1e38500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faed8009990 tx=0x7faed8005c00 comp rx=0 tx=0).stop 2023-07-13T22:13:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 -- 172.21.15.26:0/416493384 shutdown_connections 2023-07-13T22:13:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e28c20 0x5637d1e44b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27480 0x5637d1e1e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 --2- 172.21.15.26:0/416493384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 -- 172.21.15.26:0/416493384 >> 172.21.15.26:0/416493384 conn(0x5637d1e1cd50 msgr2=0x5637d1e3b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 -- 172.21.15.26:0/416493384 shutdown_connections 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.364+0000 7faf08321b80 1 -- 172.21.15.26:0/416493384 wait complete. 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 -- start start 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 Processor -- start 2023-07-13T22:13:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 0x5637d1e2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 0x5637d1e2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1e8fc70 con 0x5637d1e27480 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1e8fdb0 con 0x5637d1e28c20 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637d1e8fef0 con 0x5637d1e27dd0 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.365+0000 7faf08321b80 10 client.0 _ll_register_callbacks cb 0x7faee00956a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeeed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 0x5637d1e2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeeed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 0x5637d1e2e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59914/0 (socket says 172.21.15.26:59914) 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 0x5637d1e2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeeed9d700 1 -- 172.21.15.26:0/2310236636 learned_addr learned my addr 172.21.15.26:0/2310236636 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faee7fff700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 -- 172.21.15.26:0/2310236636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27dd0 msgr2=0x5637d1e38500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 -- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 msgr2=0x5637d1e2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 0x5637d1e2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 -- 172.21.15.26:0/2310236636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef0108850 con 0x5637d1e27480 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faeee59c700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 0x5637d1e2ddd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faee00964f0 tx=0x7faee0095ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf6a81ea357f777 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee00a1ed0 con 0x5637d1e27480 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7faee0099ce0 con 0x5637d1e27480 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.366+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faed801e040 con 0x5637d1e27480 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faee0099ea0 con 0x5637d1e27480 2023-07-13T22:13:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 10 client.14952 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5637d1bcd200 con 0x5637d1e27480 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee007c020 con 0x5637d1e27480 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faee00c5020 con 0x5637d1e27480 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 10 client.14952 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 10 client.14952 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 20 client.14952 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e94c90 0x5637d1e91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 4 client.14952 mds_command: new command op to 24467 tid=0[{"prefix": "status"}] 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.367+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5637d1d34000 con 0x5637d1e94c90 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faee7fff700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e94c90 0x5637d1e91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faee7fff700 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e94c90 0x5637d1e91580 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99b35209b0d33e18 server_cookie=905eb729331d4c7b in_seq=0 out_seq=0 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faed5ffb700 10 client.14952 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faed5ffb700 1 -- 172.21.15.26:0/2310236636 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5637d1d34000 con 0x5637d1e94c90 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faed5ffb700 10 client.14952 handle_command_reply: tid=0 2023-07-13T22:13:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faf08321b80 1 client.14952 shutdown 2023-07-13T22:13:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 msgr2=0x5637d1e2ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.368+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 0x5637d1e2ddd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faee00964f0 tx=0x7faee0095ef0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 shutdown_connections 2023-07-13T22:13:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5637d1e94c90 0x5637d1e91580 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5637d1e27480 0x5637d1e2ddd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5637d1e28c20 0x5637d1e2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faed5ffb700 0 client.14952 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 --2- 172.21.15.26:0/2310236636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5637d1e27dd0 0x5637d1e38500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 >> 172.21.15.26:0/2310236636 conn(0x5637d1e1cd50 msgr2=0x5637d1e1e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 shutdown_connections 2023-07-13T22:13:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 1 -- 172.21.15.26:0/2310236636 wait complete. 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faf08321b80 20 client.14952 trim_cache size 0 max 16384 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 4, 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24467, 2023-07-13T22:13:25.367 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 59.927885095999997, 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:25.368 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 99.234341004000001 2023-07-13T22:13:25.369 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 msgr2=0x7faedc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 0x7faedc0a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faee8009550 tx=0x7faee8007bc0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 msgr2=0x7faef0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faee000d980 tx=0x7faee000dd40 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 shutdown_connections 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faedc0a1260 0x7faedc0a3710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef00ffae0 0x7faef0197a90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faef0101660 0x7faef019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 --2- 172.21.15.26:0/1895522728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faef0100cd0 0x7faef0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.369+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 >> 172.21.15.26:0/1895522728 conn(0x7faef00fb0e0 msgr2=0x7faef00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.370+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 shutdown_connections 2023-07-13T22:13:25.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.370+0000 7faef49ff700 1 -- 172.21.15.26:0/1895522728 wait complete. 2023-07-13T22:13:25.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-13T22:13:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 -- 172.21.15.26:0/2597255604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 msgr2=0x7fe8540ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/2597255604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe8540ffef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe8440098f0 tx=0x7fe844009c00 comp rx=0 tx=0).stop 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 -- 172.21.15.26:0/2597255604 shutdown_connections 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/2597255604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854101660 0x7fe854107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/2597255604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe8540ffef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/2597255604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854100cd0 0x7fe854101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.593+0000 7fe85a5ff700 1 -- 172.21.15.26:0/2597255604 >> 172.21.15.26:0/2597255604 conn(0x7fe8540fb0e0 msgr2=0x7fe8540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.594+0000 7fe85a5ff700 1 -- 172.21.15.26:0/2597255604 shutdown_connections 2023-07-13T22:13:25.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.594+0000 7fe85a5ff700 1 -- 172.21.15.26:0/2597255604 wait complete. 2023-07-13T22:13:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.594+0000 7fe85a5ff700 1 Processor -- start 2023-07-13T22:13:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 -- start start 2023-07-13T22:13:25.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe854197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 0x7fe854198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85419f660 con 0x7fe854101660 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85419f7a0 con 0x7fe8540ffae0 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe85a5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85419f930 con 0x7fe854100cd0 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe858b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 0x7fe854198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe858b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59154/0 (socket says 172.21.15.26:59154) 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe853fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe854197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.595+0000 7fe858b9c700 1 -- 172.21.15.26:0/4186827964 learned_addr learned my addr 172.21.15.26:0/4186827964 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 -- 172.21.15.26:0/4186827964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 msgr2=0x7fe854197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe854197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 -- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 msgr2=0x7fe85419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 -- 172.21.15.26:0/4186827964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009520 con 0x7fe854100cd0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe858b9c700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8537fe700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 0x7fe854198020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe84800e9c0 tx=0x7fe84800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8517fa700 1 -- 172.21.15.26:0/4186827964 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84800c9c0 con 0x7fe854100cd0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85419fc10 con 0x7fe854100cd0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8517fa700 1 -- 172.21.15.26:0/4186827964 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe848010430 con 0x7fe854100cd0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe8517fa700 1 -- 172.21.15.26:0/4186827964 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848018680 con 0x7fe854100cd0 2023-07-13T22:13:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.596+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8541a0170 con 0x7fe854100cd0 2023-07-13T22:13:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.600+0000 7fe8517fa700 1 -- 172.21.15.26:0/4186827964 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe848010810 con 0x7fe854100cd0 2023-07-13T22:13:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.601+0000 7fe86de72b80 1 Processor -- start 2023-07-13T22:13:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.601+0000 7fe86de72b80 1 -- start start 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 0x5619a5713690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5713c20 0x5619a5714090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5617c60 con 0x5619a5713c20 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a566a9f0 con 0x5619a5712480 2023-07-13T22:13:25.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe86de72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5663e80 con 0x5619a5712dd0 2023-07-13T22:13:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5713c20 0x5619a5714090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59936/0 (socket says 172.21.15.26:59936) 2023-07-13T22:13:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 -- 172.21.15.26:0/1458813660 learned_addr learned my addr 172.21.15.26:0/1458813660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe853fff700 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 0x5619a5713690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 -- 172.21.15.26:0/1458813660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 msgr2=0x5619a5713690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe8517fa700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 0x7fe83c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 0x5619a5713690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 -- 172.21.15.26:0/1458813660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5713c20 msgr2=0x5619a5714090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5713c20 0x5619a5714090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe858b9c700 1 -- 172.21.15.26:0/1458813660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a5491d90 con 0x5619a5712480 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.602+0000 7fe853fff700 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 0x5619a5713690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:25.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe8517fa700 1 -- 172.21.15.26:0/4186827964 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe848014070 con 0x7fe854100cd0 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe858b9c700 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009500 tx=0x7fe84c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865aadfc3d409433 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe858b9c700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 0x7fe83c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe83a7fc700 1 -- 172.21.15.26:0/1458813660 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c017a40 con 0x5619a5712480 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe83a7fc700 1 -- 172.21.15.26:0/1458813660 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe84c00f040 con 0x5619a5712480 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe83a7fc700 1 -- 172.21.15.26:0/1458813660 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c022bf0 con 0x5619a5712480 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.603+0000 7fe858b9c700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 0x7fe83c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe84c020000 tx=0x7fe84c023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 -- 172.21.15.26:0/1458813660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 msgr2=0x5619a5712890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009500 tx=0x7fe84c009920 comp rx=0 tx=0).stop 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 -- 172.21.15.26:0/1458813660 shutdown_connections 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5713c20 0x5619a5714090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a5712890 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1458813660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712dd0 0x5619a5713690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 -- 172.21.15.26:0/1458813660 >> 172.21.15.26:0/1458813660 conn(0x5619a5707d50 msgr2=0x5619a570d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 -- 172.21.15.26:0/1458813660 shutdown_connections 2023-07-13T22:13:25.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.604+0000 7fe86de72b80 1 -- 172.21.15.26:0/1458813660 wait complete. 2023-07-13T22:13:25.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.605+0000 7fe86de72b80 1 -- start start 2023-07-13T22:13:25.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.605+0000 7fe86de72b80 1 Processor -- start 2023-07-13T22:13:25.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712dd0 0x5619a571c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5603a70 con 0x5619a5712480 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5603770 con 0x5619a5712dd0 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a54e78d0 con 0x5619a5713c20 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe86de72b80 10 client.0 _ll_register_callbacks cb 0x5619a5601710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe853fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe853fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59174/0 (socket says 172.21.15.26:59174) 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe853fff700 1 -- 172.21.15.26:0/1790802682 learned_addr learned my addr 172.21.15.26:0/1790802682 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe858b9c700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.606+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712dd0 0x5619a571c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 -- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 msgr2=0x5619a57236e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57236e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 -- 172.21.15.26:0/1790802682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712dd0 msgr2=0x5619a571c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712dd0 0x5619a571c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 -- 172.21.15.26:0/1790802682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c0091e0 con 0x5619a5712480 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe853fff700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8440098f0 tx=0x7fe84400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a73fdea40aaaff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe8337fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84401c070 con 0x5619a5712480 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619a55bd4a0 con 0x5619a5712480 2023-07-13T22:13:25.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe84400b910 con 0x5619a5712480 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe84400be90 con 0x5619a5712480 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe86de72b80 10 client.14964 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.607+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619a547ac10 con 0x5619a5712480 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619a55bd4a0 con 0x5619a5712480 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe844015020 con 0x5619a5712480 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 10 client.14964 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 10 client.14964 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-13T22:13:25.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 20 client.14964 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5727430 0x5619a5726830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 4 client.14964 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.608+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619a565ce60 con 0x5619a5727430 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.609+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5727430 0x5619a5726830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.609+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5727430 0x5619a5726830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c11340c629b961c5 server_cookie=95419005b1a91861 in_seq=0 out_seq=0 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.609+0000 7fe8337fe700 10 client.14964 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:25.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe8337fe700 1 -- 172.21.15.26:0/1790802682 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5619a565ce60 con 0x5619a5727430 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe8337fe700 10 client.14964 handle_command_reply: tid=0 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 client.14964 shutdown 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 msgr2=0x5619a571c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8440098f0 tx=0x7fe84400f690 comp rx=0 tx=0).stop 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 shutdown_connections 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5727430 0x5619a5726830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712480 0x5619a571c160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57236e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1790802682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712dd0 0x5619a571c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe8337fe700 0 client.14964 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 >> 172.21.15.26:0/1790802682 conn(0x5619a5707d50 msgr2=0x5619a570c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 shutdown_connections 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.610+0000 7fe86de72b80 1 -- 172.21.15.26:0/1790802682 wait complete. 2023-07-13T22:13:25.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.611+0000 7fe86de72b80 20 client.14964 trim_cache size 0 max 16384 2023-07-13T22:13:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 Processor -- start 2023-07-13T22:13:25.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 -- start start 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712480 0x5619a571ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5713c20 0x5619a570c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5632790 con 0x5619a5712dd0 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a54f3de0 con 0x5619a5713c20 2023-07-13T22:13:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.629+0000 7fe86de72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5622480 con 0x5619a5712480 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59182/0 (socket says 172.21.15.26:59182) 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 -- 172.21.15.26:0/1710413195 learned_addr learned my addr 172.21.15.26:0/1710413195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe853fff700 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712480 0x5619a571ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe858b9c700 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5713c20 0x5619a570c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 -- 172.21.15.26:0/1710413195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712480 msgr2=0x5619a571ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712480 0x5619a571ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 -- 172.21.15.26:0/1710413195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5713c20 msgr2=0x5619a570c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5713c20 0x5619a570c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 -- 172.21.15.26:0/1710413195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a54b8200 con 0x5619a5712dd0 2023-07-13T22:13:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe8537fe700 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8480961e0 tx=0x7fe8480d2530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4009ad4d76f65c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe83affd700 1 -- 172.21.15.26:0/1710413195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848099640 con 0x5619a5712dd0 2023-07-13T22:13:25.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe83affd700 1 -- 172.21.15.26:0/1710413195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe84809c030 con 0x5619a5712dd0 2023-07-13T22:13:25.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.630+0000 7fe83affd700 1 -- 172.21.15.26:0/1710413195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848099f40 con 0x5619a5712dd0 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- 172.21.15.26:0/1710413195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 msgr2=0x5619a57236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8480961e0 tx=0x7fe8480d2530 comp rx=0 tx=0).stop 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- 172.21.15.26:0/1710413195 shutdown_connections 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5713c20 0x5619a570c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5712480 0x5619a571ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 --2- 172.21.15.26:0/1710413195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a57236a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- 172.21.15.26:0/1710413195 >> 172.21.15.26:0/1710413195 conn(0x5619a5707d50 msgr2=0x5619a5730160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- 172.21.15.26:0/1710413195 shutdown_connections 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- 172.21.15.26:0/1710413195 wait complete. 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.631+0000 7fe86de72b80 1 -- start start 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 Processor -- start 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a571ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5722710 con 0x5619a5712dd0 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5722880 con 0x5619a5712480 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5722a40 con 0x5619a5713c20 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe86de72b80 10 client.0 _ll_register_callbacks cb 0x7fe844009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.632+0000 7fe8537fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59190/0 (socket says 172.21.15.26:59190) 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe853fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a571ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 -- 172.21.15.26:0/2207173448 learned_addr learned my addr 172.21.15.26:0/2207173448 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe858b9c700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 -- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 msgr2=0x5619a57221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 -- 172.21.15.26:0/2207173448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 msgr2=0x5619a571ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a571ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 -- 172.21.15.26:0/2207173448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8541a0170 con 0x5619a5712dd0 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe853fff700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a571ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:25.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8537fe700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe8480a1c00 tx=0x7fe8480a2130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b316a76e5c46b6a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8337fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8480b2460 con 0x5619a5712dd0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.633+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe84809dd10 con 0x5619a5712dd0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe84c00f040 con 0x5619a5712dd0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe848099ce0 con 0x5619a5712dd0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 10 client.14970 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5619a5490260 con 0x5619a5712dd0 2023-07-13T22:13:25.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe84c00f040 con 0x5619a5712dd0 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe8480a1ca0 con 0x5619a5712dd0 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 10 client.14970 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 20 client.14970 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5780a30 0x5619a577d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 4 client.14970 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.634+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5619a561f000 con 0x5619a5780a30 2023-07-13T22:13:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe853fff700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5780a30 0x5619a577d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe853fff700 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5780a30 0x5619a577d3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd2f342af75a7863 server_cookie=c8eb5357ffaf7e05 in_seq=0 out_seq=0 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe8337fe700 10 client.14970 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe8337fe700 1 -- 172.21.15.26:0/2207173448 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5619a561f000 con 0x5619a5780a30 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe8337fe700 10 client.14970 handle_command_reply: tid=0 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.635+0000 7fe86de72b80 1 client.14970 shutdown 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 msgr2=0x5619a571f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe8480a1c00 tx=0x7fe8480a2130 comp rx=0 tx=0).stop 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 shutdown_connections 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5619a5780a30 0x5619a577d3c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619a5713c20 0x5619a57221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5619a5712480 0x5619a571ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 --2- 172.21.15.26:0/2207173448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5619a5712dd0 0x5619a571f2c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 >> 172.21.15.26:0/2207173448 conn(0x5619a5707d50 msgr2=0x5619a570c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe8337fe700 0 client.14970 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 shutdown_connections 2023-07-13T22:13:25.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 1 -- 172.21.15.26:0/2207173448 wait complete. 2023-07-13T22:13:25.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.636+0000 7fe86de72b80 20 client.14970 trim_cache size 0 max 16384 2023-07-13T22:13:25.634 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:25.634 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 3, 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24493, 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 61.201083549000003, 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:25.635 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:25.636 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 89.317025443999995 2023-07-13T22:13:25.636 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:25.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 msgr2=0x7fe83c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 0x7fe83c0a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe84c020000 tx=0x7fe84c023000 comp rx=0 tx=0).stop 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 msgr2=0x7fe854198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 0x7fe854198020 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe84800e9c0 tx=0x7fe84800ecd0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 shutdown_connections 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe83c0a1260 0x7fe83c0a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe854101660 0x7fe85419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8540ffae0 0x7fe854197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 --2- 172.21.15.26:0/4186827964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe854100cd0 0x7fe854198020 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 >> 172.21.15.26:0/4186827964 conn(0x7fe8540fb0e0 msgr2=0x7fe8541065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 shutdown_connections 2023-07-13T22:13:25.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.637+0000 7fe85a5ff700 1 -- 172.21.15.26:0/4186827964 wait complete. 2023-07-13T22:13:25.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-13T22:13:25.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 -- 172.21.15.26:0/2757443282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc418101660 msgr2=0x7fc418107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 --2- 172.21.15.26:0/2757443282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc418101660 0x7fc418107f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc410009990 tx=0x7fc410009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 -- 172.21.15.26:0/2757443282 shutdown_connections 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 --2- 172.21.15.26:0/2757443282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc418101660 0x7fc418107f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 --2- 172.21.15.26:0/2757443282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4180ffae0 0x7fc4180ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 --2- 172.21.15.26:0/2757443282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 0x7fc418101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.860+0000 7fc4201ff700 1 -- 172.21.15.26:0/2757443282 >> 172.21.15.26:0/2757443282 conn(0x7fc4180fb0e0 msgr2=0x7fc4180fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.861+0000 7fc4201ff700 1 -- 172.21.15.26:0/2757443282 shutdown_connections 2023-07-13T22:13:25.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.861+0000 7fc4201ff700 1 -- 172.21.15.26:0/2757443282 wait complete. 2023-07-13T22:13:25.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.861+0000 7fc4201ff700 1 Processor -- start 2023-07-13T22:13:25.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 -- start start 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 0x7fc418072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc418101660 0x7fc418072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180735e0 con 0x7fc4180ffae0 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418073750 con 0x7fc418101660 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc4201ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41819f850 con 0x7fc418100cd0 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59216/0 (socket says 172.21.15.26:59216) 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc41df9b700 1 -- 172.21.15.26:0/945869295 learned_addr learned my addr 172.21.15.26:0/945869295 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.862+0000 7fc41d79a700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 0x7fc418072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41e79c700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc418101660 0x7fc418072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 -- 172.21.15.26:0/945869295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 msgr2=0x7fc418072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 0x7fc418072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 -- 172.21.15.26:0/945869295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc418101660 msgr2=0x7fc418072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc418101660 0x7fc418072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 -- 172.21.15.26:0/945869295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009c40 con 0x7fc4180ffae0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc41df9b700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c140 tx=0x7fc41400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc40effd700 1 -- 172.21.15.26:0/945869295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400e050 con 0x7fc4180ffae0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100095f0 con 0x7fc4180ffae0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc40effd700 1 -- 172.21.15.26:0/945869295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc41400f040 con 0x7fc4180ffae0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc40effd700 1 -- 172.21.15.26:0/945869295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414013660 con 0x7fc4180ffae0 2023-07-13T22:13:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.863+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41819fe50 con 0x7fc4180ffae0 2023-07-13T22:13:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.866+0000 7fc40effd700 1 -- 172.21.15.26:0/945869295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc414007500 con 0x7fc4180ffae0 2023-07-13T22:13:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.867+0000 7fc433c09b80 1 Processor -- start 2023-07-13T22:13:25.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 -- start start 2023-07-13T22:13:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc40effd700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 0x7fc4040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc40effd700 1 -- 172.21.15.26:0/945869295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc4140cc9a0 con 0x7fc4180ffae0 2023-07-13T22:13:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83dd0 0x555fffb84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb84c20 0x555fffb85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffa88c60 con 0x555fffb83dd0 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffadb9f0 con 0x555fffb84c20 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc433c09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffad4e80 con 0x555fffb83480 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc41e79c700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 0x7fc4040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:59572/0 (socket says 172.21.15.26:59572) 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc41df9b700 1 -- 172.21.15.26:0/2016033557 learned_addr learned my addr 172.21.15.26:0/2016033557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.868+0000 7fc41d79a700 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83dd0 0x555fffb84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41e79c700 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb84c20 0x555fffb85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 -- 172.21.15.26:0/2016033557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb84c20 msgr2=0x555fffb85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb84c20 0x555fffb85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 -- 172.21.15.26:0/2016033557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83dd0 msgr2=0x555fffb84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83dd0 0x555fffb84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 -- 172.21.15.26:0/2016033557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fff902d90 con 0x555fffb83480 2023-07-13T22:13:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41e79c700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 0x7fc4040a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc410005f50 tx=0x7fc410005430 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc41df9b700 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4140966d0 tx=0x7fc414099ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2633f6b422441684 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc403fff700 1 -- 172.21.15.26:0/2016033557 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400ce10 con 0x555fffb83480 2023-07-13T22:13:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc403fff700 1 -- 172.21.15.26:0/2016033557 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc414099c60 con 0x555fffb83480 2023-07-13T22:13:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.869+0000 7fc403fff700 1 -- 172.21.15.26:0/2016033557 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41409a120 con 0x555fffb83480 2023-07-13T22:13:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 -- 172.21.15.26:0/2016033557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 msgr2=0x555fffb83890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc4140966d0 tx=0x7fc414099ab0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 -- 172.21.15.26:0/2016033557 shutdown_connections 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb84c20 0x555fffb85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb83890 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 --2- 172.21.15.26:0/2016033557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83dd0 0x555fffb84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 -- 172.21.15.26:0/2016033557 >> 172.21.15.26:0/2016033557 conn(0x555fffb78d50 msgr2=0x555fffb7e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 -- 172.21.15.26:0/2016033557 shutdown_connections 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.870+0000 7fc433c09b80 1 -- 172.21.15.26:0/2016033557 wait complete. 2023-07-13T22:13:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 -- start start 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 Processor -- start 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 0x555fffb8d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83dd0 0x555fffb8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffa74a70 con 0x555fffb84c20 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffa74770 con 0x555fffb83480 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fff9588d0 con 0x555fffb83dd0 2023-07-13T22:13:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.871+0000 7fc433c09b80 10 client.0 _ll_register_callbacks cb 0x555fffa72710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41e79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 0x555fffb8d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83dd0 0x555fffb8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41e79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59228/0 (socket says 172.21.15.26:59228) 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41e79c700 1 -- 172.21.15.26:0/4217117431 learned_addr learned my addr 172.21.15.26:0/4217117431 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 -- 172.21.15.26:0/4217117431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83dd0 msgr2=0x555fffb8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83dd0 0x555fffb8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 -- 172.21.15.26:0/4217117431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 msgr2=0x555fffb94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 -- 172.21.15.26:0/4217117431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41819fe50 con 0x555fffb83480 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41e79c700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 0x555fffb8d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4140966d0 tx=0x7fc4140996f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54a4913eae22986 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc4017fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.872+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41409dc90 con 0x555fffb83480 2023-07-13T22:13:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fffa2e4a0 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc41400ce10 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4140ca5b0 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 10 client.25019 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555fff8ebc10 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc414063080 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4140a1c20 con 0x555fffb83480 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 10 client.25019 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 10 client.25019 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 20 client.25019 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffb97e30 0x555fffb97570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 4 client.25019 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.873+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fffacde60 con 0x555fffb97e30 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.874+0000 7fc41d79a700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffb97e30 0x555fffb97570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.874+0000 7fc41d79a700 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffb97e30 0x555fffb97570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e12282f6bd01b7d2 server_cookie=322785fd38fe2a86 in_seq=0 out_seq=0 2023-07-13T22:13:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.874+0000 7fc4017fa700 10 client.25019 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc4017fa700 1 -- 172.21.15.26:0/4217117431 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fffacde60 con 0x555fffb97e30 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc4017fa700 10 client.25019 handle_command_reply: tid=0 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 client.25019 shutdown 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 msgr2=0x555fffb8d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 0x555fffb8d0c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4140966d0 tx=0x7fc4140996f0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 shutdown_connections 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffb97e30 0x555fffb97570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83480 0x555fffb8d0c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4217117431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83dd0 0x555fffb8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 >> 172.21.15.26:0/4217117431 conn(0x555fffb78d50 msgr2=0x555fffb7d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc4017fa700 0 client.25019 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 shutdown_connections 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 1 -- 172.21.15.26:0/4217117431 wait complete. 2023-07-13T22:13:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.875+0000 7fc433c09b80 20 client.25019 trim_cache size 0 max 16384 2023-07-13T22:13:25.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 Processor -- start 2023-07-13T22:13:25.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 -- start start 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb8b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffb7f220 con 0x555fffb84c20 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffb8dfc0 con 0x555fffb83dd0 2023-07-13T22:13:25.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.894+0000 7fc433c09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffacde90 con 0x555fffb83480 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59230/0 (socket says 172.21.15.26:59230) 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 -- 172.21.15.26:0/1091521451 learned_addr learned my addr 172.21.15.26:0/1091521451 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41d79a700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb8b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41df9b700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 -- 172.21.15.26:0/1091521451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 msgr2=0x555fffb8af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb8af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 -- 172.21.15.26:0/1091521451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 msgr2=0x555fffb8b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb8b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 -- 172.21.15.26:0/1091521451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fff929200 con 0x555fffb84c20 2023-07-13T22:13:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41d79a700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc41e79c700 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc41000bad0 tx=0x7fc410029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22a3d389d8d665e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.895+0000 7fc40cff9700 1 -- 172.21.15.26:0/1091521451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41002e040 con 0x555fffb84c20 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc40cff9700 1 -- 172.21.15.26:0/1091521451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc410004dd0 con 0x555fffb84c20 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc40cff9700 1 -- 172.21.15.26:0/1091521451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc410028d60 con 0x555fffb84c20 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 -- 172.21.15.26:0/1091521451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 msgr2=0x555fffb94600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc41000bad0 tx=0x7fc410029000 comp rx=0 tx=0).stop 2023-07-13T22:13:25.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 -- 172.21.15.26:0/1091521451 shutdown_connections 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb84c20 0x555fffb94600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb83480 0x555fffb8af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 --2- 172.21.15.26:0/1091521451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 -- 172.21.15.26:0/1091521451 >> 172.21.15.26:0/1091521451 conn(0x555fffb78d50 msgr2=0x555fffb87a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 -- 172.21.15.26:0/1091521451 shutdown_connections 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.896+0000 7fc433c09b80 1 -- 172.21.15.26:0/1091521451 wait complete. 2023-07-13T22:13:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.897+0000 7fc433c09b80 1 -- start start 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.897+0000 7fc433c09b80 1 Processor -- start 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb84c20 0x555fffb93300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffb93a10 con 0x555fffb83480 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffb93b80 con 0x555fffb83dd0 2023-07-13T22:13:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fffb93d40 con 0x555fffb84c20 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc433c09b80 10 client.0 _ll_register_callbacks cb 0x555fffa72710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59236/0 (socket says 172.21.15.26:59236) 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 -- 172.21.15.26:0/4053630001 learned_addr learned my addr 172.21.15.26:0/4053630001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41e79c700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb84c20 0x555fffb93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41d79a700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 -- 172.21.15.26:0/4053630001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb84c20 msgr2=0x555fffb93300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb84c20 0x555fffb93300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 -- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 msgr2=0x555fffb92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb92d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.898+0000 7fc41df9b700 1 -- 172.21.15.26:0/4053630001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc408010430 con 0x555fffb83480 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc41df9b700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc414096160 tx=0x7fc4140ca7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b13edfc66a8897 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41400ce10 con 0x555fffb83480 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc4140ca020 con 0x555fffb83480 2023-07-13T22:13:25.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc414099c60 con 0x555fffb83480 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc414011e40 con 0x555fffb83480 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc433c09b80 10 client.14988 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc410020070 con 0x555fffb83480 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc414099c60 con 0x555fffb83480 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc414078b80 con 0x555fffb83480 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.899+0000 7fc433c09b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:25.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc433c09b80 10 client.14988 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-13T22:13:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc433c09b80 20 client.14988 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffbf1680 0x555fffbee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc433c09b80 4 client.14988 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-07-13T22:13:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555fffa90000 con 0x555fffbf1680 2023-07-13T22:13:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc41d79a700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffbf1680 0x555fffbee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc41d79a700 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffbf1680 0x555fffbee040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=831adb57a4037fd1 server_cookie=b05aaf6fe1ffc876 in_seq=0 out_seq=0 2023-07-13T22:13:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.900+0000 7fc4017fa700 10 client.14988 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc4017fa700 1 -- 172.21.15.26:0/4053630001 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555fffa90000 con 0x555fffbf1680 2023-07-13T22:13:25.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc4017fa700 10 client.14988 handle_command_reply: tid=0 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 client.14988 shutdown 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 msgr2=0x555fffb92840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc414096160 tx=0x7fc4140ca7e0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 shutdown_connections 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fffb83480 0x555fffb92840 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555fffbf1680 0x555fffbee040 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fffb84c20 0x555fffb93300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc4017fa700 0 client.14988 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 --2- 172.21.15.26:0/4053630001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fffb83dd0 0x555fffb92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 >> 172.21.15.26:0/4053630001 conn(0x555fffb78d50 msgr2=0x555fffb87a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 shutdown_connections 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 1 -- 172.21.15.26:0/4053630001 wait complete. 2023-07-13T22:13:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.901+0000 7fc433c09b80 20 client.14988 trim_cache size 0 max 16384 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24530, 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 61.915623506999999, 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 85.344268201000006 2023-07-13T22:13:25.901 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 msgr2=0x7fc4040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 0x7fc4040a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc410005f50 tx=0x7fc410005430 comp rx=0 tx=0).stop 2023-07-13T22:13:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 msgr2=0x7fc4180742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c140 tx=0x7fc41400c450 comp rx=0 tx=0).stop 2023-07-13T22:13:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 shutdown_connections 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc4040a11e0 0x7fc4040a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4180ffae0 0x7fc4180742c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc418101660 0x7fc418072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 --2- 172.21.15.26:0/945869295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc418100cd0 0x7fc418072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 >> 172.21.15.26:0/945869295 conn(0x7fc4180fb0e0 msgr2=0x7fc4180fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 shutdown_connections 2023-07-13T22:13:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:25.902+0000 7fc4201ff700 1 -- 172.21.15.26:0/945869295 wait complete. 2023-07-13T22:13:25.908 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:25 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2427829103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:25.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:13:25.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:25 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2427829103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:25.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:25 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2427829103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.125+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/3231696572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 msgr2=0x7f7bd40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.125+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/3231696572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd40ffef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009990 tx=0x7f7bd0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/3231696572 shutdown_connections 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/3231696572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd40ffef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/3231696572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 0x7f7bd4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/3231696572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 0x7f7bd4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/3231696572 >> 172.21.15.26:0/3231696572 conn(0x7f7bd40fb0e0 msgr2=0x7f7bd40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/3231696572 shutdown_connections 2023-07-13T22:13:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.126+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/3231696572 wait complete. 2023-07-13T22:13:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 Processor -- start 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 -- start start 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 0x7f7bd4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 0x7f7bd419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd419f5c0 con 0x7f7bd40ffae0 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd419f700 con 0x7f7bd4100cd0 2023-07-13T22:13:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bdb5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd419f890 con 0x7f7bd4101660 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59254/0 (socket says 172.21.15.26:59254) 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.127+0000 7f7bd939b700 1 -- 172.21.15.26:0/929772900 learned_addr learned my addr 172.21.15.26:0/929772900 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 0x7f7bd4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 0x7f7bd419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 -- 172.21.15.26:0/929772900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 msgr2=0x7f7bd419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 0x7f7bd419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 -- 172.21.15.26:0/929772900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 msgr2=0x7f7bd4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 0x7f7bd4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 -- 172.21.15.26:0/929772900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd00095f0 con 0x7f7bd40ffae0 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bd939b700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000b620 tx=0x7f7bd0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bca7fc700 1 -- 172.21.15.26:0/929772900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd001d070 con 0x7f7bd40ffae0 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bca7fc700 1 -- 172.21.15.26:0/929772900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00047e0 con 0x7f7bd40ffae0 2023-07-13T22:13:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bca7fc700 1 -- 172.21.15.26:0/929772900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0005560 con 0x7f7bd40ffae0 2023-07-13T22:13:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.128+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd419fb10 con 0x7f7bd40ffae0 2023-07-13T22:13:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.129+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd419fee0 con 0x7f7bd40ffae0 2023-07-13T22:13:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.132+0000 7f7bca7fc700 1 -- 172.21.15.26:0/929772900 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7bd0004950 con 0x7f7bd40ffae0 2023-07-13T22:13:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7beee57b80 1 Processor -- start 2023-07-13T22:13:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7bca7fc700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 0x7f7bc00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7beee57b80 1 -- start start 2023-07-13T22:13:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7bca7fc700 1 -- 172.21.15.26:0/929772900 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7bd00cdc10 con 0x7f7bd40ffae0 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 0x7f7bc00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.133+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f7a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7beee57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2e7ec60 con 0x5651c2f79480 2023-07-13T22:13:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7beee57b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2ed19f0 con 0x5651c2f7ac20 2023-07-13T22:13:26.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7beee57b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2ecae80 con 0x5651c2f79dd0 2023-07-13T22:13:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59260/0 (socket says 172.21.15.26:59260) 2023-07-13T22:13:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 -- 172.21.15.26:0/3538396684 learned_addr learned my addr 172.21.15.26:0/3538396684 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f7a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 -- 172.21.15.26:0/3538396684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 msgr2=0x5651c2f7a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f7a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 -- 172.21.15.26:0/3538396684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 msgr2=0x5651c2f7b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f7b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 -- 172.21.15.26:0/3538396684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651c2cf8d90 con 0x5651c2f79480 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 0x7f7bc00a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0093d0 tx=0x7f7bcc00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.134+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0097400 tx=0x7f7bd009af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dba4567095c76fa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7bbf7fe700 1 -- 172.21.15.26:0/3538396684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a32c0 con 0x5651c2f79480 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7bbf7fe700 1 -- 172.21.15.26:0/3538396684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00992a0 con 0x5651c2f79480 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7bbf7fe700 1 -- 172.21.15.26:0/3538396684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd009a6c0 con 0x5651c2f79480 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 -- 172.21.15.26:0/3538396684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 msgr2=0x5651c2f79890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0097400 tx=0x7f7bd009af80 comp rx=0 tx=0).stop 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 -- 172.21.15.26:0/3538396684 shutdown_connections 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f79890 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f7b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3538396684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f7a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.135+0000 7f7beee57b80 1 -- 172.21.15.26:0/3538396684 >> 172.21.15.26:0/3538396684 conn(0x5651c2f6ed50 msgr2=0x5651c2f74180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.136+0000 7f7beee57b80 1 -- 172.21.15.26:0/3538396684 shutdown_connections 2023-07-13T22:13:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.136+0000 7f7beee57b80 1 -- 172.21.15.26:0/3538396684 wait complete. 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.136+0000 7f7beee57b80 1 -- start start 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.136+0000 7f7beee57b80 1 Processor -- start 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f83290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2e6aa70 con 0x5651c2f79480 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2e6a770 con 0x5651c2f7ac20 2023-07-13T22:13:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2d4e8d0 con 0x5651c2f79dd0 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7beee57b80 10 client.0 _ll_register_callbacks cb 0x5651c2e68710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59270/0 (socket says 172.21.15.26:59270) 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 -- 172.21.15.26:0/2530637507 learned_addr learned my addr 172.21.15.26:0/2530637507 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f83290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 -- 172.21.15.26:0/2530637507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 msgr2=0x5651c2f83290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f83290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 -- 172.21.15.26:0/2530637507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 msgr2=0x5651c2f8a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f8a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.137+0000 7f7bd939b700 1 -- 172.21.15.26:0/2530637507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd419fee0 con 0x5651c2f79480 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009f0f0 tx=0x7f7bd009a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313fb4ddd62015e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:26.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00a3030 con 0x5651c2f79480 2023-07-13T22:13:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7bd00a0070 con 0x5651c2f79480 2023-07-13T22:13:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651c2e244a0 con 0x5651c2f79480 2023-07-13T22:13:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bd00cbb80 con 0x5651c2f79480 2023-07-13T22:13:26.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 10 client.15000 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5651c2ce1c10 con 0x5651c2f79480 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5651c2e244a0 con 0x5651c2f79480 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7bd00caf90 con 0x5651c2f79480 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 10 client.15000 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 20 client.15000 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.138+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f8db20 0x5651c2f8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.139+0000 7f7beee57b80 4 client.15000 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.139+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651c2ec3e60 con 0x5651c2f8db20 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.139+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f8db20 0x5651c2f8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.139+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f8db20 0x5651c2f8d170 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97536bebee595cfc server_cookie=bc1f5b44ecc77f1a in_seq=0 out_seq=0 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.139+0000 7f7bbcff9700 10 client.15000 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7bbcff9700 1 -- 172.21.15.26:0/2530637507 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5651c2ec3e60 con 0x5651c2f8db20 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7bbcff9700 10 client.15000 handle_command_reply: tid=0 2023-07-13T22:13:26.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 client.15000 shutdown 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 msgr2=0x5651c2f82d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7bd009f0f0 tx=0x7f7bd009a7b0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 shutdown_connections 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f8db20 0x5651c2f8d170 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79480 0x5651c2f82d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f7ac20 0x5651c2f8a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7bbcff9700 0 client.15000 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:26.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2530637507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79dd0 0x5651c2f83290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 >> 172.21.15.26:0/2530637507 conn(0x5651c2f6ed50 msgr2=0x5651c2f73860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 shutdown_connections 2023-07-13T22:13:26.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.140+0000 7f7beee57b80 1 -- 172.21.15.26:0/2530637507 wait complete. 2023-07-13T22:13:26.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.141+0000 7f7beee57b80 20 client.15000 trim_cache size 0 max 16384 2023-07-13T22:13:26.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 Processor -- start 2023-07-13T22:13:26.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 -- start start 2023-07-13T22:13:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79480 0x5651c2f7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79dd0 0x5651c2f8a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f82240 con 0x5651c2f6ed50 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f8ede0 con 0x5651c2f79dd0 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.159+0000 7f7beee57b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f83c50 con 0x5651c2f79480 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59272/0 (socket says 172.21.15.26:59272) 2023-07-13T22:13:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 -- 172.21.15.26:0/2795892123 learned_addr learned my addr 172.21.15.26:0/2795892123 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79dd0 0x5651c2f8a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79480 0x5651c2f7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 -- 172.21.15.26:0/2795892123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79480 msgr2=0x5651c2f7dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79480 0x5651c2f7dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 -- 172.21.15.26:0/2795892123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79dd0 msgr2=0x5651c2f8a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79dd0 0x5651c2f8a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 -- 172.21.15.26:0/2795892123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651c2d1f200 con 0x5651c2f6ed50 2023-07-13T22:13:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.160+0000 7f7bd939b700 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00cb4b0 tx=0x7f7bd0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3dd94b9a162766 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7bbffff700 1 -- 172.21.15.26:0/2795892123 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd00b92b0 con 0x5651c2f6ed50 2023-07-13T22:13:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7bbffff700 1 -- 172.21.15.26:0/2795892123 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00b2070 con 0x5651c2f6ed50 2023-07-13T22:13:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7bbffff700 1 -- 172.21.15.26:0/2795892123 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0095df0 con 0x5651c2f6ed50 2023-07-13T22:13:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 -- 172.21.15.26:0/2795892123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 msgr2=0x5651c2f7da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00cb4b0 tx=0x7f7bd0005c00 comp rx=0 tx=0).stop 2023-07-13T22:13:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 -- 172.21.15.26:0/2795892123 shutdown_connections 2023-07-13T22:13:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f6ed50 0x5651c2f7da80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79dd0 0x5651c2f8a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 --2- 172.21.15.26:0/2795892123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f79480 0x5651c2f7dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 -- 172.21.15.26:0/2795892123 >> 172.21.15.26:0/2795892123 conn(0x5651c2f70640 msgr2=0x5651c2f81b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 -- 172.21.15.26:0/2795892123 shutdown_connections 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.161+0000 7f7beee57b80 1 -- 172.21.15.26:0/2795892123 wait complete. 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 -- start start 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 Processor -- start 2023-07-13T22:13:26.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 0x5651c2f854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 0x5651c2f889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f88f00 con 0x5651c2f79dd0 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f89070 con 0x5651c2f79480 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651c2f89230 con 0x5651c2f6ed50 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.162+0000 7f7beee57b80 10 client.0 _ll_register_callbacks cb 0x5651c2e68710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd9b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 0x5651c2f889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd939b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 0x5651c2f854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60070/0 (socket says 172.21.15.26:60070) 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 -- 172.21.15.26:0/3463279557 learned_addr learned my addr 172.21.15.26:0/3463279557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 0x5651c2f854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:59642/0 (socket says 172.21.15.26:59642) 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 -- 172.21.15.26:0/3463279557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 msgr2=0x5651c2f854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 0x5651c2f854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 -- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 msgr2=0x5651c2f889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 0x5651c2f889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 -- 172.21.15.26:0/3463279557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651c2ce1c10 con 0x5651c2f79480 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd9b9c700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 0x5651c2f889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bd8b9a700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4005e10 tx=0x7f7bc40206f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bbf03fdc45f79d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.163+0000 7f7bbcff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4017040 con 0x5651c2f79480 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bd00992a0 con 0x5651c2f79480 2023-07-13T22:13:26.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7bc4004510 con 0x5651c2f79480 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bc4003680 con 0x5651c2f79480 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 10 client.25040 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5651c2cf7260 con 0x5651c2f79480 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd00992a0 con 0x5651c2f79480 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7bc4012990 con 0x5651c2f79480 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 10 client.25040 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:26.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 10 client.25040 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 20 client.25040 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f7ac20 0x5651c2fe3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.164+0000 7f7beee57b80 4 client.25040 mds_command: new command op to 24563 tid=0[{"prefix": "status"}] 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5651c2e86000 con 0x5651c2f7ac20 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f7ac20 0x5651c2fe3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7bd939b700 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f7ac20 0x5651c2fe3b50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7b821da348f103e server_cookie=f73de42ea3abd255 in_seq=0 out_seq=0 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7bbcff9700 10 client.25040 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:26.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7bbcff9700 1 -- 172.21.15.26:0/3463279557 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5651c2e86000 con 0x5651c2f7ac20 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7bbcff9700 10 client.25040 handle_command_reply: tid=0 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.165+0000 7f7beee57b80 1 client.25040 shutdown 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 msgr2=0x5651c2f85a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4005e10 tx=0x7f7bc40206f0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 shutdown_connections 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5651c2f7ac20 0x5651c2fe3b50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5651c2f79dd0 0x5651c2f889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5651c2f6ed50 0x5651c2f854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 --2- 172.21.15.26:0/3463279557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651c2f79480 0x5651c2f85a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7bbcff9700 0 client.25040 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 >> 172.21.15.26:0/3463279557 conn(0x5651c2f70640 msgr2=0x5651c2f7da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 shutdown_connections 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 1 -- 172.21.15.26:0/3463279557 wait complete. 2023-07-13T22:13:26.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7beee57b80 20 client.25040 trim_cache size 0 max 16384 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24563, 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:26.165 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 63.188925498000003, 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 77.085503211000002 2023-07-13T22:13:26.166 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 msgr2=0x7f7bc00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 0x7f7bc00a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0093d0 tx=0x7f7bcc00b040 comp rx=0 tx=0).stop 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 msgr2=0x7f7bd4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.166+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000b620 tx=0x7f7bd0005ef0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 shutdown_connections 2023-07-13T22:13:26.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7bc00a1260 0x7f7bc00a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd40ffae0 0x7f7bd4197a40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd4101660 0x7f7bd419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 --2- 172.21.15.26:0/929772900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bd4100cd0 0x7f7bd4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 >> 172.21.15.26:0/929772900 conn(0x7f7bd40fb0e0 msgr2=0x7f7bd4106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 shutdown_connections 2023-07-13T22:13:26.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.167+0000 7f7bdb5ff700 1 -- 172.21.15.26:0/929772900 wait complete. 2023-07-13T22:13:26.174 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-13T22:13:26.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 -- 172.21.15.26:0/1742598053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 msgr2=0x7f05380729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 --2- 172.21.15.26:0/1742598053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f05380729a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0534009990 tx=0x7f0534009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 -- 172.21.15.26:0/1742598053 shutdown_connections 2023-07-13T22:13:26.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 --2- 172.21.15.26:0/1742598053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f05380729a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 --2- 172.21.15.26:0/1742598053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05381079b0 0x7f0538109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 --2- 172.21.15.26:0/1742598053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0538072f70 0x7f05380733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 -- 172.21.15.26:0/1742598053 >> 172.21.15.26:0/1742598053 conn(0x7f05380fb130 msgr2=0x7f05380fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 -- 172.21.15.26:0/1742598053 shutdown_connections 2023-07-13T22:13:26.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.389+0000 7f05405ff700 1 -- 172.21.15.26:0/1742598053 wait complete. 2023-07-13T22:13:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 Processor -- start 2023-07-13T22:13:26.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 -- start start 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f0538198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05381079b0 0x7f053819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053819f640 con 0x7f0538074550 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053819f780 con 0x7f0538072f70 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.390+0000 7f05405ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053819f910 con 0x7f05381079b0 2023-07-13T22:13:26.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053db9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f0538198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60088/0 (socket says 172.21.15.26:60088) 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 -- 172.21.15.26:0/4255977690 learned_addr learned my addr 172.21.15.26:0/4255977690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053eb9c700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05381079b0 0x7f053819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 -- 172.21.15.26:0/4255977690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05381079b0 msgr2=0x7f053819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05381079b0 0x7f053819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 -- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 msgr2=0x7f0538198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f0538198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 -- 172.21.15.26:0/4255977690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530009c40 con 0x7f0538072f70 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053db9a700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f0538198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.391+0000 7f053e39b700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0534000c00 tx=0x7f0534005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.392+0000 7f052f7fe700 1 -- 172.21.15.26:0/4255977690 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053401d070 con 0x7f0538072f70 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.392+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05340095f0 con 0x7f0538072f70 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.392+0000 7f052f7fe700 1 -- 172.21.15.26:0/4255977690 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0534022470 con 0x7f0538072f70 2023-07-13T22:13:26.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.392+0000 7f052f7fe700 1 -- 172.21.15.26:0/4255977690 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534017650 con 0x7f0538072f70 2023-07-13T22:13:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.392+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053819fe80 con 0x7f0538072f70 2023-07-13T22:13:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.395+0000 7f052f7fe700 1 -- 172.21.15.26:0/4255977690 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f05340225e0 con 0x7f0538072f70 2023-07-13T22:13:26.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.396+0000 7f0553fd9b80 1 Processor -- start 2023-07-13T22:13:26.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.396+0000 7f0553fd9b80 1 -- start start 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec2e4c60 con 0x55a6ec3df480 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec3379f0 con 0x55a6ec3e0c20 2023-07-13T22:13:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec330e80 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59314/0 (socket says 172.21.15.26:59314) 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 -- 172.21.15.26:0/2479443370 learned_addr learned my addr 172.21.15.26:0/2479443370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053db9a700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053e39b700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f052f7fe700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 0x7f05240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 -- 172.21.15.26:0/2479443370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 msgr2=0x55a6ec3e0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 -- 172.21.15.26:0/2479443370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 msgr2=0x55a6ec3e1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053eb9c700 1 -- 172.21.15.26:0/2479443370 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ec15ed90 con 0x55a6ec3df480 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.397+0000 7f053db9a700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f052f7fe700 1 -- 172.21.15.26:0/4255977690 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05340cdda0 con 0x7f0538072f70 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f053eb9c700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 0x7f05240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f053eb9c700 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f053000e840 tx=0x7f053000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86404767b34b815 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f052cbfe700 1 -- 172.21.15.26:0/2479443370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000e040 con 0x55a6ec3df480 2023-07-13T22:13:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f052cbfe700 1 -- 172.21.15.26:0/2479443370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0530027070 con 0x55a6ec3df480 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.398+0000 7f052cbfe700 1 -- 172.21.15.26:0/2479443370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530007e30 con 0x55a6ec3df480 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- 172.21.15.26:0/2479443370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 msgr2=0x55a6ec3df890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f053eb9c700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 0x7f05240a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0530022000 tx=0x7f053001f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f053000e840 tx=0x7f053000eb50 comp rx=0 tx=0).stop 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- 172.21.15.26:0/2479443370 shutdown_connections 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3df480 0x55a6ec3df890 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/2479443370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- 172.21.15.26:0/2479443370 >> 172.21.15.26:0/2479443370 conn(0x55a6ec3d4d50 msgr2=0x55a6ec3da180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- 172.21.15.26:0/2479443370 shutdown_connections 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- 172.21.15.26:0/2479443370 wait complete. 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.399+0000 7f0553fd9b80 1 -- start start 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 Processor -- start 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec2d0a70 con 0x55a6ec3e0c20 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec2d0770 con 0x55a6ec3df480 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec1b48d0 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f0553fd9b80 10 client.0 _ll_register_callbacks cb 0x55a6ec2ce710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f053e39b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f053e39b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60100/0 (socket says 172.21.15.26:60100) 2023-07-13T22:13:26.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f053e39b700 1 -- 172.21.15.26:0/1926584384 learned_addr learned my addr 172.21.15.26:0/1926584384 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f053eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.400+0000 7f053db9a700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 -- 172.21.15.26:0/1926584384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 msgr2=0x55a6ec3e9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 -- 172.21.15.26:0/1926584384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 msgr2=0x55a6ec3f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 -- 172.21.15.26:0/1926584384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528009c90 con 0x55a6ec3df480 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053eb9c700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f053e39b700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f053400b680 tx=0x7f05340cbc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82605554fb85394 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f0521ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053409b4f0 con 0x55a6ec3df480 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6ec28a4a0 con 0x55a6ec3df480 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05340a3450 con 0x55a6ec3df480 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.401+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053409abd0 con 0x55a6ec3df480 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 10 client.25052 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f053000e460 con 0x55a6ec3df480 2023-07-13T22:13:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6ec28a4a0 con 0x55a6ec3df480 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0534096e80 con 0x55a6ec3df480 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 10 client.25052 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 10 client.25052 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 20 client.25052 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec3f4510 0x55a6ec3f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 4 client.25052 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:26.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6ec329e60 con 0x55a6ec3f4510 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.402+0000 7f053db9a700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec3f4510 0x55a6ec3f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.403+0000 7f053db9a700 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec3f4510 0x55a6ec3f35b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f1a2ea12383672 server_cookie=4363d42110698043 in_seq=0 out_seq=0 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.403+0000 7f0521ffb700 10 client.25052 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0521ffb700 1 -- 172.21.15.26:0/1926584384 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6ec329e60 con 0x55a6ec3f4510 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0521ffb700 10 client.25052 handle_command_reply: tid=0 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 client.25052 shutdown 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 msgr2=0x55a6ec3e9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f053400b680 tx=0x7f05340cbc30 comp rx=0 tx=0).stop 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 shutdown_connections 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec3f4510 0x55a6ec3f35b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3e9100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1926584384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3e9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0521ffb700 0 client.25052 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 >> 172.21.15.26:0/1926584384 conn(0x55a6ec3d4d50 msgr2=0x55a6ec3d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 shutdown_connections 2023-07-13T22:13:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1926584384 wait complete. 2023-07-13T22:13:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.404+0000 7f0553fd9b80 20 client.25052 trim_cache size 0 max 16384 2023-07-13T22:13:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 Processor -- start 2023-07-13T22:13:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 -- start start 2023-07-13T22:13:26.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec3e8cd0 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec329e60 con 0x55a6ec3df480 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec28a4a0 con 0x55a6ec3e0c20 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f053db9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f053db9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59334/0 (socket says 172.21.15.26:59334) 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f053db9a700 1 -- 172.21.15.26:0/1240028836 learned_addr learned my addr 172.21.15.26:0/1240028836 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.423+0000 7f053e39b700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053eb9c700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 -- 172.21.15.26:0/1240028836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3e0c20 msgr2=0x55a6ec3e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 -- 172.21.15.26:0/1240028836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 msgr2=0x55a6ec3f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 -- 172.21.15.26:0/1240028836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6ec147c10 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053e39b700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f053db9a700 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a6ec3e8580 tx=0x7f052801fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4dd1a134c7c21d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f052d3ff700 1 -- 172.21.15.26:0/1240028836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528016030 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f052d3ff700 1 -- 172.21.15.26:0/1240028836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0528017040 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.424+0000 7f052d3ff700 1 -- 172.21.15.26:0/1240028836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528004cd0 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1240028836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 msgr2=0x55a6ec3f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55a6ec3e8580 tx=0x7f052801fe40 comp rx=0 tx=0).stop 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1240028836 shutdown_connections 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3df480 0x55a6ec3f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/1240028836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3f0640 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1240028836 >> 172.21.15.26:0/1240028836 conn(0x55a6ec3d4d50 msgr2=0x55a6ec3f35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1240028836 shutdown_connections 2023-07-13T22:13:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- 172.21.15.26:0/1240028836 wait complete. 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.425+0000 7f0553fd9b80 1 -- start start 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 Processor -- start 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 0x55a6ec3ebd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec3ef6f0 con 0x55a6ec3e0c20 2023-07-13T22:13:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec3ef860 con 0x55a6ec3dfdd0 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6ec3efa20 con 0x55a6ec3df480 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f0553fd9b80 10 client.0 _ll_register_callbacks cb 0x55a6ec2ce710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f053eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f053eb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59338/0 (socket says 172.21.15.26:59338) 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f053eb9c700 1 -- 172.21.15.26:0/3389668745 learned_addr learned my addr 172.21.15.26:0/3389668745 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f053e39b700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 0x55a6ec3ebd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.426+0000 7f053db9a700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 -- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3dfdd0 msgr2=0x55a6ec3ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 -- 172.21.15.26:0/3389668745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 msgr2=0x55a6ec3ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 -- 172.21.15.26:0/3389668745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053000e460 con 0x55a6ec3df480 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053eb9c700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f053e39b700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 0x55a6ec3ebd00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0534097b00 tx=0x7f05340cb7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9f9e9f4a52fac9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0521ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:13:26.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053409a9f0 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05340a3450 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0530027070 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053409b090 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0553fd9b80 10 client.25039 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.427+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f0528004260 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0530027070 con 0x55a6ec3df480 2023-07-13T22:13:26.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f053409bf70 con 0x55a6ec3df480 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 10 client.25039 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 10 client.25039 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 20 client.25039 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec44d560 0x55a6ec449ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 4 client.25039 mds_command: new command op to 24575 tid=0[{"prefix": "status"}] 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a6ec2ec000 con 0x55a6ec44d560 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.428+0000 7f053db9a700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec44d560 0x55a6ec449ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f053db9a700 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec44d560 0x55a6ec449ec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c1063525fafc7 server_cookie=aa16e879e3985ced in_seq=0 out_seq=0 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0521ffb700 10 client.25039 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0521ffb700 1 -- 172.21.15.26:0/3389668745 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a6ec2ec000 con 0x55a6ec44d560 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0521ffb700 10 client.25039 handle_command_reply: tid=0 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 client.25039 shutdown 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 msgr2=0x55a6ec3ebd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 0x55a6ec3ebd00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0534097b00 tx=0x7f05340cb7b0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 shutdown_connections 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6ec3e0c20 0x55a6ec3ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6ec44d560 0x55a6ec449ec0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6ec3df480 0x55a6ec3ebd00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 --2- 172.21.15.26:0/3389668745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6ec3dfdd0 0x55a6ec3ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0521ffb700 0 client.25039 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.429+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 >> 172.21.15.26:0/3389668745 conn(0x55a6ec3d4d50 msgr2=0x55a6ec3f0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 shutdown_connections 2023-07-13T22:13:26.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f0553fd9b80 1 -- 172.21.15.26:0/3389668745 wait complete. 2023-07-13T22:13:26.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f0553fd9b80 20 client.25039 trim_cache size 0 max 16384 2023-07-13T22:13:26.433 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:26.433 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:26.433 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-07-13T22:13:26.433 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24575, 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 66.937645841999995, 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:26.434 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 74.808873269000003 2023-07-13T22:13:26.435 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 msgr2=0x7f05240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 0x7f05240a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0530022000 tx=0x7f053001f000 comp rx=0 tx=0).stop 2023-07-13T22:13:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 msgr2=0x7f0538197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0534000c00 tx=0x7f0534005720 comp rx=0 tx=0).stop 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 shutdown_connections 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05381079b0 0x7f053819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0538072f70 0x7f0538197ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f05240a11e0 0x7f05240a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 --2- 172.21.15.26:0/4255977690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0538074550 0x7f0538198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 >> 172.21.15.26:0/4255977690 conn(0x7f05380fb130 msgr2=0x7f05380fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.430+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 shutdown_connections 2023-07-13T22:13:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:26.431+0000 7f05405ff700 1 -- 172.21.15.26:0/4255977690 wait complete. 2023-07-13T22:13:26.438 INFO:tasks.cephadm:Setup complete, yielding 2023-07-13T22:13:26.439 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-13T22:13:26.449 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T22:13:26.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph orch status' 2023-07-13T22:13:26.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:26 smithi026 ceph-mon[95647]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:26.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:26 smithi117 ceph-mon[129074]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:26.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:26 smithi121 ceph-mon[129187]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:27.179 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 -- 172.21.15.26:0/252899874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4100d90 msgr2=0x7ffab41011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 --2- 172.21.15.26:0/252899874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4100d90 0x7ffab41011e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009990 tx=0x7ffaa4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 -- 172.21.15.26:0/252899874 shutdown_connections 2023-07-13T22:13:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 --2- 172.21.15.26:0/252899874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4101720 0x7ffab4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 --2- 172.21.15.26:0/252899874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 --2- 172.21.15.26:0/252899874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4100d90 0x7ffab41011e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 -- 172.21.15.26:0/252899874 >> 172.21.15.26:0/252899874 conn(0x7ffab40fb1a0 msgr2=0x7ffab40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.718+0000 7ffabc172700 1 -- 172.21.15.26:0/252899874 shutdown_connections 2023-07-13T22:13:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.719+0000 7ffabc172700 1 -- 172.21.15.26:0/252899874 wait complete. 2023-07-13T22:13:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 Processor -- start 2023-07-13T22:13:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 -- start start 2023-07-13T22:13:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4100d90 0x7ffab4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab419f650 con 0x7ffab4101720 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab419f790 con 0x7ffab40ffb90 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffabc172700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab419f920 con 0x7ffab4100d90 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffaba70f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.720+0000 7ffaba70f700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59354/0 (socket says 172.21.15.26:59354) 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 -- 172.21.15.26:0/426015229 learned_addr learned my addr 172.21.15.26:0/426015229 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffab9f0e700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffab970d700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4100d90 0x7ffab4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 -- 172.21.15.26:0/426015229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4100d90 msgr2=0x7ffab4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4100d90 0x7ffab4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 -- 172.21.15.26:0/426015229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 msgr2=0x7ffab4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 -- 172.21.15.26:0/426015229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40095f0 con 0x7ffab4101720 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffab9f0e700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaba70f700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffaac007f10 tx=0x7ffaac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac017070 con 0x7ffab4101720 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffaac00f040 con 0x7ffab4101720 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab419fc00 con 0x7ffab4101720 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.721+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005610 con 0x7ffab4101720 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.722+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab41a0150 con 0x7ffab4101720 2023-07-13T22:13:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.723+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab4192210 con 0x7ffab4101720 2023-07-13T22:13:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.725+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ffaac004ad0 con 0x7ffab4101720 2023-07-13T22:13:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.728+0000 7ffaaaffd700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 0x7ffaa00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.728+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffaac0cc7a0 con 0x7ffab4101720 2023-07-13T22:13:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.728+0000 7ffab9f0e700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 0x7ffaa00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.728+0000 7ffab9f0e700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 0x7ffaa00a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4100bf0 tx=0x7ffab0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:28.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.730+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffaac095d40 con 0x7ffab4101720 2023-07-13T22:13:28.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:28 smithi117 ceph-mon[129074]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:28.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.955+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab4060950 con 0x7ffaa00a1260 2023-07-13T22:13:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.957+0000 7ffaaaffd700 1 -- 172.21.15.26:0/426015229 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ffab4060950 con 0x7ffaa00a1260 2023-07-13T22:13:28.947 INFO:teuthology.orchestra.run.smithi026.stdout:Backend: cephadm 2023-07-13T22:13:28.948 INFO:teuthology.orchestra.run.smithi026.stdout:Available: Yes 2023-07-13T22:13:28.948 INFO:teuthology.orchestra.run.smithi026.stdout:Paused: No 2023-07-13T22:13:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 msgr2=0x7ffaa00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 0x7ffaa00a3710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffab4100bf0 tx=0x7ffab0009260 comp rx=0 tx=0).stop 2023-07-13T22:13:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 msgr2=0x7ffab419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffaac007f10 tx=0x7ffaac00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 shutdown_connections 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ffaa00a1260 0x7ffaa00a3710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffab4101720 0x7ffab419f080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffab40ffb90 0x7ffab4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 --2- 172.21.15.26:0/426015229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffab4100d90 0x7ffab4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 >> 172.21.15.26:0/426015229 conn(0x7ffab40fb1a0 msgr2=0x7ffab41066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 shutdown_connections 2023-07-13T22:13:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:28.959+0000 7ffabc172700 1 -- 172.21.15.26:0/426015229 wait complete. 2023-07-13T22:13:28.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:28 smithi026 ceph-mon[95647]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:28.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:28 smithi121 ceph-mon[129187]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:29.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-13T22:13:30.206 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:30.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:30 smithi026 ceph-mon[95647]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:30.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:30 smithi026 ceph-mon[95647]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:30.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:30 smithi117 ceph-mon[129074]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:30.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:30 smithi117 ceph-mon[129074]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:30.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:30 smithi121 ceph-mon[129187]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:30.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:30 smithi121 ceph-mon[129187]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 -- 172.21.15.26:0/3748499033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 msgr2=0x7f6ab00fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 --2- 172.21.15.26:0/3748499033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00fffa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009990 tx=0x7f6aac009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 -- 172.21.15.26:0/3748499033 shutdown_connections 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 --2- 172.21.15.26:0/3748499033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00fffa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 --2- 172.21.15.26:0/3748499033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 0x7f6ab0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 --2- 172.21.15.26:0/3748499033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 0x7f6ab01011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.635+0000 7f6ab850c700 1 -- 172.21.15.26:0/3748499033 >> 172.21.15.26:0/3748499033 conn(0x7f6ab00fb1a0 msgr2=0x7f6ab00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.636+0000 7f6ab850c700 1 -- 172.21.15.26:0/3748499033 shutdown_connections 2023-07-13T22:13:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.636+0000 7f6ab850c700 1 -- 172.21.15.26:0/3748499033 wait complete. 2023-07-13T22:13:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.637+0000 7f6ab850c700 1 Processor -- start 2023-07-13T22:13:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.637+0000 7f6ab850c700 1 -- start start 2023-07-13T22:13:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 0x7f6ab0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 0x7f6ab0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00735c0 con 0x7f6ab00ffb90 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab0073730 con 0x7f6ab0100d90 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab850c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab019f920 con 0x7f6ab0101720 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab62a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab62a8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59372/0 (socket says 172.21.15.26:59372) 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab62a8700 1 -- 172.21.15.26:0/2815178245 learned_addr learned my addr 172.21.15.26:0/2815178245 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab6aa9700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 0x7f6ab0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.638+0000 7f6ab5aa7700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 0x7f6ab0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 -- 172.21.15.26:0/2815178245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 msgr2=0x7f6ab0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 0x7f6ab0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 -- 172.21.15.26:0/2815178245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 msgr2=0x7f6ab0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 0x7f6ab0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 -- 172.21.15.26:0/2815178245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac0095f0 con 0x7f6ab00ffb90 2023-07-13T22:13:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab62a8700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6aac004710 tx=0x7f6aac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac01d070 con 0x7f6ab00ffb90 2023-07-13T22:13:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aac021410 con 0x7f6ab00ffb90 2023-07-13T22:13:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aac017650 con 0x7f6ab00ffb90 2023-07-13T22:13:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab019fba0 con 0x7f6ab00ffb90 2023-07-13T22:13:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.639+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab019ffb0 con 0x7f6ab00ffb90 2023-07-13T22:13:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.640+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab0107170 con 0x7f6ab00ffb90 2023-07-13T22:13:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.643+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6aac021580 con 0x7f6ab00ffb90 2023-07-13T22:13:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.644+0000 7f6aa77fe700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 0x7f6a9c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.644+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6aac0ce1c0 con 0x7f6ab00ffb90 2023-07-13T22:13:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.644+0000 7f6ab5aa7700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 0x7f6a9c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.645+0000 7f6ab5aa7700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 0x7f6a9c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009bf0 tx=0x7f6aa0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.646+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6aac097910 con 0x7f6ab00ffb90 2023-07-13T22:13:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.864+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab0060950 con 0x7f6a9c0a1260 2023-07-13T22:13:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.877+0000 7f6aa77fe700 1 -- 172.21.15.26:0/2815178245 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f6ab0060950 con 0x7f6a9c0a1260 2023-07-13T22:13:31.868 INFO:teuthology.orchestra.run.smithi026.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.a smithi026 running (83s) 8s ago 82s 17.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 93f18eff857e 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.b smithi117 running (80s) 9s ago 80s 32.4M - 18.0.0-4757-g3fc0089c afd12e232ec0 f2c0f1b34b6c 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.c smithi121 running (93s) 9s ago 93s 19.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 cad4444215a0 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.d smithi026 running (100s) 8s ago 99s 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 d007cff9f7c5 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.e smithi117 running (97s) 9s ago 97s 20.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 8d0b67855ae8 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.f smithi121 running (95s) 9s ago 95s 20.9M - 18.0.0-4757-g3fc0089c afd12e232ec0 56a5e85e47c3 2023-07-13T22:13:31.869 INFO:teuthology.orchestra.run.smithi026.stdout:mds.g smithi026 running (86s) 8s ago 85s 16.7M - 18.0.0-4757-g3fc0089c afd12e232ec0 c7293f35388a 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mds.h smithi117 running (91s) 9s ago 91s 20.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 1fdcbd30178e 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mds.i smithi121 running (89s) 9s ago 89s 21.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 b9db45c554d4 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mds.j smithi026 running (103s) 8s ago 102s 13.8M - 18.0.0-4757-g3fc0089c afd12e232ec0 bd91c7333278 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mds.k smithi117 running (106s) 9s ago 105s 21.2M - 18.0.0-4757-g3fc0089c afd12e232ec0 f6f5dfedac8c 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mds.l smithi121 running (108s) 9s ago 107s 19.5M - 18.0.0-4757-g3fc0089c afd12e232ec0 bc972ac8dbd4 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.x smithi026 *:9283,8765 running (8m) 8s ago 8m 524M - 18.0.0-4757-g3fc0089c afd12e232ec0 ff2264bfdf99 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.y smithi117 *:8443,8765 running (6m) 9s ago 6m 447M - 18.0.0-4757-g3fc0089c afd12e232ec0 b92160516541 2023-07-13T22:13:31.870 INFO:teuthology.orchestra.run.smithi026.stdout:mgr.z smithi121 *:8443,8765 running (6m) 9s ago 6m 445M - 18.0.0-4757-g3fc0089c afd12e232ec0 5756aa9662a6 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:mon.a smithi026 running (8m) 8s ago 8m 91.2M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 be719f8b8815 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:mon.b smithi117 running (6m) 9s ago 6m 65.3M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 91c70bb935e6 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:mon.c smithi121 running (6m) 9s ago 6m 64.2M 2048M 18.0.0-4757-g3fc0089c afd12e232ec0 a08963ebc4f4 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 smithi026 running (6m) 8s ago 6m 65.0M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 f9825bff3c52 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 smithi117 running (5m) 9s ago 5m 70.1M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 ea846599db35 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 smithi121 running (5m) 9s ago 5m 66.6M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 11309d9d1a4d 2023-07-13T22:13:31.871 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 smithi026 running (5m) 8s ago 5m 60.2M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 c50f5463e23b 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 smithi117 running (4m) 9s ago 4m 59.5M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 6f059ac2fb90 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 smithi121 running (4m) 9s ago 4m 64.8M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 7c6dffe2950a 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 smithi026 running (3m) 8s ago 3m 58.4M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 e4e567659e43 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 smithi117 running (3m) 9s ago 3m 61.1M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 38d6e2c9cf02 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 smithi121 running (3m) 9s ago 3m 60.0M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 f850c14479c1 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 smithi026 running (2m) 8s ago 2m 56.6M 4318M 18.0.0-4757-g3fc0089c afd12e232ec0 ca7f0accdbca 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 smithi117 running (2m) 9s ago 2m 61.2M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 d3bb3e832c67 2023-07-13T22:13:31.872 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 smithi121 running (2m) 9s ago 2m 56.1M 4292M 18.0.0-4757-g3fc0089c afd12e232ec0 8739f360cdf5 2023-07-13T22:13:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 msgr2=0x7f6a9c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 0x7f6a9c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009bf0 tx=0x7f6aa0009260 comp rx=0 tx=0).stop 2023-07-13T22:13:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 msgr2=0x7f6ab00742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6aac004710 tx=0x7f6aac009580 comp rx=0 tx=0).stop 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 shutdown_connections 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a9c0a1260 0x7f6a9c0a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ab00ffb90 0x7f6ab00742b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ab0101720 0x7f6ab0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 --2- 172.21.15.26:0/2815178245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0100d90 0x7f6ab0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 >> 172.21.15.26:0/2815178245 conn(0x7f6ab00fb1a0 msgr2=0x7f6ab01042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 shutdown_connections 2023-07-13T22:13:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:31.880+0000 7f6ab850c700 1 -- 172.21.15.26:0/2815178245 wait complete. 2023-07-13T22:13:32.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-13T22:13:32.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:32 smithi117 ceph-mon[129074]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:32.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:32 smithi026 ceph-mon[95647]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:32.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:32 smithi121 ceph-mon[129187]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:13:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:33.808 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:33 smithi026 ceph-mon[95647]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:33.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:33 smithi117 ceph-mon[129074]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:33.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:33 smithi121 ceph-mon[129187]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:34.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:34 smithi026 ceph-mon[95647]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 -- 172.21.15.26:0/3316264458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 msgr2=0x7f28f00fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 --2- 172.21.15.26:0/3316264458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f00fffa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009990 tx=0x7f28ec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 -- 172.21.15.26:0/3316264458 shutdown_connections 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 --2- 172.21.15.26:0/3316264458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f00fffa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 --2- 172.21.15.26:0/3316264458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 --2- 172.21.15.26:0/3316264458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f01011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 -- 172.21.15.26:0/3316264458 >> 172.21.15.26:0/3316264458 conn(0x7f28f00fb1a0 msgr2=0x7f28f00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:34.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 -- 172.21.15.26:0/3316264458 shutdown_connections 2023-07-13T22:13:34.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.717+0000 7f28f77f2700 1 -- 172.21.15.26:0/3316264458 wait complete. 2023-07-13T22:13:34.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.718+0000 7f28f77f2700 1 Processor -- start 2023-07-13T22:13:34.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.718+0000 7f28f77f2700 1 -- start start 2023-07-13T22:13:34.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019f6e0 con 0x7f28f00ffb90 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019f820 con 0x7f28f0100d90 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f77f2700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019f9b0 con 0x7f28f0101720 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:34.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47840/0 (socket says 172.21.15.26:47840) 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 -- 172.21.15.26:0/4255861586 learned_addr learned my addr 172.21.15.26:0/4255861586 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f4d8d700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f5d8f700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 -- 172.21.15.26:0/4255861586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 msgr2=0x7f28f019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 -- 172.21.15.26:0/4255861586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 msgr2=0x7f28f0198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f0198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f558e700 1 -- 172.21.15.26:0/4255861586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0095f0 con 0x7f28f00ffb90 2023-07-13T22:13:34.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f4d8d700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:34.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.719+0000 7f28f5d8f700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:34.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28f558e700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00f670 tx=0x7f28ec00f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28ec005510 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f019fc30 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec004d00 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.720+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01a00a0 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.721+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f0192260 con 0x7f28f00ffb90 2023-07-13T22:13:34.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.723+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f28ec00fc30 con 0x7f28f00ffb90 2023-07-13T22:13:34.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.724+0000 7f28e67fc700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 0x7f28dc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:34.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.725+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28ec0cdb50 con 0x7f28f00ffb90 2023-07-13T22:13:34.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.725+0000 7f28f4d8d700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 0x7f28dc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:34.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.725+0000 7f28f4d8d700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 0x7f28dc0a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005fd0 tx=0x7f28e0005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:34.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.729+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f28ec097180 con 0x7f28f00ffb90 2023-07-13T22:13:34.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.943+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f0060950 con 0x7f28dc0a1260 2023-07-13T22:13:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.949+0000 7f28e67fc700 1 -- 172.21.15.26:0/4255861586 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f28f0060950 con 0x7f28dc0a1260 2023-07-13T22:13:34.939 INFO:teuthology.orchestra.run.smithi026.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-13T22:13:34.940 INFO:teuthology.orchestra.run.smithi026.stdout:mds.all 12/12 12s ago 112s smithi026=a;smithi026=d;smithi026=g;smithi026=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi121=c;smithi121=f;smithi121=i;smithi121=l;count:12 2023-07-13T22:13:34.940 INFO:teuthology.orchestra.run.smithi026.stdout:mgr 3/3 12s ago 6m smithi026=x;smithi117=y;smithi121=z;count:3 2023-07-13T22:13:34.940 INFO:teuthology.orchestra.run.smithi026.stdout:mon 3/3 12s ago 7m smithi026:172.21.15.26=a;smithi117:172.21.15.117=b;smithi121:172.21.15.121=c;count:3 2023-07-13T22:13:34.940 INFO:teuthology.orchestra.run.smithi026.stdout:osd 12 12s ago - 2023-07-13T22:13:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 msgr2=0x7f28dc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 0x7f28dc0a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005fd0 tx=0x7f28e0005f60 comp rx=0 tx=0).stop 2023-07-13T22:13:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 msgr2=0x7f28f0197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28ec00f670 tx=0x7f28ec00f750 comp rx=0 tx=0).stop 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 shutdown_connections 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28dc0a1260 0x7f28dc0a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f00ffb90 0x7f28f0197b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28f0101720 0x7f28f019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 --2- 172.21.15.26:0/4255861586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28f0100d90 0x7f28f0198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 >> 172.21.15.26:0/4255861586 conn(0x7f28f00fb1a0 msgr2=0x7f28f0106730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 shutdown_connections 2023-07-13T22:13:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:34.951+0000 7f28f77f2700 1 -- 172.21.15.26:0/4255861586 wait complete. 2023-07-13T22:13:34.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:34 smithi117 ceph-mon[129074]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:34.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:34 smithi121 ceph-mon[129187]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:13:35.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-13T22:13:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:36.759 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:36 smithi026 ceph-mon[95647]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:36.759 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:36 smithi026 ceph-mon[95647]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:36.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:36 smithi117 ceph-mon[129074]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:36.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:36 smithi117 ceph-mon[129074]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:36.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:36 smithi121 ceph-mon[129187]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:36.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:36 smithi121 ceph-mon[129187]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:13:37.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 -- 172.21.15.26:0/215150403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a40ffb90 msgr2=0x7f96a40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:37.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 --2- 172.21.15.26:0/215150403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a40ffb90 0x7f96a40fffa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9698009990 tx=0x7f9698009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 -- 172.21.15.26:0/215150403 shutdown_connections 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 --2- 172.21.15.26:0/215150403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a40ffb90 0x7f96a40fffa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 --2- 172.21.15.26:0/215150403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 0x7f96a4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 --2- 172.21.15.26:0/215150403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a4100d90 0x7f96a41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 -- 172.21.15.26:0/215150403 >> 172.21.15.26:0/215150403 conn(0x7f96a40fb1a0 msgr2=0x7f96a40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 -- 172.21.15.26:0/215150403 shutdown_connections 2023-07-13T22:13:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.678+0000 7f96aab3b700 1 -- 172.21.15.26:0/215150403 wait complete. 2023-07-13T22:13:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.679+0000 7f96aab3b700 1 Processor -- start 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.679+0000 7f96aab3b700 1 -- start start 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a40ffb90 0x7f96a4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 0x7f96a419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a419f670 con 0x7f96a4100d90 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a419f7b0 con 0x7f96a4101720 2023-07-13T22:13:37.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96aab3b700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a419f940 con 0x7f96a40ffb90 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47858/0 (socket says 172.21.15.26:47858) 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 -- 172.21.15.26:0/3771705115 learned_addr learned my addr 172.21.15.26:0/3771705115 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a88d7700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a40ffb90 0x7f96a4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a90d8700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 0x7f96a419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 -- 172.21.15.26:0/3771705115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a40ffb90 msgr2=0x7f96a4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a40ffb90 0x7f96a4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 -- 172.21.15.26:0/3771705115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 msgr2=0x7f96a419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.680+0000 7f96a3fff700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 0x7f96a419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96a3fff700 1 -- 172.21.15.26:0/3771705115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980095f0 con 0x7f96a4100d90 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96a3fff700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f969000d9b0 tx=0x7f969000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690004220 con 0x7f96a4100d90 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9690007640 con 0x7f96a4100d90 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690005590 con 0x7f96a4100d90 2023-07-13T22:13:37.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a419fc20 con 0x7f96a4100d90 2023-07-13T22:13:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.681+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a41a0120 con 0x7f96a4100d90 2023-07-13T22:13:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.682+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a410b3a0 con 0x7f96a4100d90 2023-07-13T22:13:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.684+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9690004750 con 0x7f96a4100d90 2023-07-13T22:13:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.685+0000 7f96a1ffb700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 0x7f96940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.686+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96900cc850 con 0x7f96a4100d90 2023-07-13T22:13:37.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.686+0000 7f96a88d7700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 0x7f96940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:37.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.686+0000 7f96a88d7700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 0x7f96940a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f969800b2e0 tx=0x7f9698005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:37.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.689+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9690095df0 con 0x7f96a4100d90 2023-07-13T22:13:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.900+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f96a41076f0 con 0x7f96940a1260 2023-07-13T22:13:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.902+0000 7f96a1ffb700 1 -- 172.21.15.26:0/3771705115 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f96a41076f0 con 0x7f96940a1260 2023-07-13T22:13:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:HOST ADDR LABELS STATUS 2023-07-13T22:13:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 172.21.15.26 2023-07-13T22:13:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:smithi117 172.21.15.117 2023-07-13T22:13:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:smithi121 172.21.15.121 2023-07-13T22:13:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:3 hosts in cluster 2023-07-13T22:13:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 msgr2=0x7f96940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 0x7f96940a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f969800b2e0 tx=0x7f9698005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 msgr2=0x7f96a4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f969000d9b0 tx=0x7f969000dcc0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 shutdown_connections 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a4101720 0x7f96a419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96a40ffb90 0x7f96a4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96940a1260 0x7f96940a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 --2- 172.21.15.26:0/3771705115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96a4100d90 0x7f96a4198020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 >> 172.21.15.26:0/3771705115 conn(0x7f96a40fb1a0 msgr2=0x7f96a40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 shutdown_connections 2023-07-13T22:13:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:37.904+0000 7f96aab3b700 1 -- 172.21.15.26:0/3771705115 wait complete. 2023-07-13T22:13:38.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-13T22:13:38.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:38 smithi117 ceph-mon[129074]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:38.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:38 smithi026 ceph-mon[95647]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:38.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:38 smithi121 ceph-mon[129187]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:13:39.047 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:39.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:39 smithi117 ceph-mon[129074]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:39.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:39 smithi026 ceph-mon[95647]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:39.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:39 smithi121 ceph-mon[129187]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 -- 172.21.15.26:0/2135848144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 msgr2=0x7f14e80fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:40.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 --2- 172.21.15.26:0/2135848144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e80fffa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009990 tx=0x7f14d8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 -- 172.21.15.26:0/2135848144 shutdown_connections 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 --2- 172.21.15.26:0/2135848144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 --2- 172.21.15.26:0/2135848144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e80fffa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 --2- 172.21.15.26:0/2135848144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 -- 172.21.15.26:0/2135848144 >> 172.21.15.26:0/2135848144 conn(0x7f14e80fb1a0 msgr2=0x7f14e80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 -- 172.21.15.26:0/2135848144 shutdown_connections 2023-07-13T22:13:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.593+0000 7f14ee539700 1 -- 172.21.15.26:0/2135848144 wait complete. 2023-07-13T22:13:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.595+0000 7f14ee539700 1 Processor -- start 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.595+0000 7f14ee539700 1 -- start start 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e819d470 con 0x7f14e8101720 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e819d5b0 con 0x7f14e8100d90 2023-07-13T22:13:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ee539700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e819d740 con 0x7f14e80ffb90 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40738/0 (socket says 172.21.15.26:40738) 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:54068/0 (socket says 172.21.15.26:54068) 2023-07-13T22:13:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e7fff700 1 -- 172.21.15.26:0/3115420036 learned_addr learned my addr 172.21.15.26:0/3115420036 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 -- 172.21.15.26:0/3115420036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 msgr2=0x7f14e81958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 -- 172.21.15.26:0/3115420036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 msgr2=0x7f14e8195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14ecad6700 1 -- 172.21.15.26:0/3115420036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d80095f0 con 0x7f14e8101720 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e7fff700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.596+0000 7f14e77fe700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14ecad6700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e819cea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009f60 tx=0x7f14e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0018070 con 0x7f14e8101720 2023-07-13T22:13:40.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e8072740 con 0x7f14e8101720 2023-07-13T22:13:40.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e000f040 con 0x7f14e8101720 2023-07-13T22:13:40.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e000ccc0 con 0x7f14e8101720 2023-07-13T22:13:40.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.597+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e8072c40 con 0x7f14e8101720 2023-07-13T22:13:40.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.598+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e810b3a0 con 0x7f14e8101720 2023-07-13T22:13:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.600+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f14e0003a00 con 0x7f14e8101720 2023-07-13T22:13:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.601+0000 7f14e57fa700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 0x7f14d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.602+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f14e00cc5c0 con 0x7f14e8101720 2023-07-13T22:13:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.602+0000 7f14e7fff700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 0x7f14d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.602+0000 7f14e7fff700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 0x7f14d00a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b2e0 tx=0x7f14d8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.606+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f14e0095bf0 con 0x7f14e8101720 2023-07-13T22:13:40.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.824+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f14e8060950 con 0x7f14d00a11e0 2023-07-13T22:13:40.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.826+0000 7f14e57fa700 1 -- 172.21.15.26:0/3115420036 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f14e8060950 con 0x7f14d00a11e0 2023-07-13T22:13:40.817 INFO:teuthology.orchestra.run.smithi026.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-13T22:13:40.817 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760020480DGN 447G No 2m ago LVM detected, locked 2023-07-13T22:13:40.817 INFO:teuthology.orchestra.run.smithi026.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 2m ago LVM detected, locked 2023-07-13T22:13:40.817 INFO:teuthology.orchestra.run.smithi026.stdout:smithi121 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZF400BGN 372G No 2m ago LVM detected, locked 2023-07-13T22:13:40.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.828+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 msgr2=0x7f14d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:40.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.828+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 0x7f14d00a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b2e0 tx=0x7f14d8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.828+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 msgr2=0x7f14e819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.828+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e819cea0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009f60 tx=0x7f14e0009f90 comp rx=0 tx=0).stop 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 shutdown_connections 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f14d00a11e0 0x7f14d00a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14e8101720 0x7f14e819cea0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14e80ffb90 0x7f14e81958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 --2- 172.21.15.26:0/3115420036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14e8100d90 0x7f14e8195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:40.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 >> 172.21.15.26:0/3115420036 conn(0x7f14e80fb1a0 msgr2=0x7f14e80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:40.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 shutdown_connections 2023-07-13T22:13:40.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:40.829+0000 7f14ee539700 1 -- 172.21.15.26:0/3115420036 wait complete. 2023-07-13T22:13:40.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:40 smithi117 ceph-mon[129074]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:40.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:40 smithi026 ceph-mon[95647]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:40.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:40 smithi121 ceph-mon[129187]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:13:41.229 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-13T22:13:41.238 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T22:13:41.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-13T22:13:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:42.796 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:42 smithi026 ceph-mon[95647]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:42.797 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:42 smithi026 ceph-mon[95647]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:42.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:42 smithi117 ceph-mon[129074]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:42.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:42 smithi117 ceph-mon[129074]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:42.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:42 smithi121 ceph-mon[129187]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-13T22:13:42.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:42 smithi121 ceph-mon[129187]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:43.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 -- 172.21.15.26:0/2867552451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 msgr2=0x7f2aec0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/2867552451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec0fffa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009990 tx=0x7f2ae0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 -- 172.21.15.26:0/2867552451 shutdown_connections 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/2867552451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec0fffa0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/2867552451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec101720 0x7f2aec107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/2867552451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec100d90 0x7f2aec1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 -- 172.21.15.26:0/2867552451 >> 172.21.15.26:0/2867552451 conn(0x7f2aec0fb1a0 msgr2=0x7f2aec0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:43.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 -- 172.21.15.26:0/2867552451 shutdown_connections 2023-07-13T22:13:43.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.493+0000 7f2af2c1c700 1 -- 172.21.15.26:0/2867552451 wait complete. 2023-07-13T22:13:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.494+0000 7f2af2c1c700 1 Processor -- start 2023-07-13T22:13:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.494+0000 7f2af2c1c700 1 -- start start 2023-07-13T22:13:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:43.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec100d90 0x7f2aec195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec101720 0x7f2aec19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19d450 con 0x7f2aec0ffb90 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19d590 con 0x7f2aec100d90 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af2c1c700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec19d720 con 0x7f2aec101720 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46700/0 (socket says 172.21.15.26:46700) 2023-07-13T22:13:43.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 -- 172.21.15.26:0/1249295101 learned_addr learned my addr 172.21.15.26:0/1249295101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2aebfff700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec100d90 0x7f2aec195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af11b9700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec101720 0x7f2aec19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 -- 172.21.15.26:0/1249295101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec101720 msgr2=0x7f2aec19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec101720 0x7f2aec19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 -- 172.21.15.26:0/1249295101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec100d90 msgr2=0x7f2aec195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec100d90 0x7f2aec195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.495+0000 7f2af09b8700 1 -- 172.21.15.26:0/1249295101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae00095f0 con 0x7f2aec0ffb90 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2af09b8700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005720 tx=0x7f2ae0005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae001d070 con 0x7f2aec0ffb90 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec19d9a0 con 0x7f2aec0ffb90 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae0022470 con 0x7f2aec0ffb90 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0017650 con 0x7f2aec0ffb90 2023-07-13T22:13:43.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.496+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec19de10 con 0x7f2aec0ffb90 2023-07-13T22:13:43.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.497+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec1071d0 con 0x7f2aec0ffb90 2023-07-13T22:13:43.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.499+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2ae00225e0 con 0x7f2aec0ffb90 2023-07-13T22:13:43.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.500+0000 7f2ae9ffb700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 0x7f2adc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:43.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.501+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2ae00cddd0 con 0x7f2aec0ffb90 2023-07-13T22:13:43.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.501+0000 7f2aebfff700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 0x7f2adc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:43.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.501+0000 7f2aebfff700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 0x7f2adc0a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009c60 tx=0x7f2ad8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:43.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.504+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2ae005c790 con 0x7f2aec0ffb90 2023-07-13T22:13:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.759+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2aec02c850 con 0x7f2aec0ffb90 2023-07-13T22:13:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.760+0000 7f2ae9ffb700 1 -- 172.21.15.26:0/1249295101 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2926 (secure 0 0 0) 0x7f2ae0027b80 con 0x7f2aec0ffb90 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:e27 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:legacy client fscid: 1 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:43.751 INFO:teuthology.orchestra.run.smithi026.stdout:Filesystem 'cephfs' (1) 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:fs_name cephfs 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 27 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:created 2023-07-13T22:12:18.516293+0000 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2023-07-13T22:12:26.449971+0000 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:tableserver 0 2023-07-13T22:13:43.752 INFO:teuthology.orchestra.run.smithi026.stdout:root 0 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:session_timeout 300 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:session_autoclose 300 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:max_file_size 1099511627776 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:max_xattr_size 65536 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:required_client_features {} 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure 0 2023-07-13T22:13:43.753 INFO:teuthology.orchestra.run.smithi026.stdout:last_failure_osd_epoch 0 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:max_mds 5 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:in 0,1,2,3,4 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:up {0=24575,1=24563,2=24530,3=24493,4=24467} 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:failed 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:damaged 2023-07-13T22:13:43.754 INFO:teuthology.orchestra.run.smithi026.stdout:stopped 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:data_pools [3] 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:metadata_pool 2 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:inline_data disabled 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:balancer 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:bal_rank_mask -1 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:standby_count_wanted 0 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b{0:24575} state up:active seq 3 addr [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.755 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.i{0:24529} state up:standby-replay seq 1 addr [v2:172.21.15.121:6838/3259819874,v1:172.21.15.121:6839/3259819874] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a{1:24563} state up:active seq 5 addr [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.e{1:24509} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/1862924316,v1:172.21.15.117:6835/1862924316] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.h{2:24530} state up:active seq 7 addr [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.121:6836/1505729647,v1:172.21.15.121:6837/1505729647] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.f{3:24493} state up:active seq 9 addr [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.l{3:24451} state up:standby-replay seq 1 addr [v2:172.21.15.121:6832/930793918,v1:172.21.15.121:6833/930793918] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.756 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.k{4:24467} state up:active seq 11 addr [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.g{4:14520} state up:standby-replay seq 1 addr [v2:172.21.15.26:6838/1019342380,v1:172.21.15.26:6839/1019342380] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout:Standby daemons: 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.j{-1:14466} state up:standby seq 1 addr [v2:172.21.15.26:6834/956191234,v1:172.21.15.26:6835/956191234] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.757 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d{-1:14478} state up:standby seq 1 addr [v2:172.21.15.26:6836/3391570568,v1:172.21.15.26:6837/3391570568] compat {c=[1],r=[1],i=[7ff]}] 2023-07-13T22:13:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 msgr2=0x7f2adc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 0x7f2adc0a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009c60 tx=0x7f2ad8009260 comp rx=0 tx=0).stop 2023-07-13T22:13:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 msgr2=0x7f2aec1958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005720 tx=0x7f2ae0005750 comp rx=0 tx=0).stop 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 shutdown_connections 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2adc0a1260 0x7f2adc0a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aec0ffb90 0x7f2aec1958c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2aec101720 0x7f2aec19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 --2- 172.21.15.26:0/1249295101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2aec100d90 0x7f2aec195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 >> 172.21.15.26:0/1249295101 conn(0x7f2aec0fb1a0 msgr2=0x7f2aec104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 shutdown_connections 2023-07-13T22:13:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:43.762+0000 7f2af2c1c700 1 -- 172.21.15.26:0/1249295101 wait complete. 2023-07-13T22:13:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:44.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-13T22:13:44.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:44 smithi117 ceph-mon[129074]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:13:44.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:44 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1249295101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-13T22:13:44.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:44 smithi026 ceph-mon[95647]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:13:44.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:44 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1249295101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-13T22:13:44.986 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:44.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:44 smithi121 ceph-mon[129187]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:13:44.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:44 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1249295101' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.431+0000 7fb4efa28700 1 -- 172.21.15.26:0/443513454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 msgr2=0x7fb4e80fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.431+0000 7fb4efa28700 1 --2- 172.21.15.26:0/443513454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e80fffa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 -- 172.21.15.26:0/443513454 shutdown_connections 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 --2- 172.21.15.26:0/443513454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e80fffa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 --2- 172.21.15.26:0/443513454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8101720 0x7fb4e8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 --2- 172.21.15.26:0/443513454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8100d90 0x7fb4e81011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 -- 172.21.15.26:0/443513454 >> 172.21.15.26:0/443513454 conn(0x7fb4e80fb1a0 msgr2=0x7fb4e80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 -- 172.21.15.26:0/443513454 shutdown_connections 2023-07-13T22:13:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.432+0000 7fb4efa28700 1 -- 172.21.15.26:0/443513454 wait complete. 2023-07-13T22:13:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 Processor -- start 2023-07-13T22:13:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 -- start start 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 0x7fb4e8195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 0x7fb4e819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819d470 con 0x7fb4e80ffb90 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819d5b0 con 0x7fb4e8100d90 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.433+0000 7fb4efa28700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e819d740 con 0x7fb4e8101720 2023-07-13T22:13:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46706/0 (socket says 172.21.15.26:46706) 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 -- 172.21.15.26:0/3188224790 learned_addr learned my addr 172.21.15.26:0/3188224790 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ecfc3700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 0x7fb4e8195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4edfc5700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 0x7fb4e819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 -- 172.21.15.26:0/3188224790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 msgr2=0x7fb4e819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 0x7fb4e819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 -- 172.21.15.26:0/3188224790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 msgr2=0x7fb4e8195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 0x7fb4e8195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 -- 172.21.15.26:0/3188224790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40095f0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4edfc5700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 0x7fb4e819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ecfc3700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 0x7fb4e8195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4ed7c4700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4005740 tx=0x7fb4e4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.434+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e401d070 con 0x7fb4e80ffb90 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.435+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4e400fbb0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.435+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e819d9c0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.435+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017450 con 0x7fb4e80ffb90 2023-07-13T22:13:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.435+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e819de30 con 0x7fb4e80ffb90 2023-07-13T22:13:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.436+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e810b3a0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.438+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb4e400fd20 con 0x7fb4e80ffb90 2023-07-13T22:13:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.439+0000 7fb4de7fc700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 0x7fb4d40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.439+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb4e40cdda0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.439+0000 7fb4ecfc3700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 0x7fb4d40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.440+0000 7fb4ecfc3700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 0x7fb4d40a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8010420 tx=0x7fb4d80073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:46.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.443+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb4e40974f0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.651+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb4e8106ad0 con 0x7fb4e80ffb90 2023-07-13T22:13:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.652+0000 7fb4de7fc700 1 -- 172.21.15.26:0/3188224790 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) v1 ==== 56+0+5275 (secure 0 0 0) 0x7fb4e409ab10 con 0x7fb4e80ffb90 2023-07-13T22:13:46.642 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 85 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 23b26990-21c9-11ee-9b32-001a4aab830c 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:created 2023-07-13T22:04:53.734470+0000 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:modified 2023-07-13T22:12:52.529361+0000 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:crush_version 26 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:full_ratio 0.95 2023-07-13T22:13:46.643 INFO:teuthology.orchestra.run.smithi026.stdout:backfillfull_ratio 0.9 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:nearfull_ratio 0.85 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:require_min_compat_client luminous 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:min_compat_client luminous 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:require_osd_release reef 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:stretch_mode_enabled false 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-13T22:13:46.644 INFO:teuthology.orchestra.run.smithi026.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:max_osd 12 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6802/1617573741,v1:172.21.15.26:6803/1617573741] [v2:172.21.15.26:6804/1617573741,v1:172.21.15.26:6805/1617573741] exists,up 5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/2207798137,v1:172.21.15.117:6801/2207798137] [v2:172.21.15.117:6802/2207798137,v1:172.21.15.117:6803/2207798137] exists,up b5fb060a-61be-4007-809f-69c2e19ed034 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/1910431229,v1:172.21.15.121:6801/1910431229] [v2:172.21.15.121:6802/1910431229,v1:172.21.15.121:6803/1910431229] exists,up 79f28811-85d8-4f49-a4eb-c8a30a9b606b 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6810/3949990107,v1:172.21.15.26:6811/3949990107] [v2:172.21.15.26:6812/3949990107,v1:172.21.15.26:6813/3949990107] exists,up 609be474-a324-4303-b916-e112222f2ab4 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/4231906293,v1:172.21.15.117:6809/4231906293] [v2:172.21.15.117:6810/4231906293,v1:172.21.15.117:6811/4231906293] exists,up 244086b9-04cf-4892-a23a-ca2d9050394a 2023-07-13T22:13:46.645 INFO:teuthology.orchestra.run.smithi026.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/37316284,v1:172.21.15.121:6809/37316284] [v2:172.21.15.121:6810/37316284,v1:172.21.15.121:6811/37316284] exists,up a1497c31-c3f2-4912-b954-a9996a18cfd6 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6818/38495361,v1:172.21.15.26:6819/38495361] [v2:172.21.15.26:6820/38495361,v1:172.21.15.26:6821/38495361] exists,up a6b07e8a-bab9-4cf6-8af9-86655ea5b994 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.7 up in weight 1 up_from 51 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/2078940606,v1:172.21.15.117:6817/2078940606] [v2:172.21.15.117:6818/2078940606,v1:172.21.15.117:6819/2078940606] exists,up 102fca76-132a-4dfd-86d3-f8adfc6caf04 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.8 up in weight 1 up_from 58 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/4171528294,v1:172.21.15.121:6817/4171528294] [v2:172.21.15.121:6818/4171528294,v1:172.21.15.121:6819/4171528294] exists,up dac2076e-cc3e-47d2-8d8c-267556ea9a0f 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.9 up in weight 1 up_from 66 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.26:6826/411448379,v1:172.21.15.26:6827/411448379] [v2:172.21.15.26:6828/411448379,v1:172.21.15.26:6829/411448379] exists,up 9a2b112d-b9df-4867-bbfe-7d3455f384ca 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.10 up in weight 1 up_from 71 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/1101018481,v1:172.21.15.117:6825/1101018481] [v2:172.21.15.117:6826/1101018481,v1:172.21.15.117:6827/1101018481] exists,up a52c43f1-5b23-401c-bea3-05c5bacc3ca7 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:osd.11 up in weight 1 up_from 78 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/1646753631,v1:172.21.15.121:6825/1646753631] [v2:172.21.15.121:6826/1646753631,v1:172.21.15.121:6827/1646753631] exists,up 43cbee7d-bb99-429b-a99a-102d7497f251 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.8 [2,11] 2023-07-13T22:13:46.646 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.c [2,4] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 2.1d [8,11] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.3 [4,9] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.18 [3,9] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.28 [5,6] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.31 [4,8] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:pg_upmap_items 3.36 [4,9] 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1422542320 expires 2023-07-14T22:05:51.339234+0000 2023-07-13T22:13:46.647 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/3258657934 expires 2023-07-14T22:05:51.339234+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2685973797 expires 2023-07-14T22:05:24.923026+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/551567635 expires 2023-07-14T22:05:24.923026+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/2707453820 expires 2023-07-14T22:05:51.339234+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/2707453820 expires 2023-07-14T22:05:51.339234+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/2272047376 expires 2023-07-14T22:05:51.339234+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6800/3558776409 expires 2023-07-14T22:05:24.923026+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:6801/3558776409 expires 2023-07-14T22:05:24.923026+0000 2023-07-13T22:13:46.648 INFO:teuthology.orchestra.run.smithi026.stdout:blocklist 172.21.15.26:0/1729376159 expires 2023-07-14T22:05:24.923026+0000 2023-07-13T22:13:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 msgr2=0x7fb4d40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 0x7fb4d40a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8010420 tx=0x7fb4d80073d0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 msgr2=0x7fb4e81958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4005740 tx=0x7fb4e4005770 comp rx=0 tx=0).stop 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 shutdown_connections 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb4d40a1260 0x7fb4d40a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4e80ffb90 0x7fb4e81958e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4e8101720 0x7fb4e819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 --2- 172.21.15.26:0/3188224790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4e8100d90 0x7fb4e8195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.654+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 >> 172.21.15.26:0/3188224790 conn(0x7fb4e80fb1a0 msgr2=0x7fb4e80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.655+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 shutdown_connections 2023-07-13T22:13:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:46.655+0000 7fb4efa28700 1 -- 172.21.15.26:0/3188224790 wait complete. 2023-07-13T22:13:46.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:46 smithi117 ceph-mon[129074]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-13T22:13:46.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:46 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3188224790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-13T22:13:46.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:46 smithi026 ceph-mon[95647]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-13T22:13:46.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:46 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3188224790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-13T22:13:46.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:46 smithi121 ceph-mon[129187]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2023-07-13T22:13:46.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:46 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3188224790' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-13T22:13:47.107 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-13T22:13:47.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 -- 172.21.15.26:0/778092985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0100cf0 msgr2=0x7febf0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 --2- 172.21.15.26:0/778092985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0100cf0 0x7febf0101140 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7febe4009990 tx=0x7febe4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 -- 172.21.15.26:0/778092985 shutdown_connections 2023-07-13T22:13:47.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 --2- 172.21.15.26:0/778092985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0101680 0x7febf0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 --2- 172.21.15.26:0/778092985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 --2- 172.21.15.26:0/778092985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0100cf0 0x7febf0101140 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 -- 172.21.15.26:0/778092985 >> 172.21.15.26:0/778092985 conn(0x7febf00fb0e0 msgr2=0x7febf00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 -- 172.21.15.26:0/778092985 shutdown_connections 2023-07-13T22:13:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.337+0000 7febf61ff700 1 -- 172.21.15.26:0/778092985 wait complete. 2023-07-13T22:13:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 Processor -- start 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 -- start start 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0100cf0 0x7febf0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 0x7febf019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf019f5a0 con 0x7febf0101680 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf019f6e0 con 0x7febf00ffb00 2023-07-13T22:13:47.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.339+0000 7febf61ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf019f870 con 0x7febf0100cf0 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48152/0 (socket says 172.21.15.26:48152) 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 -- 172.21.15.26:0/1778486388 learned_addr learned my addr 172.21.15.26:0/1778486388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febeffff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 0x7febf019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febeeffd700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0100cf0 0x7febf0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 -- 172.21.15.26:0/1778486388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0100cf0 msgr2=0x7febf0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0100cf0 0x7febf0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 -- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 msgr2=0x7febf019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 0x7febf019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 -- 172.21.15.26:0/1778486388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40095f0 con 0x7febf00ffb00 2023-07-13T22:13:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febeffff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 0x7febf019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:47.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febef7fe700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7febe000e970 tx=0x7febe000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.340+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0009790 con 0x7febf00ffb00 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.341+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe000c7e0 con 0x7febf00ffb00 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.341+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf019fb50 con 0x7febf00ffb00 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.341+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0003c70 con 0x7febf00ffb00 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.341+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf01a0060 con 0x7febf00ffb00 2023-07-13T22:13:47.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.342+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf010b320 con 0x7febf00ffb00 2023-07-13T22:13:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.344+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7febe000cc50 con 0x7febf00ffb00 2023-07-13T22:13:47.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.345+0000 7febecff9700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 0x7febd80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.345+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe0014070 con 0x7febf00ffb00 2023-07-13T22:13:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.345+0000 7febeeffd700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 0x7febd80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.346+0000 7febeeffd700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 0x7febd80a36d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febe400b2e0 tx=0x7febe4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:47.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.347+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7febe0096380 con 0x7febf00ffb00 2023-07-13T22:13:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.560+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febf010b4e0 con 0x7febf00ffb00 2023-07-13T22:13:47.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.562+0000 7febecff9700 1 -- 172.21.15.26:0/1778486388 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7febe0003dd0 con 0x7febf00ffb00 2023-07-13T22:13:47.553 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:47.553 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:47.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 msgr2=0x7febd80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 0x7febd80a36d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7febe400b2e0 tx=0x7febe4005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 msgr2=0x7febf0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7febe000e970 tx=0x7febe000ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 shutdown_connections 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7febd80a1220 0x7febd80a36d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7febf0101680 0x7febf019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febf00ffb00 0x7febf0197a20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 --2- 172.21.15.26:0/1778486388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febf0100cf0 0x7febf0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 >> 172.21.15.26:0/1778486388 conn(0x7febf00fb0e0 msgr2=0x7febf00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 shutdown_connections 2023-07-13T22:13:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.565+0000 7febf61ff700 1 -- 172.21.15.26:0/1778486388 wait complete. 2023-07-13T22:13:47.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:13:47.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:13:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 -- 172.21.15.26:0/2818958830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 msgr2=0x7f5b2c107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 --2- 172.21.15.26:0/2818958830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c107c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b0a0 tx=0x7f5b2400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 -- 172.21.15.26:0/2818958830 shutdown_connections 2023-07-13T22:13:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 --2- 172.21.15.26:0/2818958830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c107c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 --2- 172.21.15.26:0/2818958830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 0x7f5b2c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 --2- 172.21.15.26:0/2818958830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 0x7f5b2c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 -- 172.21.15.26:0/2818958830 >> 172.21.15.26:0/2818958830 conn(0x7f5b2c0fa000 msgr2=0x7f5b2c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:47.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 -- 172.21.15.26:0/2818958830 shutdown_connections 2023-07-13T22:13:47.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.793+0000 7f5b32fff700 1 -- 172.21.15.26:0/2818958830 wait complete. 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.794+0000 7f5b32fff700 1 Processor -- start 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.794+0000 7f5b32fff700 1 -- start start 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 0x7f5b2c0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 0x7f5b2c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0ff1f0 con 0x7f5b2c1057a0 2023-07-13T22:13:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0ff360 con 0x7f5b2c102e80 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b32fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19f970 con 0x7f5b2c100560 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46738/0 (socket says 172.21.15.26:46738) 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 -- 172.21.15.26:0/1519230099 learned_addr learned my addr 172.21.15.26:0/1519230099 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b30d9b700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 0x7f5b2c0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b2bfff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 0x7f5b2c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 -- 172.21.15.26:0/1519230099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 msgr2=0x7f5b2c0ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 0x7f5b2c0ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 -- 172.21.15.26:0/1519230099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 msgr2=0x7f5b2c0fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 0x7f5b2c0fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.795+0000 7f5b3159c700 1 -- 172.21.15.26:0/1519230099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24009d00 con 0x7f5b2c1057a0 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b3159c700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b780 tx=0x7f5b2400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400e040 con 0x7f5b2c1057a0 2023-07-13T22:13:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b24004c50 con 0x7f5b2c1057a0 2023-07-13T22:13:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c19fbf0 con 0x7f5b2c1057a0 2023-07-13T22:13:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24005280 con 0x7f5b2c1057a0 2023-07-13T22:13:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.796+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c1a0020 con 0x7f5b2c1057a0 2023-07-13T22:13:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.797+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c0fbb70 con 0x7f5b2c1057a0 2023-07-13T22:13:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.799+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5b2401d470 con 0x7f5b2c1057a0 2023-07-13T22:13:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.800+0000 7f5b29ffb700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 0x7f5b1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.800+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5b240ce150 con 0x7f5b2c1057a0 2023-07-13T22:13:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.801+0000 7f5b30d9b700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 0x7f5b1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:47.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.801+0000 7f5b30d9b700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 0x7f5b1c0a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009c60 tx=0x7f5b20009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:47.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:47.804+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5b24097780 con 0x7f5b2c1057a0 2023-07-13T22:13:47.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:47 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1778486388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:47.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:47 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1778486388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:47.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:47 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1778486388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.011+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b2c02ca70 con 0x7f5b2c1057a0 2023-07-13T22:13:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.011+0000 7f5b29ffb700 1 -- 172.21.15.26:0/1519230099 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b2409ada0 con 0x7f5b2c1057a0 2023-07-13T22:13:48.002 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:13:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 msgr2=0x7f5b1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 0x7f5b1c0a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009c60 tx=0x7f5b20009260 comp rx=0 tx=0).stop 2023-07-13T22:13:48.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 msgr2=0x7f5b2c0fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b780 tx=0x7f5b2400bab0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 shutdown_connections 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5b1c0a11e0 0x7f5b1c0a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b2c1057a0 0x7f5b2c0fea50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2c100560 0x7f5b2c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 --2- 172.21.15.26:0/1519230099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2c102e80 0x7f5b2c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 >> 172.21.15.26:0/1519230099 conn(0x7f5b2c0fa000 msgr2=0x7f5b2c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 shutdown_connections 2023-07-13T22:13:48.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.014+0000 7f5b32fff700 1 -- 172.21.15.26:0/1519230099 wait complete. 2023-07-13T22:13:48.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:13:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 -- 172.21.15.26:0/1614232033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed481079a0 msgr2=0x7fed48109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/1614232033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed481079a0 0x7fed48109e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fed4000b0a0 tx=0x7fed4000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 -- 172.21.15.26:0/1614232033 shutdown_connections 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/1614232033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed481079a0 0x7fed48109e20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/1614232033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 0x7fed480729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/1614232033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed48072f70 0x7fed480733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 -- 172.21.15.26:0/1614232033 >> 172.21.15.26:0/1614232033 conn(0x7fed480fb0e0 msgr2=0x7fed480fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.239+0000 7fed4f5ff700 1 -- 172.21.15.26:0/1614232033 shutdown_connections 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.240+0000 7fed4f5ff700 1 -- 172.21.15.26:0/1614232033 wait complete. 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.240+0000 7fed4f5ff700 1 Processor -- start 2023-07-13T22:13:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 -- start start 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 0x7fed481980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481079a0 0x7fed4819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819f710 con 0x7fed48072f70 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819f850 con 0x7fed481079a0 2023-07-13T22:13:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4f5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4819f9e0 con 0x7fed48074550 2023-07-13T22:13:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46760/0 (socket says 172.21.15.26:46760) 2023-07-13T22:13:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 -- 172.21.15.26:0/2272958280 learned_addr learned my addr 172.21.15.26:0/2272958280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4cb9a700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 0x7fed481980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4db9c700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481079a0 0x7fed4819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 -- 172.21.15.26:0/2272958280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 msgr2=0x7fed481980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 0x7fed481980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.241+0000 7fed4d39b700 1 -- 172.21.15.26:0/2272958280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481079a0 msgr2=0x7fed4819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed4d39b700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481079a0 0x7fed4819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed4d39b700 1 -- 172.21.15.26:0/2272958280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed40009d00 con 0x7fed48072f70 2023-07-13T22:13:48.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed4d39b700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e970 tx=0x7fed4400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44009790 con 0x7fed48072f70 2023-07-13T22:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed4400c7e0 con 0x7fed48072f70 2023-07-13T22:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4819fcc0 con 0x7fed48072f70 2023-07-13T22:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed440054e0 con 0x7fed48072f70 2023-07-13T22:13:48.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.242+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed481a01d0 con 0x7fed48072f70 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.243+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48106050 con 0x7fed48072f70 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.245+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fed4400c950 con 0x7fed48072f70 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.246+0000 7fed3e7fc700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 0x7fed340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.247+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fed44014070 con 0x7fed48072f70 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.247+0000 7fed4cb9a700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 0x7fed340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.247+0000 7fed4cb9a700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 0x7fed340a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed48074270 tx=0x7fed38009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:48.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.250+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fed440962a0 con 0x7fed48072f70 2023-07-13T22:13:48.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.461+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed4802cac0 con 0x7fed48072f70 2023-07-13T22:13:48.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.461+0000 7fed3e7fc700 1 -- 172.21.15.26:0/2272958280 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed440998c0 con 0x7fed48072f70 2023-07-13T22:13:48.452 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:13:48.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 msgr2=0x7fed340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 0x7fed340a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fed48074270 tx=0x7fed38009260 comp rx=0 tx=0).stop 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 msgr2=0x7fed48197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e970 tx=0x7fed4400ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 shutdown_connections 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed340a11e0 0x7fed340a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed48072f70 0x7fed48197b90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed481079a0 0x7fed4819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 --2- 172.21.15.26:0/2272958280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed48074550 0x7fed481980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 >> 172.21.15.26:0/2272958280 conn(0x7fed480fb0e0 msgr2=0x7fed480fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:48.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 shutdown_connections 2023-07-13T22:13:48.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.464+0000 7fed4f5ff700 1 -- 172.21.15.26:0/2272958280 wait complete. 2023-07-13T22:13:48.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:13:48.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 -- 172.21.15.26:0/2500688266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c100d40 msgr2=0x7ff08c101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 --2- 172.21.15.26:0/2500688266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c100d40 0x7ff08c101190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff080009990 tx=0x7ff080009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 -- 172.21.15.26:0/2500688266 shutdown_connections 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 --2- 172.21.15.26:0/2500688266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c0ffb50 0x7ff08c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 --2- 172.21.15.26:0/2500688266 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c1016d0 0x7ff08c107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 --2- 172.21.15.26:0/2500688266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c100d40 0x7ff08c101190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 -- 172.21.15.26:0/2500688266 >> 172.21.15.26:0/2500688266 conn(0x7ff08c0fb170 msgr2=0x7ff08c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:48.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 -- 172.21.15.26:0/2500688266 shutdown_connections 2023-07-13T22:13:48.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.690+0000 7ff0919ff700 1 -- 172.21.15.26:0/2500688266 wait complete. 2023-07-13T22:13:48.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 Processor -- start 2023-07-13T22:13:48.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 -- start start 2023-07-13T22:13:48.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c19f580 con 0x7ff08c100d40 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c19f6c0 con 0x7ff08c0ffb50 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.691+0000 7ff0919ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c19f850 con 0x7ff08c1016d0 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36458/0 (socket says 172.21.15.26:36458) 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48222/0 (socket says 172.21.15.26:48222) 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08b7fe700 1 -- 172.21.15.26:0/873246356 learned_addr learned my addr 172.21.15.26:0/873246356 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46764/0 (socket says 172.21.15.26:46764) 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 -- 172.21.15.26:0/873246356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 msgr2=0x7ff08c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 -- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 msgr2=0x7ff08c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 -- 172.21.15.26:0/873246356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800095f0 con 0x7ff08c0ffb50 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08b7fe700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08a7fc700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.692+0000 7ff08affd700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00e9c0 tx=0x7ff07c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.693+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c004220 con 0x7ff08c0ffb50 2023-07-13T22:13:48.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.693+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff07c004380 con 0x7ff08c0ffb50 2023-07-13T22:13:48.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.693+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c003be0 con 0x7ff08c0ffb50 2023-07-13T22:13:48.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.693+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c19fb30 con 0x7ff08c0ffb50 2023-07-13T22:13:48.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.693+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c19fff0 con 0x7ff08c0ffb50 2023-07-13T22:13:48.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.694+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c192160 con 0x7ff08c0ffb50 2023-07-13T22:13:48.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.696+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff07c003d40 con 0x7ff08c0ffb50 2023-07-13T22:13:48.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.697+0000 7ff0909fd700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 0x7ff0740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:48.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.697+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff07c014070 con 0x7ff08c0ffb50 2023-07-13T22:13:48.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.697+0000 7ff08a7fc700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 0x7ff0740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:48.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.698+0000 7ff08a7fc700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 0x7ff0740a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b2e0 tx=0x7ff080005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:48.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.701+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff07c096320 con 0x7ff08c0ffb50 2023-07-13T22:13:48.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.912+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff08c02cac0 con 0x7ff08c0ffb50 2023-07-13T22:13:48.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.913+0000 7ff0909fd700 1 -- 172.21.15.26:0/873246356 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff07c099940 con 0x7ff08c0ffb50 2023-07-13T22:13:48.903 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:13:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 msgr2=0x7ff0740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 0x7ff0740a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b2e0 tx=0x7ff080005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 msgr2=0x7ff08c197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00e9c0 tx=0x7ff07c00ecd0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 shutdown_connections 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff08c1016d0 0x7ff08c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff08c0ffb50 0x7ff08c197a00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff0740a11e0 0x7ff0740a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 --2- 172.21.15.26:0/873246356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff08c100d40 0x7ff08c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 >> 172.21.15.26:0/873246356 conn(0x7ff08c0fb170 msgr2=0x7ff08c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.915+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 shutdown_connections 2023-07-13T22:13:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:48.916+0000 7ff0919ff700 1 -- 172.21.15.26:0/873246356 wait complete. 2023-07-13T22:13:48.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:48 smithi026 ceph-mon[95647]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:13:48.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:48 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1519230099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:48.922 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:48 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2272958280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:48.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:13:48.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:48 smithi117 ceph-mon[129074]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:13:48.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:48 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1519230099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:48.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:48 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2272958280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:48.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:48 smithi121 ceph-mon[129187]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:13:48.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:48 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1519230099' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:48.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:48 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2272958280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:49.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1423034152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f8101680 msgr2=0x7fc2f8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1423034152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f8101680 0x7fc2f8107f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000b0a0 tx=0x7fc2f000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1423034152 shutdown_connections 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1423034152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f8101680 0x7fc2f8107f40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1423034152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f80ffb00 0x7fc2f80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1423034152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8100cf0 0x7fc2f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.137+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1423034152 >> 172.21.15.26:0/1423034152 conn(0x7fc2f80fb0e0 msgr2=0x7fc2f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.138+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1423034152 shutdown_connections 2023-07-13T22:13:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.138+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1423034152 wait complete. 2023-07-13T22:13:49.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 Processor -- start 2023-07-13T22:13:49.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 -- start start 2023-07-13T22:13:49.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f8100cf0 0x7fc2f8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8101680 0x7fc2f819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819d480 con 0x7fc2f80ffb00 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819d5c0 con 0x7fc2f8100cf0 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2ff7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f819d750 con 0x7fc2f8101680 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2fd59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.139+0000 7fc2fd59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46784/0 (socket says 172.21.15.26:46784) 2023-07-13T22:13:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 -- 172.21.15.26:0/1250728958 learned_addr learned my addr 172.21.15.26:0/1250728958 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fcd9a700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f8100cf0 0x7fc2f8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fdd9c700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8101680 0x7fc2f819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 -- 172.21.15.26:0/1250728958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8101680 msgr2=0x7fc2f819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8101680 0x7fc2f819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 -- 172.21.15.26:0/1250728958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f8100cf0 msgr2=0x7fc2f8195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f8100cf0 0x7fc2f8195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 -- 172.21.15.26:0/1250728958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4009520 con 0x7fc2f80ffb00 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2fd59b700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400ea50 tx=0x7fc2f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f400cbd0 con 0x7fc2f80ffb00 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f0009d00 con 0x7fc2f80ffb00 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2f400cd30 con 0x7fc2f80ffb00 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.140+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4018680 con 0x7fc2f80ffb00 2023-07-13T22:13:49.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.141+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f819de00 con 0x7fc2f80ffb00 2023-07-13T22:13:49.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.142+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f810b330 con 0x7fc2f80ffb00 2023-07-13T22:13:49.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.144+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc2f4010430 con 0x7fc2f80ffb00 2023-07-13T22:13:49.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.145+0000 7fc2ee7fc700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 0x7fc2e40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.145+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc2f4014070 con 0x7fc2f80ffb00 2023-07-13T22:13:49.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.145+0000 7fc2fcd9a700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 0x7fc2e40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.146+0000 7fc2fcd9a700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 0x7fc2e40a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009c60 tx=0x7fc2e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:49.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.149+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc2f40d0030 con 0x7fc2f80ffb00 2023-07-13T22:13:49.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.405+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2f810b550 con 0x7fc2f80ffb00 2023-07-13T22:13:49.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.405+0000 7fc2ee7fc700 1 -- 172.21.15.26:0/1250728958 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fc2f810b550 con 0x7fc2f80ffb00 2023-07-13T22:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:49.396 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:13:49.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 msgr2=0x7fc2e40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 0x7fc2e40a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009c60 tx=0x7fc2e8009260 comp rx=0 tx=0).stop 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 msgr2=0x7fc2f8195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2f400ea50 tx=0x7fc2f400ed60 comp rx=0 tx=0).stop 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 shutdown_connections 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc2e40a1220 0x7fc2e40a36d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2f80ffb00 0x7fc2f8195900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc2f8101680 0x7fc2f819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 --2- 172.21.15.26:0/1250728958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2f8100cf0 0x7fc2f8195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 >> 172.21.15.26:0/1250728958 conn(0x7fc2f80fb0e0 msgr2=0x7fc2f80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 shutdown_connections 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.408+0000 7fc2ff7ff700 1 -- 172.21.15.26:0/1250728958 wait complete. 2023-07-13T22:13:49.400 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:49.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:49.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 -- 172.21.15.26:0/3046260640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b40100cf0 msgr2=0x7f1b40101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 --2- 172.21.15.26:0/3046260640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b40100cf0 0x7f1b40101140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009990 tx=0x7f1b34009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 -- 172.21.15.26:0/3046260640 shutdown_connections 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 --2- 172.21.15.26:0/3046260640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40101680 0x7f1b40107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 --2- 172.21.15.26:0/3046260640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b400ffb00 0x7f1b400fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 --2- 172.21.15.26:0/3046260640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b40100cf0 0x7f1b40101140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 -- 172.21.15.26:0/3046260640 >> 172.21.15.26:0/3046260640 conn(0x7f1b400fb0e0 msgr2=0x7f1b400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:49.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 -- 172.21.15.26:0/3046260640 shutdown_connections 2023-07-13T22:13:49.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.633+0000 7f1b45fff700 1 -- 172.21.15.26:0/3046260640 wait complete. 2023-07-13T22:13:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.634+0000 7f1b45fff700 1 Processor -- start 2023-07-13T22:13:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.634+0000 7f1b45fff700 1 -- start start 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.634+0000 7f1b45fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.634+0000 7f1b45fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 0x7f1b40195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.634+0000 7f1b45fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b40101680 0x7f1b4019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b45fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019d410 con 0x7f1b400ffb00 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b45fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019d550 con 0x7f1b40100cf0 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b45fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4019d6e0 con 0x7f1b40101680 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46800/0 (socket says 172.21.15.26:46800) 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 -- 172.21.15.26:0/92703208 learned_addr learned my addr 172.21.15.26:0/92703208 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3ffff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b40101680 0x7f1b4019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3effd700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 0x7f1b40195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 -- 172.21.15.26:0/92703208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b40101680 msgr2=0x7f1b4019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b40101680 0x7f1b4019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 -- 172.21.15.26:0/92703208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 msgr2=0x7f1b40195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 0x7f1b40195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 -- 172.21.15.26:0/92703208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340095f0 con 0x7f1b400ffb00 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3effd700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 0x7f1b40195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.635+0000 7f1b3f7fe700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000e970 tx=0x7f1b3000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.636+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30009790 con 0x7f1b400ffb00 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.636+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b3000eec0 con 0x7f1b400ffb00 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.636+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30003cf0 con 0x7f1b400ffb00 2023-07-13T22:13:49.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.636+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4019d9c0 con 0x7f1b400ffb00 2023-07-13T22:13:49.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.636+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4019de80 con 0x7f1b400ffb00 2023-07-13T22:13:49.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.637+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b40107050 con 0x7f1b400ffb00 2023-07-13T22:13:49.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.639+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1b3000ccd0 con 0x7f1b400ffb00 2023-07-13T22:13:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.640+0000 7f1b3cff9700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 0x7f1b280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.640+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b30014070 con 0x7f1b400ffb00 2023-07-13T22:13:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.640+0000 7f1b3effd700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 0x7f1b280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.641+0000 7f1b3effd700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 0x7f1b280a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b2e0 tx=0x7f1b34005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.644+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1b300962e0 con 0x7f1b400ffb00 2023-07-13T22:13:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.856+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b4002c850 con 0x7f1b400ffb00 2023-07-13T22:13:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.857+0000 7f1b3cff9700 1 -- 172.21.15.26:0/92703208 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f1b30099900 con 0x7f1b400ffb00 2023-07-13T22:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:49.848 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.859+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 msgr2=0x7f1b280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.859+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 0x7f1b280a36d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400b2e0 tx=0x7f1b34005fd0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 msgr2=0x7f1b40195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000e970 tx=0x7f1b3000ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 shutdown_connections 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1b280a1220 0x7f1b280a36d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b400ffb00 0x7f1b40195890 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b40101680 0x7f1b4019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 --2- 172.21.15.26:0/92703208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b40100cf0 0x7f1b40195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 >> 172.21.15.26:0/92703208 conn(0x7f1b400fb0e0 msgr2=0x7f1b40104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 shutdown_connections 2023-07-13T22:13:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:49.860+0000 7f1b45fff700 1 -- 172.21.15.26:0/92703208 wait complete. 2023-07-13T22:13:49.869 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:13:49.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:13:49.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/873246356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:49.930 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1250728958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:49.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/873246356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:49.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1250728958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:49.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/873246356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:49.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1250728958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 -- 172.21.15.26:0/1397575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 msgr2=0x7f0850101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 --2- 172.21.15.26:0/1397575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850101130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0844009990 tx=0x7f0844009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 -- 172.21.15.26:0/1397575020 shutdown_connections 2023-07-13T22:13:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 --2- 172.21.15.26:0/1397575020 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f0850107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 --2- 172.21.15.26:0/1397575020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 0x7f08500fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 --2- 172.21.15.26:0/1397575020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850101130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 -- 172.21.15.26:0/1397575020 >> 172.21.15.26:0/1397575020 conn(0x7f08500fb130 msgr2=0x7f08500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 -- 172.21.15.26:0/1397575020 shutdown_connections 2023-07-13T22:13:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.086+0000 7f08553ff700 1 -- 172.21.15.26:0/1397575020 wait complete. 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.087+0000 7f08553ff700 1 Processor -- start 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.087+0000 7f08553ff700 1 -- start start 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 0x7f0850197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f085019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085019f620 con 0x7f0850100ce0 2023-07-13T22:13:50.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085019f760 con 0x7f08500ffaf0 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f08553ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085019f8f0 con 0x7f0850101670 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46830/0 (socket says 172.21.15.26:46830) 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 -- 172.21.15.26:0/3763006586 learned_addr learned my addr 172.21.15.26:0/3763006586 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084effd700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 0x7f0850197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084f7fe700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f085019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 -- 172.21.15.26:0/3763006586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 msgr2=0x7f085019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f085019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 -- 172.21.15.26:0/3763006586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 msgr2=0x7f0850197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 0x7f0850197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084e7fc700 1 -- 172.21.15.26:0/3763006586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440095f0 con 0x7f0850100ce0 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.088+0000 7f084f7fe700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f085019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f084e7fc700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0844004760 tx=0x7f0844004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084401d070 con 0x7f0850100ce0 2023-07-13T22:13:50.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0844004cf0 con 0x7f0850100ce0 2023-07-13T22:13:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085019fb70 con 0x7f0850100ce0 2023-07-13T22:13:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0844005280 con 0x7f0850100ce0 2023-07-13T22:13:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.089+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085019ffc0 con 0x7f0850100ce0 2023-07-13T22:13:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.090+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0850192180 con 0x7f0850100ce0 2023-07-13T22:13:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.092+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f08440053e0 con 0x7f0850100ce0 2023-07-13T22:13:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.093+0000 7f0837fff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 0x7f08380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.093+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08440cdf20 con 0x7f0850100ce0 2023-07-13T22:13:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.093+0000 7f084effd700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 0x7f08380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.094+0000 7f084effd700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 0x7f08380a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0850100b40 tx=0x7f0840007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.097+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0844097670 con 0x7f0850100ce0 2023-07-13T22:13:50.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.306+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f085002caa0 con 0x7f0850100ce0 2023-07-13T22:13:50.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.306+0000 7f0837fff700 1 -- 172.21.15.26:0/3763006586 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f084409ac90 con 0x7f0850100ce0 2023-07-13T22:13:50.297 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:13:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 msgr2=0x7f08380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 0x7f08380a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0850100b40 tx=0x7f0840007400 comp rx=0 tx=0).stop 2023-07-13T22:13:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 msgr2=0x7f0850197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0844004760 tx=0x7f0844004790 comp rx=0 tx=0).stop 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 shutdown_connections 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0850101670 0x7f085019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08500ffaf0 0x7f0850197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08380a11e0 0x7f08380a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 --2- 172.21.15.26:0/3763006586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0850100ce0 0x7f0850197fe0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.308+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 >> 172.21.15.26:0/3763006586 conn(0x7f08500fb130 msgr2=0x7f08501065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.309+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 shutdown_connections 2023-07-13T22:13:50.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.309+0000 7f08553ff700 1 -- 172.21.15.26:0/3763006586 wait complete. 2023-07-13T22:13:50.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:13:50.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 -- 172.21.15.26:0/709665224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad40ffaa0 msgr2=0x7f1ad40ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/709665224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40ffeb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009990 tx=0x7f1ad0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 -- 172.21.15.26:0/709665224 shutdown_connections 2023-07-13T22:13:50.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/709665224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40ffeb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/709665224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad41016e0 0x7f1ad4107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/709665224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 0x7f1ad41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 -- 172.21.15.26:0/709665224 >> 172.21.15.26:0/709665224 conn(0x7f1ad40fb0c0 msgr2=0x7f1ad40fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:50.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 -- 172.21.15.26:0/709665224 shutdown_connections 2023-07-13T22:13:50.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.533+0000 7f1adb9ff700 1 -- 172.21.15.26:0/709665224 wait complete. 2023-07-13T22:13:50.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 Processor -- start 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 -- start start 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 0x7f1ad4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40735e0 con 0x7f1ad41016e0 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4073750 con 0x7f1ad4100d30 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.534+0000 7f1adb9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad419f870 con 0x7f1ad40ffaa0 2023-07-13T22:13:50.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46850/0 (socket says 172.21.15.26:46850) 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 -- 172.21.15.26:0/973033037 learned_addr learned my addr 172.21.15.26:0/973033037 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad8f9a700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 0x7f1ad4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad979b700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 -- 172.21.15.26:0/973033037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad40ffaa0 msgr2=0x7f1ad40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 -- 172.21.15.26:0/973033037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 msgr2=0x7f1ad4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 0x7f1ad4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 -- 172.21.15.26:0/973033037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4009520 con 0x7f1ad41016e0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1ad9f9c700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1acc007ee0 tx=0x7f1acc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.535+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc017070 con 0x7f1ad41016e0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.536+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00095f0 con 0x7f1ad41016e0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.536+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1acc00f040 con 0x7f1ad41016e0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.536+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc005550 con 0x7f1ad41016e0 2023-07-13T22:13:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.536+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad419fed0 con 0x7f1ad41016e0 2023-07-13T22:13:50.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.537+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad410b2c0 con 0x7f1ad41016e0 2023-07-13T22:13:50.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.539+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1acc00db20 con 0x7f1ad41016e0 2023-07-13T22:13:50.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.540+0000 7f1aca7fc700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 0x7f1ac00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.540+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1acc0cc820 con 0x7f1ad41016e0 2023-07-13T22:13:50.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.541+0000 7f1ad979b700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 0x7f1ac00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.541+0000 7f1ad979b700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 0x7f1ac00a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b2e0 tx=0x7f1ad0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.542+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1acc095dc0 con 0x7f1ad41016e0 2023-07-13T22:13:50.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.752+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ad4106ab0 con 0x7f1ad41016e0 2023-07-13T22:13:50.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.752+0000 7f1aca7fc700 1 -- 172.21.15.26:0/973033037 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1acc0993e0 con 0x7f1ad41016e0 2023-07-13T22:13:50.743 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:13:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 msgr2=0x7f1ac00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 0x7f1ac00a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000b2e0 tx=0x7f1ad0005fb0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 msgr2=0x7f1ad4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1acc007ee0 tx=0x7f1acc00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 shutdown_connections 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ac00a11e0 0x7f1ac00a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad41016e0 0x7f1ad4072e40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ad40ffaa0 0x7f1ad40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 --2- 172.21.15.26:0/973033037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ad4100d30 0x7f1ad4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 >> 172.21.15.26:0/973033037 conn(0x7f1ad40fb0c0 msgr2=0x7f1ad40fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 shutdown_connections 2023-07-13T22:13:50.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.755+0000 7f1adb9ff700 1 -- 172.21.15.26:0/973033037 wait complete. 2023-07-13T22:13:50.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:13:50.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:50 smithi117 ceph-mon[129074]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:13:50.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:50 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/92703208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:50.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:50 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3763006586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:50 smithi026 ceph-mon[95647]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:13:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:50 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/92703208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:50.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:50 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3763006586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/2311873796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34100ce0 msgr2=0x7f0b34101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/2311873796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34100ce0 0x7f0b34101130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009990 tx=0x7f0b24009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/2311873796 shutdown_connections 2023-07-13T22:13:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/2311873796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34101670 0x7f0b34107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/2311873796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 0x7f0b340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/2311873796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34100ce0 0x7f0b34101130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/2311873796 >> 172.21.15.26:0/2311873796 conn(0x7f0b340fb130 msgr2=0x7f0b340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:50.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.980+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/2311873796 shutdown_connections 2023-07-13T22:13:50.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.981+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/2311873796 wait complete. 2023-07-13T22:13:50.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 Processor -- start 2023-07-13T22:13:50.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 -- start start 2023-07-13T22:13:50.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 0x7f0b34197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34100ce0 0x7f0b34197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419f5e0 con 0x7f0b34101670 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419f720 con 0x7f0b340ffaf0 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3b7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419f8b0 con 0x7f0b34100ce0 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b39d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b39d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46864/0 (socket says 172.21.15.26:46864) 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b39d9c700 1 -- 172.21.15.26:0/4272684751 learned_addr learned my addr 172.21.15.26:0/4272684751 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:50.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b38d9a700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34100ce0 0x7f0b34197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.982+0000 7f0b3959b700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 0x7f0b34197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 -- 172.21.15.26:0/4272684751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34100ce0 msgr2=0x7f0b34197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34100ce0 0x7f0b34197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 -- 172.21.15.26:0/4272684751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 msgr2=0x7f0b34197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 0x7f0b34197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 -- 172.21.15.26:0/4272684751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240095f0 con 0x7f0b34101670 2023-07-13T22:13:50.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b39d9c700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00ef30 tx=0x7f0b2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c018070 con 0x7f0b34101670 2023-07-13T22:13:50.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3419fb30 con 0x7f0b34101670 2023-07-13T22:13:50.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b2c00f040 con 0x7f0b34101670 2023-07-13T22:13:50.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00cd00 con 0x7f0b34101670 2023-07-13T22:13:50.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.983+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b341a0090 con 0x7f0b34101670 2023-07-13T22:13:50.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.984+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3410b310 con 0x7f0b34101670 2023-07-13T22:13:50.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.986+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0b2c003a00 con 0x7f0b34101670 2023-07-13T22:13:50.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.989+0000 7f0b2a7fc700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 0x7f0b200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:50.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.989+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0b2c0cc5d0 con 0x7f0b34101670 2023-07-13T22:13:50.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.989+0000 7f0b3959b700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 0x7f0b200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:50.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.990+0000 7f0b3959b700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 0x7f0b200a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b34100b40 tx=0x7f0b30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:50.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:50.993+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0b2c095c00 con 0x7f0b34101670 2023-07-13T22:13:50.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:50 smithi121 ceph-mon[129187]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:13:50.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:50 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/92703208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:50.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:50 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3763006586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.200+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b341a0a20 con 0x7f0b34101670 2023-07-13T22:13:51.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.200+0000 7f0b2a7fc700 1 -- 172.21.15.26:0/4272684751 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b2c0b7e60 con 0x7f0b34101670 2023-07-13T22:13:51.191 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:13:51.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 msgr2=0x7f0b200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 0x7f0b200a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b34100b40 tx=0x7f0b30009260 comp rx=0 tx=0).stop 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 msgr2=0x7f0b3419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00ef30 tx=0x7f0b2c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 shutdown_connections 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0b200a11e0 0x7f0b200a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b34101670 0x7f0b3419f010 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b340ffaf0 0x7f0b34197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 --2- 172.21.15.26:0/4272684751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b34100ce0 0x7f0b34197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 >> 172.21.15.26:0/4272684751 conn(0x7f0b340fb130 msgr2=0x7f0b340fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 shutdown_connections 2023-07-13T22:13:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.203+0000 7f0b3b7ff700 1 -- 172.21.15.26:0/4272684751 wait complete. 2023-07-13T22:13:51.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:13:51.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.442+0000 7fd951dff700 1 -- 172.21.15.26:0/1120964560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c10bd60 msgr2=0x7fd94c10e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.442+0000 7fd951dff700 1 --2- 172.21.15.26:0/1120964560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c10bd60 0x7fd94c10e1e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd94400b0a0 tx=0x7fd94400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.443+0000 7fd951dff700 1 -- 172.21.15.26:0/1120964560 shutdown_connections 2023-07-13T22:13:51.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.443+0000 7fd951dff700 1 --2- 172.21.15.26:0/1120964560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c10bd60 0x7fd94c10e1e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.443+0000 7fd951dff700 1 --2- 172.21.15.26:0/1120964560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.443+0000 7fd951dff700 1 --2- 172.21.15.26:0/1120964560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c071820 0x7fd94c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.443+0000 7fd951dff700 1 -- 172.21.15.26:0/1120964560 >> 172.21.15.26:0/1120964560 conn(0x7fd94c06ca40 msgr2=0x7fd94c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:51.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.445+0000 7fd951dff700 1 -- 172.21.15.26:0/1120964560 shutdown_connections 2023-07-13T22:13:51.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.446+0000 7fd951dff700 1 -- 172.21.15.26:0/1120964560 wait complete. 2023-07-13T22:13:51.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.448+0000 7fd951dff700 1 Processor -- start 2023-07-13T22:13:51.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.448+0000 7fd951dff700 1 -- start start 2023-07-13T22:13:51.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c1139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 0x7fd94c113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c10bd60 0x7fd94c1b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c116ff0 con 0x7fd94c071820 2023-07-13T22:13:51.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c117160 con 0x7fd94c10bd60 2023-07-13T22:13:51.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd951dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c1b4e10 con 0x7fd94c070ed0 2023-07-13T22:13:51.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd94affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 0x7fd94c113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd94affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 0x7fd94c113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46884/0 (socket says 172.21.15.26:46884) 2023-07-13T22:13:51.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.449+0000 7fd94affd700 1 -- 172.21.15.26:0/2135418490 learned_addr learned my addr 172.21.15.26:0/2135418490 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:51.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c1139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94bfff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c10bd60 0x7fd94c1b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 -- 172.21.15.26:0/2135418490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c10bd60 msgr2=0x7fd94c1b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c10bd60 0x7fd94c1b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 -- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 msgr2=0x7fd94c113f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 0x7fd94c113f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.451+0000 7fd94b7fe700 1 -- 172.21.15.26:0/2135418490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944009d00 con 0x7fd94c070ed0 2023-07-13T22:13:51.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.453+0000 7fd94b7fe700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c1139d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd94000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:51.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.454+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940011830 con 0x7fd94c070ed0 2023-07-13T22:13:51.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.454+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd940011e50 con 0x7fd94c070ed0 2023-07-13T22:13:51.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.454+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940010d20 con 0x7fd94c070ed0 2023-07-13T22:13:51.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.454+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c1b4f50 con 0x7fd94c070ed0 2023-07-13T22:13:51.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.455+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c1b53a0 con 0x7fd94c070ed0 2023-07-13T22:13:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.457+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c11a180 con 0x7fd94c070ed0 2023-07-13T22:13:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.462+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd940010380 con 0x7fd94c070ed0 2023-07-13T22:13:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.463+0000 7fd948ff9700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 0x7fd9340a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.464+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9400cc120 con 0x7fd94c070ed0 2023-07-13T22:13:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.464+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd9400992e0 con 0x7fd94c070ed0 2023-07-13T22:13:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.465+0000 7fd94affd700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 0x7fd9340a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.465+0000 7fd94affd700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 0x7fd9340a38f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd70 tx=0x7fd93c00b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:51.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.706+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd940095c20 con 0x7fd94c070ed0 2023-07-13T22:13:51.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.725+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd94c1b5a70 con 0x7fd94c070ed0 2023-07-13T22:13:51.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.726+0000 7fd948ff9700 1 -- 172.21.15.26:0/2135418490 <== mon.1 v2:172.21.15.121:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fd94006f2a0 con 0x7fd94c070ed0 2023-07-13T22:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:51.717 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:13:51.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 msgr2=0x7fd9340a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 0x7fd9340a38f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00bd70 tx=0x7fd93c00b4a0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 msgr2=0x7fd94c1139d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c1139d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd94000b730 comp rx=0 tx=0).stop 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 shutdown_connections 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd94c10bd60 0x7fd94c1b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd94c070ed0 0x7fd94c1139d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9340a1440 0x7fd9340a38f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 --2- 172.21.15.26:0/2135418490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd94c071820 0x7fd94c113f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 >> 172.21.15.26:0/2135418490 conn(0x7fd94c06ca40 msgr2=0x7fd94c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 shutdown_connections 2023-07-13T22:13:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.729+0000 7fd951dff700 1 -- 172.21.15.26:0/2135418490 wait complete. 2023-07-13T22:13:51.721 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:13:51.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:13:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 -- 172.21.15.26:0/1611803516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 msgr2=0x7f48b4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 --2- 172.21.15.26:0/1611803516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4101140 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009990 tx=0x7f48a0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 -- 172.21.15.26:0/1611803516 shutdown_connections 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 --2- 172.21.15.26:0/1611803516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 0x7f48b4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 --2- 172.21.15.26:0/1611803516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 0x7f48b40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 --2- 172.21.15.26:0/1611803516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4101140 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 -- 172.21.15.26:0/1611803516 >> 172.21.15.26:0/1611803516 conn(0x7f48b40fb0e0 msgr2=0x7f48b40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.953+0000 7f48bafff700 1 -- 172.21.15.26:0/1611803516 shutdown_connections 2023-07-13T22:13:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.954+0000 7f48bafff700 1 -- 172.21.15.26:0/1611803516 wait complete. 2023-07-13T22:13:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:51 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/973033037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:51 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4272684751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.955+0000 7f48bafff700 1 Processor -- start 2023-07-13T22:13:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.955+0000 7f48bafff700 1 -- start start 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 0x7f48b4195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 0x7f48b419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419d3b0 con 0x7f48b4100cf0 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419d4f0 con 0x7f48b40ffb00 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48bafff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b419d680 con 0x7f48b4101680 2023-07-13T22:13:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46906/0 (socket says 172.21.15.26:46906) 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 -- 172.21.15.26:0/3244969661 learned_addr learned my addr 172.21.15.26:0/3244969661 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b959c700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 0x7f48b419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b8d9b700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 0x7f48b4195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 -- 172.21.15.26:0/3244969661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 msgr2=0x7f48b419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 0x7f48b419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 -- 172.21.15.26:0/3244969661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 msgr2=0x7f48b4195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 0x7f48b4195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.956+0000 7f48b3fff700 1 -- 172.21.15.26:0/3244969661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00095f0 con 0x7f48b4100cf0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48b3fff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48a0004760 tx=0x7f48a0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a001d070 con 0x7f48b4100cf0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b419d900 con 0x7f48b4100cf0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48a0004830 con 0x7f48b4100cf0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0005350 con 0x7f48b4100cf0 2023-07-13T22:13:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.957+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b419dcd0 con 0x7f48b4100cf0 2023-07-13T22:13:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.958+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b410b320 con 0x7f48b4100cf0 2023-07-13T22:13:51.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:51 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/973033037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:51 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4272684751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.961+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f48a00054d0 con 0x7f48b4100cf0 2023-07-13T22:13:51.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.963+0000 7f48b1ffb700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 0x7f48a40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:51.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.963+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f48a00cdfe0 con 0x7f48b4100cf0 2023-07-13T22:13:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.963+0000 7f48b8d9b700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 0x7f48a40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.964+0000 7f48b8d9b700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 0x7f48a40a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48b4100b50 tx=0x7f48a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:51.965+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f48a0097790 con 0x7f48b4100cf0 2023-07-13T22:13:51.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:51 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/973033037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:13:51.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:51 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4272684751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:13:52.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.181+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48b410b4e0 con 0x7f48b4100cf0 2023-07-13T22:13:52.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.182+0000 7f48b1ffb700 1 -- 172.21.15.26:0/3244969661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f48a009adb0 con 0x7f48b4100cf0 2023-07-13T22:13:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:13:52.173 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:13:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.184+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 msgr2=0x7f48a40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.184+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 0x7f48a40a3650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f48b4100b50 tx=0x7f48a8009260 comp rx=0 tx=0).stop 2023-07-13T22:13:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.184+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 msgr2=0x7f48b4195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.184+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48a0004760 tx=0x7f48a0004790 comp rx=0 tx=0).stop 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 shutdown_connections 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48b4101680 0x7f48b419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f48b40ffb00 0x7f48b4195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f48a40a11a0 0x7f48a40a3650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 --2- 172.21.15.26:0/3244969661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b4100cf0 0x7f48b4195d70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 >> 172.21.15.26:0/3244969661 conn(0x7f48b40fb0e0 msgr2=0x7f48b40fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 shutdown_connections 2023-07-13T22:13:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.185+0000 7f48bafff700 1 -- 172.21.15.26:0/3244969661 wait complete. 2023-07-13T22:13:52.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-13T22:12:18.516293+0000', 'modified': '2023-07-13T22:12:26.449971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24575, 'mds_1': 24563, 'mds_2': 24530, 'mds_3': 24493, 'mds_4': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14520': {'gid': 14520, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.26:6839/1019342380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6838', 'nonce': 1019342380}, {'type': 'v1', 'addr': '172.21.15.26:6839', 'nonce': 1019342380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6833/930793918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 930793918}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 930793918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24467': {'gid': 24467, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.117:6833/2196716047', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 2196716047}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 2196716047}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.121:6835/59384395', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 59384395}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 59384395}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/1505729647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1505729647}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1505729647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/1862924316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 1862924316}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 1862924316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6839/3259819874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 3259819874}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 3259819874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.117:6837/1128080760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 1128080760}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 1128080760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.26:6841/2542800089', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6840', 'nonce': 2542800089}, {'type': 'v1', 'addr': '172.21.15.26:6841', 'nonce': 2542800089}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24575': {'gid': 24575, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.117:6839/1698058160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 1698058160}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 1698058160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-13T22:13:52.195 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-07-13T22:13:52.195 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2023-07-13T22:13:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 -- 172.21.15.26:0/1514566133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1280072f70 msgr2=0x7f12800733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 --2- 172.21.15.26:0/1514566133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1280072f70 0x7f12800733e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1268009990 tx=0x7f1268009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 -- 172.21.15.26:0/1514566133 shutdown_connections 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 --2- 172.21.15.26:0/1514566133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12801079a0 0x7f1280109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 --2- 172.21.15.26:0/1514566133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280074550 0x7f12800729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 --2- 172.21.15.26:0/1514566133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1280072f70 0x7f12800733e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 -- 172.21.15.26:0/1514566133 >> 172.21.15.26:0/1514566133 conn(0x7f12800fb0e0 msgr2=0x7f12800fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 -- 172.21.15.26:0/1514566133 shutdown_connections 2023-07-13T22:13:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.412+0000 7f12851ff700 1 -- 172.21.15.26:0/1514566133 wait complete. 2023-07-13T22:13:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 Processor -- start 2023-07-13T22:13:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 -- start start 2023-07-13T22:13:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1280074550 0x7f1280198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12801079a0 0x7f128019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019f640 con 0x7f12801079a0 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019f780 con 0x7f1280072f70 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.413+0000 7f12851ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019f910 con 0x7f1280074550 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48356/0 (socket says 172.21.15.26:48356) 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 -- 172.21.15.26:0/714416353 learned_addr learned my addr 172.21.15.26:0/714416353 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127f59e700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12801079a0 0x7f128019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f1277fff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1280074550 0x7f1280198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 -- 172.21.15.26:0/714416353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1280074550 msgr2=0x7f1280198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1280074550 0x7f1280198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 -- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12801079a0 msgr2=0x7f128019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12801079a0 0x7f128019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 -- 172.21.15.26:0/714416353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680095f0 con 0x7f1280072f70 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127ed9d700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f127000e970 tx=0x7f127000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.414+0000 7f127cd99700 1 -- 172.21.15.26:0/714416353 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270009790 con 0x7f1280072f70 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.415+0000 7f127cd99700 1 -- 172.21.15.26:0/714416353 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127000c7e0 con 0x7f1280072f70 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.415+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128019fbf0 con 0x7f1280072f70 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.415+0000 7f127cd99700 1 -- 172.21.15.26:0/714416353 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700054e0 con 0x7f1280072f70 2023-07-13T22:13:52.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.415+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a0150 con 0x7f1280072f70 2023-07-13T22:13:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.419+0000 7f127cd99700 1 -- 172.21.15.26:0/714416353 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f127000c950 con 0x7f1280072f70 2023-07-13T22:13:52.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.420+0000 7f1298be0b80 1 Processor -- start 2023-07-13T22:13:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.420+0000 7f1298be0b80 1 -- start start 2023-07-13T22:13:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 0x560d6765c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d6765e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d67561c60 con 0x560d6765cdd0 2023-07-13T22:13:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d675b49f0 con 0x560d6765dc20 2023-07-13T22:13:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1298be0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d675ade80 con 0x560d6765c480 2023-07-13T22:13:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127cd99700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 0x7f126c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 0x560d6765c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46952/0 (socket says 172.21.15.26:46952) 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 0x560d6765c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36606/0 (socket says 172.21.15.26:36606) 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127ed9d700 1 -- 172.21.15.26:0/2721229616 learned_addr learned my addr 172.21.15.26:0/2721229616 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f1277fff700 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d6765e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.421+0000 7f127cd99700 1 -- 172.21.15.26:0/714416353 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1270014070 con 0x7f1280072f70 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127f59e700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 0x7f126c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 -- 172.21.15.26:0/2721229616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 msgr2=0x560d6765c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 0x560d6765c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 -- 172.21.15.26:0/2721229616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 msgr2=0x560d6765e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d6765e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 -- 172.21.15.26:0/2721229616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d673dbd90 con 0x560d6765cdd0 2023-07-13T22:13:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f1277fff700 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d6765e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127ed9d700 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12700962c0 tx=0x7f12700962f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c651fe674c6b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f12757fa700 1 -- 172.21.15.26:0/2721229616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270099860 con 0x560d6765cdd0 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f127f59e700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 0x7f126c0a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1278003fd0 tx=0x7f1278004b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f12757fa700 1 -- 172.21.15.26:0/2721229616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127000cc50 con 0x560d6765cdd0 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.422+0000 7f12757fa700 1 -- 172.21.15.26:0/2721229616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700ca700 con 0x560d6765cdd0 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- 172.21.15.26:0/2721229616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 msgr2=0x560d6765d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12700962c0 tx=0x7f12700962f0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- 172.21.15.26:0/2721229616 shutdown_connections 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d6765e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765c480 0x560d6765c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 --2- 172.21.15.26:0/2721229616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765cdd0 0x560d6765d690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- 172.21.15.26:0/2721229616 >> 172.21.15.26:0/2721229616 conn(0x560d67651d50 msgr2=0x560d67657180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- 172.21.15.26:0/2721229616 shutdown_connections 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- 172.21.15.26:0/2721229616 wait complete. 2023-07-13T22:13:52.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.423+0000 7f1298be0b80 1 -- start start 2023-07-13T22:13:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 Processor -- start 2023-07-13T22:13:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67666600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6754da70 con 0x560d6765c480 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6754d770 con 0x560d6765cdd0 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d674318d0 con 0x560d6765dc20 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f1298be0b80 10 client.0 _ll_register_callbacks cb 0x560d6754b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.424+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46956/0 (socket says 172.21.15.26:46956) 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 -- 172.21.15.26:0/3215922440 learned_addr learned my addr 172.21.15.26:0/3215922440 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f1277fff700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67666600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127f59e700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 -- 172.21.15.26:0/3215922440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 msgr2=0x560d6766d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 -- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 msgr2=0x560d67666600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67666600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 -- 172.21.15.26:0/3215922440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128010aa40 con 0x560d6765c480 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f127ed9d700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1270095c80 tx=0x7f127000c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652bbdb2a0b8dd1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f12667fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700995f0 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f127009df30 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d675074a0 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.425+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127000b2c0 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 10 client.15144 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f12801a0150 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d675074a0 con 0x560d6765c480 2023-07-13T22:13:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1270004a30 con 0x560d6765c480 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 10 client.15144 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 20 client.15144 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d67670f20 0x560d67670570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 4 client.15144 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d675a6e60 con 0x560d67670f20 2023-07-13T22:13:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.426+0000 7f1277fff700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d67670f20 0x560d67670570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.427+0000 7f12667fc700 10 client.15144 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.427+0000 7f1277fff700 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d67670f20 0x560d67670570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42d55cb1801ceaf6 server_cookie=5140de762e980a0d in_seq=0 out_seq=0 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f12667fc700 1 -- 172.21.15.26:0/3215922440 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560d675a6e60 con 0x560d67670f20 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f12667fc700 10 client.15144 handle_command_reply: tid=0 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 client.15144 shutdown 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 msgr2=0x560d676660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1270095c80 tx=0x7f127000c440 comp rx=0 tx=0).stop 2023-07-13T22:13:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 shutdown_connections 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d67670f20 0x560d67670570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d676660c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f12667fc700 0 client.15144 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 --2- 172.21.15.26:0/3215922440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67666600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 >> 172.21.15.26:0/3215922440 conn(0x560d67651d50 msgr2=0x560d67656860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 shutdown_connections 2023-07-13T22:13:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 1 -- 172.21.15.26:0/3215922440 wait complete. 2023-07-13T22:13:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.428+0000 7f1298be0b80 20 client.15144 trim_cache size 0 max 16384 2023-07-13T22:13:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 Processor -- start 2023-07-13T22:13:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 -- start start 2023-07-13T22:13:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 0x560d6766d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d67670570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6743de20 con 0x560d6765c480 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6756c4b0 con 0x560d6765dc20 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f1298be0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d676721b0 con 0x560d6765cdd0 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46960/0 (socket says 172.21.15.26:46960) 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.447+0000 7f127ed9d700 1 -- 172.21.15.26:0/4025209412 learned_addr learned my addr 172.21.15.26:0/4025209412 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127f59e700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d67670570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f1277fff700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 0x560d6766d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 -- 172.21.15.26:0/4025209412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 msgr2=0x560d6766d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 0x560d6766d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 -- 172.21.15.26:0/4025209412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 msgr2=0x560d67670570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d67670570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 -- 172.21.15.26:0/4025209412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d673c4c10 con 0x560d6765c480 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f1277fff700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 0x560d6766d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127f59e700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d67670570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f127000cde0 tx=0x7f127000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a902300cfaedae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f1275ffb700 1 -- 172.21.15.26:0/4025209412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270099f00 con 0x560d6765c480 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f1275ffb700 1 -- 172.21.15.26:0/4025209412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1270099860 con 0x560d6765c480 2023-07-13T22:13:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.448+0000 7f1275ffb700 1 -- 172.21.15.26:0/4025209412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700a2100 con 0x560d6765c480 2023-07-13T22:13:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- 172.21.15.26:0/4025209412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 msgr2=0x560d67679c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f127000cde0 tx=0x7f127000cf50 comp rx=0 tx=0).stop 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- 172.21.15.26:0/4025209412 shutdown_connections 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67679c50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765dc20 0x560d67670570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4025209412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765cdd0 0x560d6766d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- 172.21.15.26:0/4025209412 >> 172.21.15.26:0/4025209412 conn(0x560d67651d50 msgr2=0x560d676532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- 172.21.15.26:0/4025209412 shutdown_connections 2023-07-13T22:13:52.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- 172.21.15.26:0/4025209412 wait complete. 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.449+0000 7f1298be0b80 1 -- start start 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 Processor -- start 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6766c200 con 0x560d6765c480 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6766c370 con 0x560d6765cdd0 2023-07-13T22:13:52.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d6766c530 con 0x560d6765dc20 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1298be0b80 10 client.0 _ll_register_callbacks cb 0x7f1270005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1277fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1277fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48396/0 (socket says 172.21.15.26:48396) 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f1277fff700 1 -- 172.21.15.26:0/4106904155 learned_addr learned my addr 172.21.15.26:0/4106904155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f127ed9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46970/0 (socket says 172.21.15.26:46970) 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.450+0000 7f127f59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 -- 172.21.15.26:0/4106904155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 msgr2=0x560d6766bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 -- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 msgr2=0x560d67668db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 -- 172.21.15.26:0/4106904155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a0150 con 0x560d6765c480 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f1277fff700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f127ed9d700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f127000efd0 tx=0x7f12700ca670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc2b9682b2b766e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f12667fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:52.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270004bf0 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127000cc50 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f127000b2c0 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127009df30 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f1298be0b80 10 client.15150 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.451+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560d67402200 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127000cc50 con 0x560d6765c480 2023-07-13T22:13:52.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1270098e60 con 0x560d6765c480 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 10 client.15150 resolve_mds: resolved name 'k' to daemon mds.k 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 20 client.15150 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d676c9df0 0x560d676c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 4 client.15150 mds_command: new command op to 24467 tid=0[{"prefix": "status"}] 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560d67569000 con 0x560d676c9df0 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.452+0000 7f1277fff700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d676c9df0 0x560d676c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1277fff700 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d676c9df0 0x560d676c6d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=47887bef84e4073 server_cookie=b64cd6de042c7e5b in_seq=0 out_seq=0 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f12667fc700 10 client.15150 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f12667fc700 1 -- 172.21.15.26:0/4106904155 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560d67569000 con 0x560d676c9df0 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f12667fc700 10 client.15150 handle_command_reply: tid=0 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 client.15150 shutdown 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 msgr2=0x560d67668870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f127000efd0 tx=0x7f12700ca670 comp rx=0 tx=0).stop 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 shutdown_connections 2023-07-13T22:13:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560d676c9df0 0x560d676c6d90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560d6765c480 0x560d67668870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560d6765dc20 0x560d6766bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 --2- 172.21.15.26:0/4106904155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560d6765cdd0 0x560d67668db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 >> 172.21.15.26:0/4106904155 conn(0x560d67651d50 msgr2=0x560d67679c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f12667fc700 0 client.15150 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 shutdown_connections 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.453+0000 7f1298be0b80 1 -- 172.21.15.26:0/4106904155 wait complete. 2023-07-13T22:13:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f1298be0b80 20 client.15150 trim_cache size 0 max 16384 2023-07-13T22:13:52.453 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 4, 2023-07-13T22:13:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24467, 2023-07-13T22:13:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 87.013636923999996, 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 126.320092903 2023-07-13T22:13:52.454 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 msgr2=0x7f126c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 0x7f126c0a3680 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1278003fd0 tx=0x7f1278004b40 comp rx=0 tx=0).stop 2023-07-13T22:13:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 msgr2=0x7f1280197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f127000e970 tx=0x7f127000ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 shutdown_connections 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a11d0 0x7f126c0a3680 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12801079a0 0x7f128019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1280072f70 0x7f1280197ac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 --2- 172.21.15.26:0/714416353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1280074550 0x7f1280198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 >> 172.21.15.26:0/714416353 conn(0x7f12800fb0e0 msgr2=0x7f12800fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 shutdown_connections 2023-07-13T22:13:52.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.454+0000 7f12851ff700 1 -- 172.21.15.26:0/714416353 wait complete. 2023-07-13T22:13:52.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-07-13T22:13:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 -- 172.21.15.26:0/2538143840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f4101660 msgr2=0x7fa1f4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/2538143840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f4101660 0x7fa1f4107f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00b0a0 tx=0x7fa1ec00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 -- 172.21.15.26:0/2538143840 shutdown_connections 2023-07-13T22:13:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/2538143840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f4101660 0x7fa1f4107f20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/2538143840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/2538143840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4100cd0 0x7fa1f4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 -- 172.21.15.26:0/2538143840 >> 172.21.15.26:0/2538143840 conn(0x7fa1f40fb0e0 msgr2=0x7fa1f40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 -- 172.21.15.26:0/2538143840 shutdown_connections 2023-07-13T22:13:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.677+0000 7fa1fbbff700 1 -- 172.21.15.26:0/2538143840 wait complete. 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.678+0000 7fa1fbbff700 1 Processor -- start 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.678+0000 7fa1fbbff700 1 -- start start 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f4100cd0 0x7fa1f4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4101660 0x7fa1f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40735e0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f4073750 con 0x7fa1f4101660 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fbbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f419f9c0 con 0x7fa1f4100cd0 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46984/0 (socket says 172.21.15.26:46984) 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 -- 172.21.15.26:0/906376308 learned_addr learned my addr 172.21.15.26:0/906376308 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f919a700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f4100cd0 0x7fa1f4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4101660 0x7fa1f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 -- 172.21.15.26:0/906376308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f4100cd0 msgr2=0x7fa1f4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f4100cd0 0x7fa1f4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 -- 172.21.15.26:0/906376308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4101660 msgr2=0x7fa1f4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4101660 0x7fa1f4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.679+0000 7fa1f999b700 1 -- 172.21.15.26:0/906376308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009d00 con 0x7fa1f40ffae0 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1f999b700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e970 tx=0x7fa1f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1eaffd700 1 -- 172.21.15.26:0/906376308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0009790 con 0x7fa1f40ffae0 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1eaffd700 1 -- 172.21.15.26:0/906376308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f000eec0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1eaffd700 1 -- 172.21.15.26:0/906376308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0003cf0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f419fca0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.680+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f41a01b0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.683+0000 7fa1eaffd700 1 -- 172.21.15.26:0/906376308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa1f000ccd0 con 0x7fa1f40ffae0 2023-07-13T22:13:52.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 Processor -- start 2023-07-13T22:13:52.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1eaffd700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 0x7fa1e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 -- start start 2023-07-13T22:13:52.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1eaffd700 1 -- 172.21.15.26:0/906376308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa1f0014070 con 0x7fa1f40ffae0 2023-07-13T22:13:52.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6eb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 0x7fa1e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a5efc60 con 0x555a7a6ea480 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a6429f0 con 0x555a7a6eadd0 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa20f602b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a63be80 con 0x555a7a6ebc20 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46996/0 (socket says 172.21.15.26:46996) 2023-07-13T22:13:52.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.685+0000 7fa1f999b700 1 -- 172.21.15.26:0/1483670063 learned_addr learned my addr 172.21.15.26:0/1483670063 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f919a700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 -- 172.21.15.26:0/1483670063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 msgr2=0x555a7a6ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 0x7fa1e00a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4074920 tx=0x7fa1ec0094c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 -- 172.21.15.26:0/1483670063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 msgr2=0x555a7a6eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6eb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 -- 172.21.15.26:0/1483670063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7a469d90 con 0x555a7a6ea480 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f919a700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1f999b700 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0096940 tx=0x7fa1f0099830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9182c4667a4f99ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1dffff700 1 -- 172.21.15.26:0/1483670063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f009df00 con 0x555a7a6ea480 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1dffff700 1 -- 172.21.15.26:0/1483670063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f00a2320 con 0x555a7a6ea480 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.686+0000 7fa1dffff700 1 -- 172.21.15.26:0/1483670063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0099620 con 0x555a7a6ea480 2023-07-13T22:13:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 -- 172.21.15.26:0/1483670063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 msgr2=0x555a7a6ea890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0096940 tx=0x7fa1f0099830 comp rx=0 tx=0).stop 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 -- 172.21.15.26:0/1483670063 shutdown_connections 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6ea890 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 --2- 172.21.15.26:0/1483670063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 -- 172.21.15.26:0/1483670063 >> 172.21.15.26:0/1483670063 conn(0x555a7a6dfd50 msgr2=0x555a7a6e5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 -- 172.21.15.26:0/1483670063 shutdown_connections 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.687+0000 7fa20f602b80 1 -- 172.21.15.26:0/1483670063 wait complete. 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.688+0000 7fa20f602b80 1 -- start start 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.688+0000 7fa20f602b80 1 Processor -- start 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a5dba70 con 0x555a7a6ea480 2023-07-13T22:13:52.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a5db770 con 0x555a7a6eadd0 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a4bf8d0 con 0x555a7a6ebc20 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa20f602b80 10 client.0 _ll_register_callbacks cb 0x555a7a5d9710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47006/0 (socket says 172.21.15.26:47006) 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa1f919a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa1f999b700 1 -- 172.21.15.26:0/3163306992 learned_addr learned my addr 172.21.15.26:0/3163306992 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.689+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 -- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 msgr2=0x555a7a6fb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 -- 172.21.15.26:0/3163306992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 msgr2=0x555a7a6f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 -- 172.21.15.26:0/3163306992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f41a01b0 con 0x555a7a6ea480 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f919a700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1f999b700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0003700 tx=0x7fa1f009df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc41c97e1557dda server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00adbd0 con 0x555a7a6ea480 2023-07-13T22:13:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa1f00add30 con 0x555a7a6ea480 2023-07-13T22:13:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a7a5954a0 con 0x555a7a6ea480 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f00ca250 con 0x555a7a6ea480 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa20f602b80 10 client.15162 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a7a452c10 con 0x555a7a6ea480 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.690+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f007e020 con 0x555a7a6ea480 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1f0098e30 con 0x555a7a6ea480 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 10 client.15162 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 20 client.15162 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a6ff550 0x555a7a6fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 4 client.15162 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a7a634e60 con 0x555a7a6ff550 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.691+0000 7fa1f919a700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a6ff550 0x555a7a6fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.692+0000 7fa1f919a700 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a6ff550 0x555a7a6fe610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=57312433d9bc7311 server_cookie=ca91aad9988c0fa5 in_seq=0 out_seq=0 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.692+0000 7fa1dd7fa700 10 client.15162 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:52.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.692+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/3163306992 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a7a634e60 con 0x555a7a6ff550 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.692+0000 7fa1dd7fa700 10 client.15162 handle_command_reply: tid=0 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 client.15162 shutdown 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 msgr2=0x555a7a6f4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0003700 tx=0x7fa1f009df80 comp rx=0 tx=0).stop 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 shutdown_connections 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a6ff550 0x555a7a6fe610 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f4160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa1dd7fa700 0 client.15162 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 --2- 172.21.15.26:0/3163306992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 >> 172.21.15.26:0/3163306992 conn(0x555a7a6dfd50 msgr2=0x555a7a6e4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 shutdown_connections 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 1 -- 172.21.15.26:0/3163306992 wait complete. 2023-07-13T22:13:52.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.693+0000 7fa20f602b80 20 client.15162 trim_cache size 0 max 16384 2023-07-13T22:13:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 Processor -- start 2023-07-13T22:13:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 -- start start 2023-07-13T22:13:52.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6eadd0 0x555a7a6f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a6e6630 con 0x555a7a6eadd0 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a5e5470 con 0x555a7a6ea480 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.713+0000 7fa20f602b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a60a790 con 0x555a7a6ebc20 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f919a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6eadd0 0x555a7a6f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48468/0 (socket says 172.21.15.26:48468) 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 -- 172.21.15.26:0/721357319 learned_addr learned my addr 172.21.15.26:0/721357319 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 -- 172.21.15.26:0/721357319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 msgr2=0x555a7a6fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 -- 172.21.15.26:0/721357319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6eadd0 msgr2=0x555a7a6f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6eadd0 0x555a7a6f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 -- 172.21.15.26:0/721357319 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a7a490200 con 0x555a7a6ea480 2023-07-13T22:13:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:52.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1f999b700 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c5b0 tx=0x7fa1f0098480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f960b7b613fb0e8b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.714+0000 7fa1e8bff700 1 -- 172.21.15.26:0/721357319 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0098e30 con 0x555a7a6ea480 2023-07-13T22:13:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa1e8bff700 1 -- 172.21.15.26:0/721357319 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f007e070 con 0x555a7a6ea480 2023-07-13T22:13:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa1e8bff700 1 -- 172.21.15.26:0/721357319 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a20b0 con 0x555a7a6ea480 2023-07-13T22:13:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 -- 172.21.15.26:0/721357319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 msgr2=0x555a7a6f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c5b0 tx=0x7fa1f0098480 comp rx=0 tx=0).stop 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 -- 172.21.15.26:0/721357319 shutdown_connections 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6ea480 0x555a7a6f1fb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 --2- 172.21.15.26:0/721357319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6eadd0 0x555a7a6f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 -- 172.21.15.26:0/721357319 >> 172.21.15.26:0/721357319 conn(0x555a7a6dfd50 msgr2=0x555a7a6fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 -- 172.21.15.26:0/721357319 shutdown_connections 2023-07-13T22:13:52.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.715+0000 7fa20f602b80 1 -- 172.21.15.26:0/721357319 wait complete. 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 -- start start 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 Processor -- start 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a6fa690 con 0x555a7a6ea480 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a6fa800 con 0x555a7a6eadd0 2023-07-13T22:13:52.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a7a6fa9c0 con 0x555a7a6ebc20 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.716+0000 7fa20f602b80 10 client.0 _ll_register_callbacks cb 0x7fa1ec009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47036/0 (socket says 172.21.15.26:47036) 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 -- 172.21.15.26:0/4047334832 learned_addr learned my addr 172.21.15.26:0/4047334832 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f919a700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1fa19c700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 -- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 msgr2=0x555a7a6fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 -- 172.21.15.26:0/4047334832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 msgr2=0x555a7a6f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 -- 172.21.15.26:0/4047334832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a7a452c10 con 0x555a7a6ea480 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1f999b700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00999a0 tx=0x7fa1f009df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3704b872d1c0e5fa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00a20b0 con 0x555a7a6ea480 2023-07-13T22:13:52.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa1f007f080 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.717+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f00a2320 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1f00c0030 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 10 client.15168 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a7a468260 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1f00a2320 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa1f009a130 con 0x555a7a6ea480 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 10 client.15168 resolve_mds: resolved name 'f' to daemon mds.f 2023-07-13T22:13:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 20 client.15168 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a757fc0 0x555a7a754df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 4 client.15168 mds_command: new command op to 24493 tid=0[{"prefix": "status"}] 2023-07-13T22:13:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a7a5f7000 con 0x555a7a757fc0 2023-07-13T22:13:52.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.718+0000 7fa1f919a700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a757fc0 0x555a7a754df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa1f919a700 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a757fc0 0x555a7a754df0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15ff0ff260e40bd0 server_cookie=264c28fc0aff016 in_seq=0 out_seq=0 2023-07-13T22:13:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa1dd7fa700 10 client.15168 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:52.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa1dd7fa700 1 -- 172.21.15.26:0/4047334832 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x555a7a5f7000 con 0x555a7a757fc0 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa1dd7fa700 10 client.15168 handle_command_reply: tid=0 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa20f602b80 1 client.15168 shutdown 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 msgr2=0x555a7a6f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00999a0 tx=0x7fa1f009df80 comp rx=0 tx=0).stop 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 shutdown_connections 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.719+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555a7a757fc0 0x555a7a754df0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555a7a6ea480 0x555a7a6f6bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555a7a6ebc20 0x555a7a6fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1dd7fa700 0 client.15168 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 --2- 172.21.15.26:0/4047334832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a7a6eadd0 0x555a7a6f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 >> 172.21.15.26:0/4047334832 conn(0x555a7a6dfd50 msgr2=0x555a7a6f1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 shutdown_connections 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 1 -- 172.21.15.26:0/4047334832 wait complete. 2023-07-13T22:13:52.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa20f602b80 20 client.15168 trim_cache size 0 max 16384 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 3, 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24493, 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 88.285934912000002, 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 116.401876958 2023-07-13T22:13:52.720 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:52.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 msgr2=0x7fa1e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 0x7fa1e00a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4074920 tx=0x7fa1ec0094c0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 msgr2=0x7fa1f40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e970 tx=0x7fa1f000ec80 comp rx=0 tx=0).stop 2023-07-13T22:13:52.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 shutdown_connections 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa1e00a11d0 0x7fa1e00a3680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1f40ffae0 0x7fa1f40742c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1f4101660 0x7fa1f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 --2- 172.21.15.26:0/906376308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1f4100cd0 0x7fa1f4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.720+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 >> 172.21.15.26:0/906376308 conn(0x7fa1f40fb0e0 msgr2=0x7fa1f40fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.721+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 shutdown_connections 2023-07-13T22:13:52.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.721+0000 7fa1fbbff700 1 -- 172.21.15.26:0/906376308 wait complete. 2023-07-13T22:13:52.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:52 smithi026 ceph-mon[95647]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:52.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:52 smithi026 ceph-mon[95647]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-13T22:13:52.726 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:52 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2135418490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:52.727 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:52 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3244969661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:52.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-13T22:13:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 -- 172.21.15.26:0/1422714085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 msgr2=0x7f0108107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 --2- 172.21.15.26:0/1422714085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f0108107f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f010000b0a0 tx=0x7f010000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 -- 172.21.15.26:0/1422714085 shutdown_connections 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 --2- 172.21.15.26:0/1422714085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f0108107f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 --2- 172.21.15.26:0/1422714085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01080ffae0 0x7f01080ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 --2- 172.21.15.26:0/1422714085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0108100cd0 0x7f0108101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 -- 172.21.15.26:0/1422714085 >> 172.21.15.26:0/1422714085 conn(0x7f01080fb0e0 msgr2=0x7f01080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 -- 172.21.15.26:0/1422714085 shutdown_connections 2023-07-13T22:13:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.942+0000 7f010efff700 1 -- 172.21.15.26:0/1422714085 wait complete. 2023-07-13T22:13:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 Processor -- start 2023-07-13T22:13:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 -- start start 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01080ffae0 0x7f0108197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0108100cd0 0x7f01081980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010819f6f0 con 0x7f0108101660 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010819f830 con 0x7f0108100cd0 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.943+0000 7f010efff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010819f9c0 con 0x7f01080ffae0 2023-07-13T22:13:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45302/0 (socket says 172.21.15.26:45302) 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 -- 172.21.15.26:0/2692422788 learned_addr learned my addr 172.21.15.26:0/2692422788 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f0107fff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0108100cd0 0x7f01081980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010cd9b700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01080ffae0 0x7f0108197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 -- 172.21.15.26:0/2692422788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01080ffae0 msgr2=0x7f0108197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01080ffae0 0x7f0108197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 -- 172.21.15.26:0/2692422788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0108100cd0 msgr2=0x7f01081980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0108100cd0 0x7f01081980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 -- 172.21.15.26:0/2692422788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0100009d00 con 0x7f0108101660 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.944+0000 7f010d59c700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f010000d010 tx=0x7f010000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.945+0000 7f0105ffb700 1 -- 172.21.15.26:0/2692422788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000e040 con 0x7f0108101660 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.945+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010819fc40 con 0x7f0108101660 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.945+0000 7f0105ffb700 1 -- 172.21.15.26:0/2692422788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0100003e10 con 0x7f0108101660 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.945+0000 7f0105ffb700 1 -- 172.21.15.26:0/2692422788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01000052a0 con 0x7f0108101660 2023-07-13T22:13:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.945+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01081a0010 con 0x7f0108101660 2023-07-13T22:13:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.948+0000 7f0105ffb700 1 -- 172.21.15.26:0/2692422788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f010001d480 con 0x7f0108101660 2023-07-13T22:13:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.949+0000 7f0122936b80 1 Processor -- start 2023-07-13T22:13:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.949+0000 7f0122936b80 1 -- start start 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da8d5c60 con 0x5573da9d0480 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da9289f0 con 0x5573da9d0dd0 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0122936b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da921e80 con 0x5573da9d1c20 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0105ffb700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 0x7f00f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0105ffb700 1 -- 172.21.15.26:0/2692422788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f01000cdc50 con 0x7f0108101660 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f010cd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45308/0 (socket says 172.21.15.26:45308) 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0107fff700 1 -- 172.21.15.26:0/998106601 learned_addr learned my addr 172.21.15.26:0/998106601 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f010d59c700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 0x7f00f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f0107fff700 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.950+0000 7f010cd9b700 1 -- 172.21.15.26:0/998106601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 msgr2=0x5573da9d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010cd9b700 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010cd9b700 1 -- 172.21.15.26:0/998106601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 msgr2=0x5573da9d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010cd9b700 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010cd9b700 1 -- 172.21.15.26:0/998106601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573da74fd90 con 0x5573da9d0dd0 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f0107fff700 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010d59c700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 0x7f00f80a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f010009ae00 tx=0x7f010001db90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f010cd9b700 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9d1690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c80 tx=0x7f00fc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7abbfac7af2cde server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f00f2ffd700 1 -- 172.21.15.26:0/998106601 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011cb0 con 0x5573da9d0dd0 2023-07-13T22:13:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f00f2ffd700 1 -- 172.21.15.26:0/998106601 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc011e10 con 0x5573da9d0dd0 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.951+0000 7f00f2ffd700 1 -- 172.21.15.26:0/998106601 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005280 con 0x5573da9d0dd0 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- 172.21.15.26:0/998106601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 msgr2=0x5573da9d1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9d1690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c80 tx=0x7f00fc009f90 comp rx=0 tx=0).stop 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- 172.21.15.26:0/998106601 shutdown_connections 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 --2- 172.21.15.26:0/998106601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9d1690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- 172.21.15.26:0/998106601 >> 172.21.15.26:0/998106601 conn(0x5573da9c5d50 msgr2=0x5573da9cb180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- 172.21.15.26:0/998106601 shutdown_connections 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- 172.21.15.26:0/998106601 wait complete. 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.952+0000 7f0122936b80 1 -- start start 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 Processor -- start 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da8c1a70 con 0x5573da9d0480 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da8c1770 con 0x5573da9d0dd0 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da7a58d0 con 0x5573da9d1c20 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0122936b80 10 client.0 _ll_register_callbacks cb 0x5573da8bf710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.953+0000 7f0107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f0107fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45324/0 (socket says 172.21.15.26:45324) 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f0107fff700 1 -- 172.21.15.26:0/547865961 learned_addr learned my addr 172.21.15.26:0/547865961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f0107fff700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 -- 172.21.15.26:0/547865961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 msgr2=0x5573da9e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9e1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 -- 172.21.15.26:0/547865961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 msgr2=0x5573da9da100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 -- 172.21.15.26:0/547865961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4013460 con 0x5573da9d0dd0 2023-07-13T22:13:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f0107fff700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:52.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:52 smithi117 ceph-mon[129074]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:52.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:52 smithi117 ceph-mon[129074]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-13T22:13:52.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:52 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2135418490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:52.950 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:52 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3244969661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f010cd9b700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9da640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c80 tx=0x7f00fc011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3168d580ed8f7045 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f00e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:13:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.954+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc005080 con 0x5573da9d0dd0 2023-07-13T22:13:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573da87b4a0 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f00fc0051e0 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00fc0053a0 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 10 client.25214 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f00fc0098e0 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573da87b4a0 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00fc022400 con 0x5573da9d0dd0 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 10 client.25214 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 10 client.25214 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 20 client.25214 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573da9e5270 0x5573da9e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 4 client.25214 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.955+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573da91ae60 con 0x5573da9e5270 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.956+0000 7f0107fff700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573da9e5270 0x5573da9e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.956+0000 7f0107fff700 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573da9e5270 0x5573da9e46d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=784a3938c0c0c80 server_cookie=5bedbcb4ba4a9037 in_seq=0 out_seq=0 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.956+0000 7f00e7fff700 10 client.25214 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f00e7fff700 1 -- 172.21.15.26:0/547865961 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5573da91ae60 con 0x5573da9e5270 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f00e7fff700 10 client.25214 handle_command_reply: tid=0 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 client.25214 shutdown 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 msgr2=0x5573da9da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9da640 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009c80 tx=0x7f00fc011b40 comp rx=0 tx=0).stop 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 shutdown_connections 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0480 0x5573da9da100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573da9e5270 0x5573da9e46d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9e1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 --2- 172.21.15.26:0/547865961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0dd0 0x5573da9da640 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 >> 172.21.15.26:0/547865961 conn(0x5573da9c5d50 msgr2=0x5573da9ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f00e7fff700 0 client.25214 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 shutdown_connections 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.957+0000 7f0122936b80 1 -- 172.21.15.26:0/547865961 wait complete. 2023-07-13T22:13:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.958+0000 7f0122936b80 20 client.25214 trim_cache size 0 max 16384 2023-07-13T22:13:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.976+0000 7f0122936b80 1 Processor -- start 2023-07-13T22:13:52.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.976+0000 7f0122936b80 1 -- start start 2023-07-13T22:13:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0480 0x5573da9edfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9ca860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da9cc630 con 0x5573da9d0dd0 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da9db2f0 con 0x5573da9d0480 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0122936b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573da7b1de0 con 0x5573da9d1c20 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45334/0 (socket says 172.21.15.26:45334) 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 -- 172.21.15.26:0/2487274236 learned_addr learned my addr 172.21.15.26:0/2487274236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0107fff700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0480 0x5573da9edfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0107fff700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 -- 172.21.15.26:0/2487274236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 msgr2=0x5573da9ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 -- 172.21.15.26:0/2487274236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0480 msgr2=0x5573da9edfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0480 0x5573da9edfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 -- 172.21.15.26:0/2487274236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573da738c10 con 0x5573da9d0dd0 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f0107fff700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.977+0000 7f010cd9b700 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239515a1132a4b56 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f00f37fe700 1 -- 172.21.15.26:0/2487274236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011bb0 con 0x5573da9d0dd0 2023-07-13T22:13:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f00f37fe700 1 -- 172.21.15.26:0/2487274236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc004b20 con 0x5573da9d0dd0 2023-07-13T22:13:52.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f00f37fe700 1 -- 172.21.15.26:0/2487274236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc022400 con 0x5573da9d0dd0 2023-07-13T22:13:52.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 -- 172.21.15.26:0/2487274236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 msgr2=0x5573da9e15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f00fc000c00 tx=0x7f00fc001e70 comp rx=0 tx=0).stop 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 -- 172.21.15.26:0/2487274236 shutdown_connections 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d1c20 0x5573da9ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d0480 0x5573da9edfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 --2- 172.21.15.26:0/2487274236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 -- 172.21.15.26:0/2487274236 >> 172.21.15.26:0/2487274236 conn(0x5573da9c5d50 msgr2=0x5573da9cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 -- 172.21.15.26:0/2487274236 shutdown_connections 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.978+0000 7f0122936b80 1 -- 172.21.15.26:0/2487274236 wait complete. 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.979+0000 7f0122936b80 1 -- start start 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.979+0000 7f0122936b80 1 Processor -- start 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 0x5573da9dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d1c20 0x5573da9dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573daa390f0 con 0x5573da9d0dd0 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573daa39230 con 0x5573da9d1c20 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573daa39370 con 0x5573da9d0480 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0122936b80 10 client.0 _ll_register_callbacks cb 0x7f00f40098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45348/0 (socket says 172.21.15.26:45348) 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 -- 172.21.15.26:0/410057405 learned_addr learned my addr 172.21.15.26:0/410057405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0107fff700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 0x5573da9dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0107fff700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d1c20 0x5573da9dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 -- 172.21.15.26:0/410057405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 msgr2=0x5573da9dc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 0x5573da9dc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 -- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d1c20 msgr2=0x5573da9dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d1c20 0x5573da9dc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f010cd9b700 1 -- 172.21.15.26:0/410057405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0098e0 con 0x5573da9d0dd0 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.980+0000 7f0107fff700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 0x5573da9dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f010cd9b700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0227c0 tx=0x7f00fc022ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827b5f15d953ce19 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f00e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc02a3b0 con 0x5573da9d0dd0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f00fc02a510 con 0x5573da9d0dd0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00fc011e10 con 0x5573da9d0dd0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00fc045030 con 0x5573da9d0dd0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f0122936b80 10 client.15186 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5573da776200 con 0x5573da9d0dd0 2023-07-13T22:13:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.981+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc011e10 con 0x5573da9d0dd0 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00fc004420 con 0x5573da9d0dd0 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 10 client.15186 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 20 client.15186 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573daa3db10 0x5573daa3a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 4 client.15186 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5573da8dd000 con 0x5573daa3db10 2023-07-13T22:13:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0107fff700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573daa3db10 0x5573daa3a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f0107fff700 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573daa3db10 0x5573daa3a500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1d05ce685c791b6 server_cookie=b39722aa40700a31 in_seq=0 out_seq=0 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.982+0000 7f00e7fff700 10 client.15186 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f00e7fff700 1 -- 172.21.15.26:0/410057405 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5573da8dd000 con 0x5573daa3db10 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f00e7fff700 10 client.15186 handle_command_reply: tid=0 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 client.15186 shutdown 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 msgr2=0x5573da9e15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0227c0 tx=0x7f00fc022ae0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 shutdown_connections 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5573daa3db10 0x5573daa3a500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573da9d1c20 0x5573da9dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5573da9d0480 0x5573da9dc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 --2- 172.21.15.26:0/410057405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5573da9d0dd0 0x5573da9e15e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f00e7fff700 0 client.15186 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 >> 172.21.15.26:0/410057405 conn(0x5573da9c5d50 msgr2=0x5573da9e46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 shutdown_connections 2023-07-13T22:13:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 1 -- 172.21.15.26:0/410057405 wait complete. 2023-07-13T22:13:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.983+0000 7f0122936b80 20 client.15186 trim_cache size 0 max 16384 2023-07-13T22:13:52.982 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 2, 2023-07-13T22:13:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24530, 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 88.998711080999996, 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 112.427354616 2023-07-13T22:13:52.984 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 msgr2=0x7f00f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 0x7f00f80a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f010009ae00 tx=0x7f010001db90 comp rx=0 tx=0).stop 2023-07-13T22:13:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 msgr2=0x7f010819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f010000d010 tx=0x7f010000bc80 comp rx=0 tx=0).stop 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 shutdown_connections 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f00f80a1150 0x7f00f80a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0108101660 0x7f010819f120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01080ffae0 0x7f0108197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 --2- 172.21.15.26:0/2692422788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0108100cd0 0x7f01081980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 >> 172.21.15.26:0/2692422788 conn(0x7f01080fb0e0 msgr2=0x7f0108106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 shutdown_connections 2023-07-13T22:13:52.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:52.984+0000 7f010efff700 1 -- 172.21.15.26:0/2692422788 wait complete. 2023-07-13T22:13:52.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-13T22:13:52.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:52 smithi121 ceph-mon[129187]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:13:52.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:52 smithi121 ceph-mon[129187]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-13T22:13:52.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:52 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2135418490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:13:52.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:52 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3244969661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:13:53.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 -- 172.21.15.26:0/2136257276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 msgr2=0x7f04d00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 --2- 172.21.15.26:0/2136257276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00fff00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009990 tx=0x7f04c4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 -- 172.21.15.26:0/2136257276 shutdown_connections 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 --2- 172.21.15.26:0/2136257276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0101670 0x7f04d0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 --2- 172.21.15.26:0/2136257276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00fff00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 --2- 172.21.15.26:0/2136257276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0100ce0 0x7f04d0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 -- 172.21.15.26:0/2136257276 >> 172.21.15.26:0/2136257276 conn(0x7f04d00fb130 msgr2=0x7f04d00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 -- 172.21.15.26:0/2136257276 shutdown_connections 2023-07-13T22:13:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.206+0000 7f04d6fff700 1 -- 172.21.15.26:0/2136257276 wait complete. 2023-07-13T22:13:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.207+0000 7f04d6fff700 1 Processor -- start 2023-07-13T22:13:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.207+0000 7f04d6fff700 1 -- start start 2023-07-13T22:13:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.207+0000 7f04d6fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d6fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0100ce0 0x7f04d0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d6fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0101670 0x7f04d0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d6fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00735e0 con 0x7f04d0101670 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d6fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0073750 con 0x7f04d00ffaf0 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d6fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d019f8d0 con 0x7f04d0100ce0 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d559c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0101670 0x7f04d0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34082/0 (socket says 172.21.15.26:34082) 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 -- 172.21.15.26:0/1884254958 learned_addr learned my addr 172.21.15.26:0/1884254958 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04cffff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0100ce0 0x7f04d0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 -- 172.21.15.26:0/1884254958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0100ce0 msgr2=0x7f04d0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0100ce0 0x7f04d0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 -- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0101670 msgr2=0x7f04d0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0101670 0x7f04d0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.208+0000 7f04d4d9b700 1 -- 172.21.15.26:0/1884254958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c40095f0 con 0x7f04d00ffaf0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04c4004730 tx=0x7f04c4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04cdffb700 1 -- 172.21.15.26:0/1884254958 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c401d070 con 0x7f04d00ffaf0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04cdffb700 1 -- 172.21.15.26:0/1884254958 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c4021410 con 0x7f04d00ffaf0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d019fb50 con 0x7f04d00ffaf0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04cdffb700 1 -- 172.21.15.26:0/1884254958 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c4017650 con 0x7f04d00ffaf0 2023-07-13T22:13:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.209+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d019ffd0 con 0x7f04d00ffaf0 2023-07-13T22:13:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.212+0000 7f04cdffb700 1 -- 172.21.15.26:0/1884254958 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f04c4021580 con 0x7f04d00ffaf0 2023-07-13T22:13:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 Processor -- start 2023-07-13T22:13:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04cdffb700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 0x7f04c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04cdffb700 1 -- 172.21.15.26:0/1884254958 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04c40cdcc0 con 0x7f04d00ffaf0 2023-07-13T22:13:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 -- start start 2023-07-13T22:13:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04d559c700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 0x7f04c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 0x56276df2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276de33c60 con 0x56276df2edd0 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276de869f0 con 0x56276df2e480 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.214+0000 7f04ea832b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276de7fe80 con 0x56276df2fc20 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 0x56276df2f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34096/0 (socket says 172.21.15.26:34096) 2023-07-13T22:13:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 -- 172.21.15.26:0/4046680697 learned_addr learned my addr 172.21.15.26:0/4046680697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d559c700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 0x7f04c00a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04d0074920 tx=0x7f04c800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d559c700 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 -- 172.21.15.26:0/4046680697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 msgr2=0x56276df30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 -- 172.21.15.26:0/4046680697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 msgr2=0x56276df2f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 0x56276df2f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 -- 172.21.15.26:0/4046680697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276dcadd90 con 0x56276df2e480 2023-07-13T22:13:53.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04cffff700 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 0x56276df2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04c409b0a0 tx=0x7f04c409acd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2131ce4a15d01e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.215+0000 7f04baffd700 1 -- 172.21.15.26:0/4046680697 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c409f140 con 0x56276df2e480 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04baffd700 1 -- 172.21.15.26:0/4046680697 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c40a34d0 con 0x56276df2e480 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04baffd700 1 -- 172.21.15.26:0/4046680697 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c409a840 con 0x56276df2e480 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 -- 172.21.15.26:0/4046680697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 msgr2=0x56276df2e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04c409b0a0 tx=0x7f04c409acd0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 -- 172.21.15.26:0/4046680697 shutdown_connections 2023-07-13T22:13:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2e480 0x56276df2e890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 --2- 172.21.15.26:0/4046680697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2edd0 0x56276df2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 -- 172.21.15.26:0/4046680697 >> 172.21.15.26:0/4046680697 conn(0x56276df23d50 msgr2=0x56276df29180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 -- 172.21.15.26:0/4046680697 shutdown_connections 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.216+0000 7f04ea832b80 1 -- 172.21.15.26:0/4046680697 wait complete. 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.217+0000 7f04ea832b80 1 -- start start 2023-07-13T22:13:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.217+0000 7f04ea832b80 1 Processor -- start 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df38290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df3f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276de1fa70 con 0x56276df2e480 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276de1f770 con 0x56276df2edd0 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276dd038d0 con 0x56276df2fc20 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04ea832b80 10 client.0 _ll_register_callbacks cb 0x56276de1d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:53.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04d4d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45396/0 (socket says 172.21.15.26:45396) 2023-07-13T22:13:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04d4d9b700 1 -- 172.21.15.26:0/3313624828 learned_addr learned my addr 172.21.15.26:0/3313624828 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.218+0000 7f04cffff700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df38290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d559c700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df3f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 -- 172.21.15.26:0/3313624828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 msgr2=0x56276df3f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df3f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 -- 172.21.15.26:0/3313624828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 msgr2=0x56276df38290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df38290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 -- 172.21.15.26:0/3313624828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc0098e0 con 0x56276df2e480 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04cffff700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df38290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f04c4096e90 tx=0x7f04c409b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f0fcea3d15757f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04affff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b9070 con 0x56276df2e480 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04c40a3040 con 0x56276df2e480 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56276ddd94a0 con 0x56276df2e480 2023-07-13T22:13:53.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c409f1c0 con 0x56276df2e480 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04ea832b80 10 client.15198 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.219+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f04d019ffd0 con 0x56276df2e480 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56276ddd94a0 con 0x56276df2e480 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04c4097820 con 0x56276df2e480 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 10 client.15198 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 20 client.15198 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:53.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df43070 0x56276df42200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 4 client.15198 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56276de78e60 con 0x56276df43070 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.220+0000 7f04cffff700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df43070 0x56276df42200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.221+0000 7f04cffff700 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df43070 0x56276df42200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ec93fbd81cc1f29 server_cookie=af54d38796c63cb4 in_seq=0 out_seq=0 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.221+0000 7f04affff700 10 client.15198 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.221+0000 7f04affff700 1 -- 172.21.15.26:0/3313624828 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56276de78e60 con 0x56276df43070 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.221+0000 7f04affff700 10 client.15198 handle_command_reply: tid=0 2023-07-13T22:13:53.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.221+0000 7f04ea832b80 1 client.15198 shutdown 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 msgr2=0x56276df37d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f04c4096e90 tx=0x7f04c409b290 comp rx=0 tx=0).stop 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 shutdown_connections 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df43070 0x56276df42200 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df37d50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df2fc20 0x56276df3f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04affff700 0 client.15198 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 --2- 172.21.15.26:0/3313624828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df38290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 >> 172.21.15.26:0/3313624828 conn(0x56276df23d50 msgr2=0x56276df28860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 shutdown_connections 2023-07-13T22:13:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 1 -- 172.21.15.26:0/3313624828 wait complete. 2023-07-13T22:13:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.222+0000 7f04ea832b80 20 client.15198 trim_cache size 0 max 16384 2023-07-13T22:13:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.240+0000 7f04ea832b80 1 Processor -- start 2023-07-13T22:13:53.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.240+0000 7f04ea832b80 1 -- start start 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276df2a0a0 con 0x56276df2e480 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276df43d60 con 0x56276df2edd0 2023-07-13T22:13:53.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04ea832b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276ddd94a0 con 0x56276df23d50 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45410/0 (socket says 172.21.15.26:45410) 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 -- 172.21.15.26:0/988533387 learned_addr learned my addr 172.21.15.26:0/988533387 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04d559c700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 -- 172.21.15.26:0/988533387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 msgr2=0x56276df32a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df32a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 -- 172.21.15.26:0/988533387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 msgr2=0x56276df3f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04cffff700 1 -- 172.21.15.26:0/988533387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276dc96c10 con 0x56276df2e480 2023-07-13T22:13:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.241+0000 7f04d559c700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04cffff700 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56276df38db0 tx=0x7f04bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e074033c417204 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04bb7fe700 1 -- 172.21.15.26:0/988533387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc017040 con 0x56276df2e480 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04bb7fe700 1 -- 172.21.15.26:0/988533387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc018040 con 0x56276df2e480 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04bb7fe700 1 -- 172.21.15.26:0/988533387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc005280 con 0x56276df2e480 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04ea832b80 1 -- 172.21.15.26:0/988533387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 msgr2=0x56276df32fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.242+0000 7f04ea832b80 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56276df38db0 tx=0x7f04bc009260 comp rx=0 tx=0).stop 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 -- 172.21.15.26:0/988533387 shutdown_connections 2023-07-13T22:13:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df32a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 --2- 172.21.15.26:0/988533387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df32fc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 -- 172.21.15.26:0/988533387 >> 172.21.15.26:0/988533387 conn(0x56276df36e00 msgr2=0x56276df3fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 -- 172.21.15.26:0/988533387 shutdown_connections 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 -- 172.21.15.26:0/988533387 wait complete. 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 -- start start 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.243+0000 7f04ea832b80 1 Processor -- start 2023-07-13T22:13:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276df3ec00 con 0x56276df2e480 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276df3ed70 con 0x56276df2edd0 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276df3ef30 con 0x56276df23d50 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04ea832b80 10 client.0 _ll_register_callbacks cb 0x7f04bc009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45420/0 (socket says 172.21.15.26:45420) 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 -- 172.21.15.26:0/1041704237 learned_addr learned my addr 172.21.15.26:0/1041704237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04d559c700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 -- 172.21.15.26:0/1041704237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 msgr2=0x56276df3b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df3b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 -- 172.21.15.26:0/1041704237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 msgr2=0x56276df3e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04cffff700 1 -- 172.21.15.26:0/1041704237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c80090c0 con 0x56276df2e480 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.244+0000 7f04d559c700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04cffff700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005e10 tx=0x7f04bc0121e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf1e3bf2f7aaa7a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04affff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0210f0 con 0x56276df2e480 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04c40a34d0 con 0x56276df2e480 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f04bc021250 con 0x56276df2e480 2023-07-13T22:13:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04bc021410 con 0x56276df2e480 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04ea832b80 10 client.15204 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f04bc012570 con 0x56276df2e480 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.245+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c40a34d0 con 0x56276df2e480 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f04bc02a450 con 0x56276df2e480 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 10 client.15204 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 20 client.15204 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df2fc20 0x56276df99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 4 client.15204 mds_command: new command op to 24563 tid=0[{"prefix": "status"}] 2023-07-13T22:13:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56276de3b000 con 0x56276df2fc20 2023-07-13T22:13:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df2fc20 0x56276df99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04d4d9b700 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df2fc20 0x56276df99670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4da22e4d6e946122 server_cookie=ea3cc1ebcccf52fa in_seq=0 out_seq=0 2023-07-13T22:13:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.246+0000 7f04affff700 10 client.15204 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04affff700 1 -- 172.21.15.26:0/1041704237 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56276de3b000 con 0x56276df2fc20 2023-07-13T22:13:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04affff700 10 client.15204 handle_command_reply: tid=0 2023-07-13T22:13:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 client.15204 shutdown 2023-07-13T22:13:53.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 msgr2=0x56276df3b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005e10 tx=0x7f04bc0121e0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 shutdown_connections 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56276df2edd0 0x56276df3e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56276df23d50 0x56276df3b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x56276df2fc20 0x56276df99670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 --2- 172.21.15.26:0/1041704237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56276df2e480 0x56276df3b6a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04affff700 0 client.15204 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:13:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 >> 172.21.15.26:0/1041704237 conn(0x56276df36e00 msgr2=0x56276df32a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 shutdown_connections 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 1 -- 172.21.15.26:0/1041704237 wait complete. 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.247+0000 7f04ea832b80 20 client.15204 trim_cache size 0 max 16384 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 1, 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24563, 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 90.271183366000002, 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 104.16775691700001 2023-07-13T22:13:53.248 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 msgr2=0x7f04c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 0x7f04c00a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f04d0074920 tx=0x7f04c800b040 comp rx=0 tx=0).stop 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 msgr2=0x7f04d00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f04c4004730 tx=0x7f04c4004760 comp rx=0 tx=0).stop 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 shutdown_connections 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f04c00a11d0 0x7f04c00a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d0101670 0x7f04d0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f04d00ffaf0 0x7f04d00742c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 --2- 172.21.15.26:0/1884254958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04d0100ce0 0x7f04d0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 >> 172.21.15.26:0/1884254958 conn(0x7f04d00fb130 msgr2=0x7f04d00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 shutdown_connections 2023-07-13T22:13:53.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.248+0000 7f04d6fff700 1 -- 172.21.15.26:0/1884254958 wait complete. 2023-07-13T22:13:53.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-13T22:13:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 -- 172.21.15.26:0/3044472406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 msgr2=0x7f52600733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 --2- 172.21.15.26:0/3044472406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f52600733e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52540098f0 tx=0x7f5254009c00 comp rx=0 tx=0).stop 2023-07-13T22:13:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 -- 172.21.15.26:0/3044472406 shutdown_connections 2023-07-13T22:13:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 --2- 172.21.15.26:0/3044472406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52601079a0 0x7f5260109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 --2- 172.21.15.26:0/3044472406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5260074550 0x7f52600729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 --2- 172.21.15.26:0/3044472406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f52600733e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 -- 172.21.15.26:0/3044472406 >> 172.21.15.26:0/3044472406 conn(0x7f52600fb0e0 msgr2=0x7f52600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.470+0000 7f52659ff700 1 -- 172.21.15.26:0/3044472406 shutdown_connections 2023-07-13T22:13:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.471+0000 7f52659ff700 1 -- 172.21.15.26:0/3044472406 wait complete. 2023-07-13T22:13:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.471+0000 7f52659ff700 1 Processor -- start 2023-07-13T22:13:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 -- start start 2023-07-13T22:13:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 0x7f5260198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52601079a0 0x7f526019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526019f6b0 con 0x7f5260074550 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526019f7f0 con 0x7f52601079a0 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f52659ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526019f980 con 0x7f5260072f70 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f525effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f525e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 0x7f5260198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f525effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:43974/0 (socket says 172.21.15.26:43974) 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f525effd700 1 -- 172.21.15.26:0/2635170155 learned_addr learned my addr 172.21.15.26:0/2635170155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.472+0000 7f525f7fe700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52601079a0 0x7f526019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 -- 172.21.15.26:0/2635170155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52601079a0 msgr2=0x7f526019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52601079a0 0x7f526019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 -- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 msgr2=0x7f5260198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 0x7f5260198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 -- 172.21.15.26:0/2635170155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500095f0 con 0x7f5260072f70 2023-07-13T22:13:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525e7fc700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 0x7f5260198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f525effd700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f525000ec70 tx=0x7f525000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f52649fd700 1 -- 172.21.15.26:0/2635170155 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52500099d0 con 0x7f5260072f70 2023-07-13T22:13:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5254009520 con 0x7f5260072f70 2023-07-13T22:13:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f52649fd700 1 -- 172.21.15.26:0/2635170155 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52500044e0 con 0x7f5260072f70 2023-07-13T22:13:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f52649fd700 1 -- 172.21.15.26:0/2635170155 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250010430 con 0x7f5260072f70 2023-07-13T22:13:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.473+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526019ffe0 con 0x7f5260072f70 2023-07-13T22:13:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.477+0000 7f52649fd700 1 -- 172.21.15.26:0/2635170155 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5250003f10 con 0x7f5260072f70 2023-07-13T22:13:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.478+0000 7f5279288b80 1 Processor -- start 2023-07-13T22:13:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.478+0000 7f5279288b80 1 -- start start 2023-07-13T22:13:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb182890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb183690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb087c60 con 0x55fdcb182480 2023-07-13T22:13:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb0da9f0 con 0x55fdcb183c20 2023-07-13T22:13:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f5279288b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb0d3e80 con 0x55fdcb182dd0 2023-07-13T22:13:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f52649fd700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 0x7f52480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f52649fd700 1 -- 172.21.15.26:0/2635170155 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5250014070 con 0x7f5260072f70 2023-07-13T22:13:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb183690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34162/0 (socket says 172.21.15.26:34162) 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525e7fc700 1 -- 172.21.15.26:0/1141840370 learned_addr learned my addr 172.21.15.26:0/1141840370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525f7fe700 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb182890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.479+0000 7f525f7fe700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 0x7f52480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 -- 172.21.15.26:0/1141840370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 msgr2=0x55fdcb183690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb183690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 -- 172.21.15.26:0/1141840370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 msgr2=0x55fdcb182890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb182890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 -- 172.21.15.26:0/1141840370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdcaf01d90 con 0x55fdcb183c20 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525f7fe700 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb182890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525e7fc700 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5254005e90 tx=0x7f5254005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=142035244c816059 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f525f7fe700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 0x7f52480a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5258009ec0 tx=0x7f525800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f5245ffb700 1 -- 172.21.15.26:0/1141840370 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525401c050 con 0x55fdcb183c20 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f5245ffb700 1 -- 172.21.15.26:0/1141840370 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525401d040 con 0x55fdcb183c20 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.480+0000 7f5245ffb700 1 -- 172.21.15.26:0/1141840370 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540053c0 con 0x55fdcb183c20 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 -- 172.21.15.26:0/1141840370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 msgr2=0x55fdcb184090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5254005e90 tx=0x7f5254005dc0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 -- 172.21.15.26:0/1141840370 shutdown_connections 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb182890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb184090 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 --2- 172.21.15.26:0/1141840370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb183690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 -- 172.21.15.26:0/1141840370 >> 172.21.15.26:0/1141840370 conn(0x55fdcb177d50 msgr2=0x55fdcb17d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 -- 172.21.15.26:0/1141840370 shutdown_connections 2023-07-13T22:13:53.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.481+0000 7f5279288b80 1 -- 172.21.15.26:0/1141840370 wait complete. 2023-07-13T22:13:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.482+0000 7f5279288b80 1 -- start start 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.482+0000 7f5279288b80 1 Processor -- start 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb183c20 0x55fdcb1935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb073a70 con 0x55fdcb182480 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb073770 con 0x55fdcb182dd0 2023-07-13T22:13:53.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcaf578d0 con 0x55fdcb183c20 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f5279288b80 10 client.0 _ll_register_callbacks cb 0x55fdcb071710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f525effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f525effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45462/0 (socket says 172.21.15.26:45462) 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f525effd700 1 -- 172.21.15.26:0/631329538 learned_addr learned my addr 172.21.15.26:0/631329538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f525e7fc700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.483+0000 7f525f7fe700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb183c20 0x55fdcb1935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 -- 172.21.15.26:0/631329538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb183c20 msgr2=0x55fdcb1935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb183c20 0x55fdcb1935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 -- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 msgr2=0x55fdcb18c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 -- 172.21.15.26:0/631329538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526010aa40 con 0x55fdcb182480 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525e7fc700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f525effd700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f525000c5b0 tx=0x7f5250099e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4a7b30a09323d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f523effd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:13:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525009a220 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f525009e250 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdcb02d4a0 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f525009f0b0 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f5279288b80 10 client.15216 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fdcaeeac10 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.484+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdcb02d4a0 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f525009a480 con 0x55fdcb182480 2023-07-13T22:13:53.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 20 client.15216 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1972b0 0x55fdcb1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 4 client.15216 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdcb0cce60 con 0x55fdcb1972b0 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.485+0000 7f525e7fc700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1972b0 0x55fdcb1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.486+0000 7f525e7fc700 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1972b0 0x55fdcb1966e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f95ae9b399e6d80 server_cookie=195617fee12ac85a in_seq=0 out_seq=0 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.486+0000 7f523effd700 10 client.15216 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f523effd700 1 -- 172.21.15.26:0/631329538 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fdcb0cce60 con 0x55fdcb1972b0 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f523effd700 10 client.15216 handle_command_reply: tid=0 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 client.15216 shutdown 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 msgr2=0x55fdcb18c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f525000c5b0 tx=0x7f5250099e70 comp rx=0 tx=0).stop 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 shutdown_connections 2023-07-13T22:13:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb18c050 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1972b0 0x55fdcb1966e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb183c20 0x55fdcb1935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 --2- 172.21.15.26:0/631329538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f523effd700 0 client.15216 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 >> 172.21.15.26:0/631329538 conn(0x55fdcb177d50 msgr2=0x55fdcb17c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 shutdown_connections 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 1 -- 172.21.15.26:0/631329538 wait complete. 2023-07-13T22:13:53.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.487+0000 7f5279288b80 20 client.15216 trim_cache size 0 max 16384 2023-07-13T22:13:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 Processor -- start 2023-07-13T22:13:53.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 -- start start 2023-07-13T22:13:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb193590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb18a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb07d470 con 0x55fdcb182480 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb0a2790 con 0x55fdcb183c20 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f5279288b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcaf63de0 con 0x55fdcb182dd0 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f525effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f525effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45464/0 (socket says 172.21.15.26:45464) 2023-07-13T22:13:53.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f525effd700 1 -- 172.21.15.26:0/3284122567 learned_addr learned my addr 172.21.15.26:0/3284122567 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f525e7fc700 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb193590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.506+0000 7f525f7fe700 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb18a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 -- 172.21.15.26:0/3284122567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 msgr2=0x55fdcb193590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb193590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 -- 172.21.15.26:0/3284122567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 msgr2=0x55fdcb18a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb18a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 -- 172.21.15.26:0/3284122567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdcaf28200 con 0x55fdcb182480 2023-07-13T22:13:53.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f525effd700 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5250096c90 tx=0x7f52500987d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc9b7a9354be83d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f52467fc700 1 -- 172.21.15.26:0/3284122567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250062080 con 0x55fdcb182480 2023-07-13T22:13:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f52467fc700 1 -- 172.21.15.26:0/3284122567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525009e250 con 0x55fdcb182480 2023-07-13T22:13:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.507+0000 7f52467fc700 1 -- 172.21.15.26:0/3284122567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250099760 con 0x55fdcb182480 2023-07-13T22:13:53.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- 172.21.15.26:0/3284122567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 msgr2=0x55fdcb1966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5250096c90 tx=0x7f52500987d0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- 172.21.15.26:0/3284122567 shutdown_connections 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb182480 0x55fdcb1966e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb183c20 0x55fdcb18a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 --2- 172.21.15.26:0/3284122567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182dd0 0x55fdcb193590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- 172.21.15.26:0/3284122567 >> 172.21.15.26:0/3284122567 conn(0x55fdcb177d50 msgr2=0x55fdcb18ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- 172.21.15.26:0/3284122567 shutdown_connections 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- 172.21.15.26:0/3284122567 wait complete. 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 -- start start 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.508+0000 7f5279288b80 1 Processor -- start 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 0x55fdcb18e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb193760 con 0x55fdcb183c20 2023-07-13T22:13:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb1938d0 con 0x55fdcb182dd0 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdcb1eb6b0 con 0x55fdcb182480 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f5279288b80 10 client.0 _ll_register_callbacks cb 0x7f5250099ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f525f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f525effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 0x55fdcb18e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f525f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45480/0 (socket says 172.21.15.26:45480) 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f525f7fe700 1 -- 172.21.15.26:0/475165271 learned_addr learned my addr 172.21.15.26:0/475165271 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.509+0000 7f525e7fc700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 -- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 msgr2=0x55fdcb18ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 -- 172.21.15.26:0/475165271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 msgr2=0x55fdcb1eb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 -- 172.21.15.26:0/475165271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdcaeeac10 con 0x55fdcb182480 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525f7fe700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f525effd700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 0x55fdcb18e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f525009f270 tx=0x7f5250016470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df0966284801b67a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f523effd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250062030 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f525401d040 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5250063070 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5250099d60 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f5279288b80 10 client.25231 fetch_fsmap learned FSMap version 27 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fdcaf00260 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.510+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f525401d040 con 0x55fdcb182480 2023-07-13T22:13:53.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5250099180 con 0x55fdcb182480 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 10 client.25231 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 10 client.25231 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 20 client.25231 populate_metadata read hostname 'smithi026' 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1efcc0 0x55fdcb1ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 4 client.25231 mds_command: new command op to 24575 tid=0[{"prefix": "status"}] 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fdcb08f000 con 0x55fdcb1efcc0 2023-07-13T22:13:53.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f525e7fc700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1efcc0 0x55fdcb1ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f525e7fc700 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1efcc0 0x55fdcb1ec820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f68d29e181a766 server_cookie=dce36ed93b7aca70 in_seq=0 out_seq=0 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.511+0000 7f523effd700 10 client.25231 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f523effd700 1 -- 172.21.15.26:0/475165271 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fdcb08f000 con 0x55fdcb1efcc0 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f523effd700 10 client.25231 handle_command_reply: tid=0 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 client.25231 shutdown 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 msgr2=0x55fdcb18e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 0x55fdcb18e800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f525009f270 tx=0x7f5250016470 comp rx=0 tx=0).stop 2023-07-13T22:13:53.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 shutdown_connections 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fdcb183c20 0x55fdcb1eb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fdcb1efcc0 0x55fdcb1ec820 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fdcb182480 0x55fdcb18e800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 --2- 172.21.15.26:0/475165271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fdcb182dd0 0x55fdcb18ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f523effd700 0 client.25231 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 >> 172.21.15.26:0/475165271 conn(0x55fdcb177d50 msgr2=0x55fdcb1966e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 shutdown_connections 2023-07-13T22:13:53.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 1 -- 172.21.15.26:0/475165271 wait complete. 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.512+0000 7f5279288b80 20 client.25231 trim_cache size 0 max 16384 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "23b26990-21c9-11ee-9b32-001a4aab830c", 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 24575, 2023-07-13T22:13:53.511 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 94.021058420000003, 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 27, 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 85, 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 83, 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 101.89228554899999 2023-07-13T22:13:53.512 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:13:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 msgr2=0x7f52480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 0x7f52480a3600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5258009ec0 tx=0x7f525800b040 comp rx=0 tx=0).stop 2023-07-13T22:13:53.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 msgr2=0x7f5260197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f525000ec70 tx=0x7f525000ef80 comp rx=0 tx=0).stop 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 shutdown_connections 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52601079a0 0x7f526019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5260072f70 0x7f5260197b30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f52480a1150 0x7f52480a3600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 --2- 172.21.15.26:0/2635170155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5260074550 0x7f5260198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:53.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 >> 172.21.15.26:0/2635170155 conn(0x7f52600fb0e0 msgr2=0x7f52600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 shutdown_connections 2023-07-13T22:13:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:53.513+0000 7f52659ff700 1 -- 172.21.15.26:0/2635170155 wait complete. 2023-07-13T22:13:53.521 INFO:teuthology.run_tasks:Running task kclient... 2023-07-13T22:13:53.545 INFO:tasks.kclient:Mounting kernel clients... 2023-07-13T22:13:53.545 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-13T22:13:53.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:53.546 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-07-13T22:13:53.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:53.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-07-13T22:13:53.704 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-07-13T22:13:53.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:13:53.708 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-07-13T22:13:53.708 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-13T22:13:53.708 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:13:53.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph fs ls 2023-07-13T22:13:54.440 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:13:54.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:54 smithi117 ceph-mon[129074]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-13T22:13:54.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:54 smithi026 ceph-mon[95647]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-13T22:13:54.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:54 smithi121 ceph-mon[129187]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2023-07-13T22:13:55.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.928+0000 7fdd55877700 1 -- 172.21.15.26:0/429507253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd500ffe00 msgr2=0x7fdd501001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.928+0000 7fdd55877700 1 --2- 172.21.15.26:0/429507253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd500ffe00 0x7fdd501001d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd400099c0 tx=0x7fdd40009cd0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 -- 172.21.15.26:0/429507253 shutdown_connections 2023-07-13T22:13:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 --2- 172.21.15.26:0/429507253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd500ffe00 0x7fdd501001d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 --2- 172.21.15.26:0/429507253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd50104ea0 0x7fdd5010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 --2- 172.21.15.26:0/429507253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 0x7fdd50104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 -- 172.21.15.26:0/429507253 >> 172.21.15.26:0/429507253 conn(0x7fdd500fb660 msgr2=0x7fdd500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:55.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 -- 172.21.15.26:0/429507253 shutdown_connections 2023-07-13T22:13:55.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.929+0000 7fdd55877700 1 -- 172.21.15.26:0/429507253 wait complete. 2023-07-13T22:13:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 Processor -- start 2023-07-13T22:13:55.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 -- start start 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 0x7fdd50196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd50104ea0 0x7fdd50199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50199b50 con 0x7fdd50104ea0 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50199cc0 con 0x7fdd500ffe00 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd55877700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50199e80 con 0x7fdd501007a0 2023-07-13T22:13:55.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.930+0000 7fdd4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34206/0 (socket says 172.21.15.26:34206) 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 -- 172.21.15.26:0/3243005343 learned_addr learned my addr 172.21.15.26:0/3243005343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4f7fe700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd50104ea0 0x7fdd50199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4e7fc700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 0x7fdd50196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 -- 172.21.15.26:0/3243005343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 msgr2=0x7fdd50196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 0x7fdd50196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 -- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd50104ea0 msgr2=0x7fdd50199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd50104ea0 0x7fdd50199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 -- 172.21.15.26:0/3243005343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400095f0 con 0x7fdd500ffe00 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd4effd700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b580 tx=0x7fdd4000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.931+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4001d070 con 0x7fdd500ffe00 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.932+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd40005050 con 0x7fdd500ffe00 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.932+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5019a100 con 0x7fdd500ffe00 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.932+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd40004a40 con 0x7fdd500ffe00 2023-07-13T22:13:55.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.932+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd501a2ab0 con 0x7fdd500ffe00 2023-07-13T22:13:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.933+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5004e210 con 0x7fdd500ffe00 2023-07-13T22:13:55.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.935+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdd40005430 con 0x7fdd500ffe00 2023-07-13T22:13:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.936+0000 7fdd54875700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 0x7fdd380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:13:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.936+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd400cd6b0 con 0x7fdd500ffe00 2023-07-13T22:13:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.936+0000 7fdd4e7fc700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 0x7fdd380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:13:55.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.937+0000 7fdd4e7fc700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 0x7fdd380a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd50197230 tx=0x7fdd440073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:13:55.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:55.938+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdd40096e40 con 0x7fdd500ffe00 2023-07-13T22:13:56.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.188+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdd501a2c40 con 0x7fdd500ffe00 2023-07-13T22:13:56.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.189+0000 7fdd54875700 1 -- 172.21.15.26:0/3243005343 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdd4009a460 con 0x7fdd500ffe00 2023-07-13T22:13:56.179 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-13T22:13:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 msgr2=0x7fdd380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 0x7fdd380a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd50197230 tx=0x7fdd440073d0 comp rx=0 tx=0).stop 2023-07-13T22:13:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 msgr2=0x7fdd50195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:13:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000b580 tx=0x7fdd4000f690 comp rx=0 tx=0).stop 2023-07-13T22:13:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 shutdown_connections 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdd380a11d0 0x7fdd380a3680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd50104ea0 0x7fdd50199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd500ffe00 0x7fdd50195cc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 --2- 172.21.15.26:0/3243005343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd501007a0 0x7fdd50196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 >> 172.21.15.26:0/3243005343 conn(0x7fdd500fb660 msgr2=0x7fdd500fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 shutdown_connections 2023-07-13T22:13:56.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:13:56.191+0000 7fdd55877700 1 -- 172.21.15.26:0/3243005343 wait complete. 2023-07-13T22:13:56.626 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-13T22:13:56.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-13T22:13:56.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-07-13T22:13:56.627 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-13T22:13:56.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-13T22:13:56.627 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-13T22:13:56.627 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-13T22:13:56.628 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-13T22:13:56.628 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:13:56.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:56.628 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-07-13T22:13:56.673 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-13T22:13:56.673 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-07-13T22:13:56.674 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-13T22:13:56.675 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41613sec preferred_lft 41613sec 2023-07-13T22:13:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-07-13T22:13:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-07-13T22:13:56.676 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-13T22:13:56.676 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-07-13T22:13:56.677 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-07-13T22:13:56.678 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-13T22:13:56.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:56.678 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-07-13T22:13:56.678 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-07-13T22:13:56.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2023-07-13T22:13:56.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2023-07-13T22:13:56.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2023-07-13T22:13:56.679 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-13T22:13:56.680 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-07-13T22:13:56.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:56 smithi026 ceph-mon[95647]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:13:56.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:56 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3243005343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-13T22:13:56.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:56 smithi121 ceph-mon[129187]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:13:56.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:56 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3243005343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-13T22:13:57.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:57.153 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-13T22:13:57.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:56 smithi117 ceph-mon[129074]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:13:57.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:56 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3243005343' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-13T22:13:57.222 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-07-13T22:13:57.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:57.226 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-07-13T22:13:57.288 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-07-13T22:13:57.288 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-13T22:13:57.288 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-13T22:13:57.288 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-13T22:13:57.289 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-13T22:13:57.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:57.290 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-07-13T22:13:57.290 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-07-13T22:13:57.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-13T22:13:57.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-13T22:13:57.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-13T22:13:57.291 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-07-13T22:13:58.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.078 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-07-13T22:13:58.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.123 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2023-07-13T22:13:58.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.207 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-07-13T22:13:58.207 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-07-13T22:13:58.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:13:58.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-13T22:13:58.208 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-07-13T22:13:58.394 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-13T22:13:58.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.395 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-07-13T22:13:58.395 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-07-13T22:13:58.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-13T22:13:58.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-13T22:13:58.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-13T22:13:58.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-13T22:13:58.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-13T22:13:58.397 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-07-13T22:13:58.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.680 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-07-13T22:13:58.680 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-07-13T22:13:58.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-07-13T22:13:58.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2023-07-13T22:13:58.681 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-07-13T22:13:58.855 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:13:58 smithi026 ceph-mon[95647]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-13T22:13:58.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:13:58.860 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-13T22:13:58.903 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-13T22:13:58.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-13T22:13:58.903 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:13:58.985 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-13T22:13:58.986 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-13T22:13:58.986 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/304bc8c3-9709-4ccf-9b77-737ad5afecf4 2023-07-13T22:13:58.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:58.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/304bc8c3-9709-4ccf-9b77-737ad5afecf4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-07-13T22:13:58.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:13:58 smithi121 ceph-mon[129187]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-13T22:13:59.141 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-07-13T22:13:59.141 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-07-13T22:13:59.141 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2023-07-13T22:13:59.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:13:58 smithi117 ceph-mon[129074]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2023-07-13T22:13:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-07-13T22:13:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-07-13T22:13:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:invalid new device string format 2023-07-13T22:13:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.117:3300,172.21.15.121:3300" 2023-07-13T22:13:59.632 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.26:3300,172.21.15.117:3300,172.21.15.121:3300:/volumes/_nogroup/sv_1/304bc8c3-9709-4ccf-9b77-737ad5afecf4 2023-07-13T22:13:59.633 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=23b26990-21c9-11ee-9b32-001a4aab830c" will pass to kernel 2023-07-13T22:13:59.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-13T22:13:59.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:59.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-13T22:13:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:59.718 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-13T22:13:59.811 INFO:teuthology.orchestra.run.smithi026.stdout:client25240 2023-07-13T22:13:59.811 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:13:59.812 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:13:59.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:59.857 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-13T22:13:59.983 INFO:teuthology.orchestra.run.smithi026.stdout:client25240 2023-07-13T22:13:59.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:13:59.984 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-13T22:14:00.071 INFO:teuthology.orchestra.run.smithi026.stdout:client25240 2023-07-13T22:14:00.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:14:00.072 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/23b26990-21c9-11ee-9b32-001a4aab830c.client25240/status') 2023-07-13T22:14:00.180 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.25240 (3)172.21.15.26:0/2781527281 2023-07-13T22:14:00.181 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2023-07-13T22:14:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2023-07-13T22:14:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2023-07-13T22:14:00.182 INFO:teuthology.orchestra.run.smithi026.stderr:71 bytes copied, 5.5732e-05 s, 1.3 MB/s 2023-07-13T22:14:00.184 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-13T22:14:00.194 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-13T22:14:00.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:14:00.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 -- 172.21.15.26:0/3995386514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc0ffb00 msgr2=0x7f50fc0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 --2- 172.21.15.26:0/3995386514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc0ffb00 0x7f50fc0fff10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009990 tx=0x7f50f8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 -- 172.21.15.26:0/3995386514 shutdown_connections 2023-07-13T22:14:00.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 --2- 172.21.15.26:0/3995386514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc0ffb00 0x7f50fc0fff10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 --2- 172.21.15.26:0/3995386514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc101680 0x7f50fc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 --2- 172.21.15.26:0/3995386514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc100cf0 0x7f50fc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 -- 172.21.15.26:0/3995386514 >> 172.21.15.26:0/3995386514 conn(0x7f50fc0fb0e0 msgr2=0x7f50fc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 -- 172.21.15.26:0/3995386514 shutdown_connections 2023-07-13T22:14:00.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.444+0000 7f51033ff700 1 -- 172.21.15.26:0/3995386514 wait complete. 2023-07-13T22:14:00.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.445+0000 7f51033ff700 1 Processor -- start 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.445+0000 7f51033ff700 1 -- start start 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc0ffb00 0x7f50fc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 0x7f50fc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc19f5a0 con 0x7f50fc100cf0 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc19f6e0 con 0x7f50fc0ffb00 2023-07-13T22:14:00.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f51033ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc19f870 con 0x7f50fc101680 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45530/0 (socket says 172.21.15.26:45530) 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 -- 172.21.15.26:0/138320103 learned_addr learned my addr 172.21.15.26:0/138320103 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510119b700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc0ffb00 0x7f50fc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510199c700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 0x7f50fc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 -- 172.21.15.26:0/138320103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 msgr2=0x7f50fc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 0x7f50fc19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 -- 172.21.15.26:0/138320103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc0ffb00 msgr2=0x7f50fc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.446+0000 7f510099a700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc0ffb00 0x7f50fc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f510099a700 1 -- 172.21.15.26:0/138320103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec009520 con 0x7f50fc100cf0 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f510199c700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 0x7f50fc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f510099a700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00eb30 tx=0x7f50ec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec009880 con 0x7f50fc100cf0 2023-07-13T22:14:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50ec0044e0 con 0x7f50fc100cf0 2023-07-13T22:14:00.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f80095f0 con 0x7f50fc100cf0 2023-07-13T22:14:00.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003e90 con 0x7f50fc100cf0 2023-07-13T22:14:00.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.447+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc19fed0 con 0x7f50fc100cf0 2023-07-13T22:14:00.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.448+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc1921e0 con 0x7f50fc100cf0 2023-07-13T22:14:00.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.450+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f50ec004000 con 0x7f50fc100cf0 2023-07-13T22:14:00.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.452+0000 7f50f27fc700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 0x7f50e80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.452+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50ec014070 con 0x7f50fc100cf0 2023-07-13T22:14:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.452+0000 7f510119b700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 0x7f50e80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.453+0000 7f510119b700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 0x7f50e80a3650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.454+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f50ec096aa0 con 0x7f50fc100cf0 2023-07-13T22:14:00.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.669+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50fc02c850 con 0x7f50fc100cf0 2023-07-13T22:14:00.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.670+0000 7f50f27fc700 1 -- 172.21.15.26:0/138320103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f50ec09a0c0 con 0x7f50fc100cf0 2023-07-13T22:14:00.661 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:00.661 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:14:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.672+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 msgr2=0x7f50e80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.672+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 0x7f50e80a3650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50f800b2e0 tx=0x7f50f8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 msgr2=0x7f50fc197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00eb30 tx=0x7f50ec00ee40 comp rx=0 tx=0).stop 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 shutdown_connections 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50fc101680 0x7f50fc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50fc0ffb00 0x7f50fc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f50e80a11a0 0x7f50e80a3650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 --2- 172.21.15.26:0/138320103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50fc100cf0 0x7f50fc197f60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 >> 172.21.15.26:0/138320103 conn(0x7f50fc0fb0e0 msgr2=0x7f50fc1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:00.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 shutdown_connections 2023-07-13T22:14:00.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.673+0000 7f51033ff700 1 -- 172.21.15.26:0/138320103 wait complete. 2023-07-13T22:14:00.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:14:00.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:14:00.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 -- 172.21.15.26:0/880852121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 msgr2=0x7f0724073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 --2- 172.21.15.26:0/880852121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724073380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0714009990 tx=0x7f0714009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 -- 172.21.15.26:0/880852121 shutdown_connections 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 --2- 172.21.15.26:0/880852121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0724107980 0x7f0724109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 --2- 172.21.15.26:0/880852121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07240744f0 0x7f0724072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 --2- 172.21.15.26:0/880852121 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724073380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 -- 172.21.15.26:0/880852121 >> 172.21.15.26:0/880852121 conn(0x7f07240fb130 msgr2=0x7f07240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 -- 172.21.15.26:0/880852121 shutdown_connections 2023-07-13T22:14:00.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.901+0000 7f072bfff700 1 -- 172.21.15.26:0/880852121 wait complete. 2023-07-13T22:14:00.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 Processor -- start 2023-07-13T22:14:00.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 -- start start 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 0x7f072419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072419f5a0 con 0x7f07240744f0 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072419f6e0 con 0x7f0724107980 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.902+0000 7f072bfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072419f870 con 0x7f0724072f10 2023-07-13T22:14:00.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45548/0 (socket says 172.21.15.26:45548) 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 -- 172.21.15.26:0/1320871000 learned_addr learned my addr 172.21.15.26:0/1320871000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f0729d9b700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072a59c700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 0x7f072419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 -- 172.21.15.26:0/1320871000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 msgr2=0x7f0724197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 -- 172.21.15.26:0/1320871000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 msgr2=0x7f072419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:00.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 0x7f072419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 -- 172.21.15.26:0/1320871000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07140095f0 con 0x7f07240744f0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072a59c700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 0x7f072419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.903+0000 7f072959a700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f071400b680 tx=0x7f0714005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.904+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071401d070 con 0x7f07240744f0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.904+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0714022470 con 0x7f07240744f0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.904+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072419faf0 con 0x7f07240744f0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.904+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714017650 con 0x7f07240744f0 2023-07-13T22:14:00.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.904+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072419ff70 con 0x7f07240744f0 2023-07-13T22:14:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.905+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07241060c0 con 0x7f07240744f0 2023-07-13T22:14:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.907+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f07140225e0 con 0x7f07240744f0 2023-07-13T22:14:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.908+0000 7f071affd700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 0x7f07100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.908+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f07140cddc0 con 0x7f07240744f0 2023-07-13T22:14:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.908+0000 7f0729d9b700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 0x7f07100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.909+0000 7f0729d9b700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 0x7f07100a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0720009c60 tx=0x7f0720009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:00.912+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0714097550 con 0x7f07240744f0 2023-07-13T22:14:00.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:00 smithi026 ceph-mon[95647]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-13T22:14:00.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:00 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/138320103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:00.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:00 smithi121 ceph-mon[129187]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-13T22:14:00.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:00 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/138320103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.119+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0724061120 con 0x7f07240744f0 2023-07-13T22:14:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.120+0000 7f071affd700 1 -- 172.21.15.26:0/1320871000 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f071409ab70 con 0x7f07240744f0 2023-07-13T22:14:01.110 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:14:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 msgr2=0x7f07100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 0x7f07100a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0720009c60 tx=0x7f0720009260 comp rx=0 tx=0).stop 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 msgr2=0x7f0724197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f071400b680 tx=0x7f0714005740 comp rx=0 tx=0).stop 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 shutdown_connections 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0724107980 0x7f072419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0724072f10 0x7f0724197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f07100a11d0 0x7f07100a3680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 --2- 172.21.15.26:0/1320871000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07240744f0 0x7f0724197f60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 >> 172.21.15.26:0/1320871000 conn(0x7f07240fb130 msgr2=0x7f07240fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 shutdown_connections 2023-07-13T22:14:01.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.122+0000 7f072bfff700 1 -- 172.21.15.26:0/1320871000 wait complete. 2023-07-13T22:14:01.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:14:01.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:00 smithi117 ceph-mon[129074]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 10 op/s 2023-07-13T22:14:01.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:00 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/138320103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 -- 172.21.15.26:0/2047938463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0100560 msgr2=0x7fd3a0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/2047938463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0100560 0x7fd3a0102940 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009990 tx=0x7fd39c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 -- 172.21.15.26:0/2047938463 shutdown_connections 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/2047938463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/2047938463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0100560 0x7fd3a0102940 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/2047938463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0102e80 0x7fd3a0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 -- 172.21.15.26:0/2047938463 >> 172.21.15.26:0/2047938463 conn(0x7fd3a00fa000 msgr2=0x7fd3a00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 -- 172.21.15.26:0/2047938463 shutdown_connections 2023-07-13T22:14:01.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.348+0000 7fd3a81ff700 1 -- 172.21.15.26:0/2047938463 wait complete. 2023-07-13T22:14:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.349+0000 7fd3a81ff700 1 Processor -- start 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.349+0000 7fd3a81ff700 1 -- start start 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0100560 0x7fd3a00ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0102e80 0x7fd3a00fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00ff1f0 con 0x7fd3a01057a0 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00ff360 con 0x7fd3a0102e80 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a81ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a019f8c0 con 0x7fd3a0100560 2023-07-13T22:14:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45564/0 (socket says 172.21.15.26:45564) 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 -- 172.21.15.26:0/4040445130 learned_addr learned my addr 172.21.15.26:0/4040445130 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a5f9b700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0100560 0x7fd3a00ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a579a700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0102e80 0x7fd3a00fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 -- 172.21.15.26:0/4040445130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0100560 msgr2=0x7fd3a00ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0100560 0x7fd3a00ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 -- 172.21.15.26:0/4040445130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0102e80 msgr2=0x7fd3a00fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0102e80 0x7fd3a00fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.350+0000 7fd3a679c700 1 -- 172.21.15.26:0/4040445130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009c40 con 0x7fd3a01057a0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd3a679c700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c370 tx=0x7fd39800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398017070 con 0x7fd3a01057a0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c0095f0 con 0x7fd3a01057a0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd39800f040 con 0x7fd3a01057a0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e050 con 0x7fd3a01057a0 2023-07-13T22:14:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.351+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a019fec0 con 0x7fd3a01057a0 2023-07-13T22:14:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.352+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a00fbb70 con 0x7fd3a01057a0 2023-07-13T22:14:01.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.354+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd39800cb60 con 0x7fd3a01057a0 2023-07-13T22:14:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.356+0000 7fd396ffd700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 0x7fd38c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.356+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd3980cc280 con 0x7fd3a01057a0 2023-07-13T22:14:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.356+0000 7fd3a5f9b700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 0x7fd38c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.357+0000 7fd3a5f9b700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 0x7fd38c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00b2e0 tx=0x7fd39c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.360+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd398095980 con 0x7fd3a01057a0 2023-07-13T22:14:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.573+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3a0061120 con 0x7fd3a01057a0 2023-07-13T22:14:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.573+0000 7fd396ffd700 1 -- 172.21.15.26:0/4040445130 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd398098fa0 con 0x7fd3a01057a0 2023-07-13T22:14:01.564 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:14:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 msgr2=0x7fd38c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 0x7fd38c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd39c00b2e0 tx=0x7fd39c005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 msgr2=0x7fd3a00fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd39800c370 tx=0x7fd39800c680 comp rx=0 tx=0).stop 2023-07-13T22:14:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 shutdown_connections 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd38c0a1150 0x7fd38c0a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3a01057a0 0x7fd3a00fea50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3a0100560 0x7fd3a00ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 --2- 172.21.15.26:0/4040445130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a0102e80 0x7fd3a00fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.575+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 >> 172.21.15.26:0/4040445130 conn(0x7fd3a00fa000 msgr2=0x7fd3a01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.576+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 shutdown_connections 2023-07-13T22:14:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.576+0000 7fd3a81ff700 1 -- 172.21.15.26:0/4040445130 wait complete. 2023-07-13T22:14:01.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:14:01.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 -- 172.21.15.26:0/1507641410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c101660 msgr2=0x7fa10c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1507641410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c101660 0x7fa10c107f20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa10400b0a0 tx=0x7fa10400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 -- 172.21.15.26:0/1507641410 shutdown_connections 2023-07-13T22:14:01.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1507641410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c101660 0x7fa10c107f20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1507641410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1507641410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 0x7fa10c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 -- 172.21.15.26:0/1507641410 >> 172.21.15.26:0/1507641410 conn(0x7fa10c0fb0e0 msgr2=0x7fa10c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:01.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.805+0000 7fa1147ff700 1 -- 172.21.15.26:0/1507641410 shutdown_connections 2023-07-13T22:14:01.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.806+0000 7fa1147ff700 1 -- 172.21.15.26:0/1507641410 wait complete. 2023-07-13T22:14:01.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.807+0000 7fa1147ff700 1 Processor -- start 2023-07-13T22:14:01.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.807+0000 7fa1147ff700 1 -- start start 2023-07-13T22:14:01.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 0x7fa10c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c101660 0x7fa10c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0735e0 con 0x7fa10c0ffae0 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c073750 con 0x7fa10c100cd0 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa1147ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c19f9b0 con 0x7fa10c101660 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45592/0 (socket says 172.21.15.26:45592) 2023-07-13T22:14:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 -- 172.21.15.26:0/1196526973 learned_addr learned my addr 172.21.15.26:0/1196526973 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:01.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa111d9a700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 0x7fa10c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa112d9c700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c101660 0x7fa10c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 -- 172.21.15.26:0/1196526973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c101660 msgr2=0x7fa10c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c101660 0x7fa10c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 -- 172.21.15.26:0/1196526973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 msgr2=0x7fa10c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 0x7fa10c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.808+0000 7fa11259b700 1 -- 172.21.15.26:0/1196526973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104009d00 con 0x7fa10c0ffae0 2023-07-13T22:14:01.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa11259b700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa10800e970 tx=0x7fa10800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108009790 con 0x7fa10c0ffae0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10800c7e0 con 0x7fa10c0ffae0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa108003c70 con 0x7fa10c0ffae0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c19fc90 con 0x7fa10c0ffae0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.809+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c1a0120 con 0x7fa10c0ffae0 2023-07-13T22:14:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.810+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c107000 con 0x7fa10c0ffae0 2023-07-13T22:14:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.812+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa10800cc50 con 0x7fa10c0ffae0 2023-07-13T22:14:01.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.814+0000 7fa1037fe700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 0x7fa0f80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:01.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.815+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa108014070 con 0x7fa10c0ffae0 2023-07-13T22:14:01.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.815+0000 7fa111d9a700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 0x7fa0f80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:01.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.815+0000 7fa111d9a700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 0x7fa0f80a3680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009c60 tx=0x7fa0fc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:01.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:01.819+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa108096170 con 0x7fa10c0ffae0 2023-07-13T22:14:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:01 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1320871000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:01.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:01 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4040445130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:01.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:01 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1320871000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:01.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:01 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4040445130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:02.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.030+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa10c061270 con 0x7fa10c0ffae0 2023-07-13T22:14:02.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.030+0000 7fa1037fe700 1 -- 172.21.15.26:0/1196526973 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa108003dd0 con 0x7fa10c0ffae0 2023-07-13T22:14:02.021 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:14:02.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.032+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 msgr2=0x7fa0f80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.032+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 0x7fa0f80a3680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009c60 tx=0x7fa0fc009260 comp rx=0 tx=0).stop 2023-07-13T22:14:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.032+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 msgr2=0x7fa10c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.032+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa10800e970 tx=0x7fa10800ec80 comp rx=0 tx=0).stop 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 shutdown_connections 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa0f80a11d0 0x7fa0f80a3680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa10c0ffae0 0x7fa10c0742c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa10c101660 0x7fa10c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 --2- 172.21.15.26:0/1196526973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa10c100cd0 0x7fa10c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 >> 172.21.15.26:0/1196526973 conn(0x7fa10c0fb0e0 msgr2=0x7fa10c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:02.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 shutdown_connections 2023-07-13T22:14:02.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.033+0000 7fa1147ff700 1 -- 172.21.15.26:0/1196526973 wait complete. 2023-07-13T22:14:02.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:14:02.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:01 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1320871000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:02.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:01 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4040445130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 -- 172.21.15.26:0/3402574553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe80ffb00 msgr2=0x7f5fe80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 --2- 172.21.15.26:0/3402574553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe80ffb00 0x7f5fe80fff10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009990 tx=0x7f5fdc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 -- 172.21.15.26:0/3402574553 shutdown_connections 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 --2- 172.21.15.26:0/3402574553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe8101680 0x7f5fe8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 --2- 172.21.15.26:0/3402574553 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe80ffb00 0x7f5fe80fff10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 --2- 172.21.15.26:0/3402574553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 -- 172.21.15.26:0/3402574553 >> 172.21.15.26:0/3402574553 conn(0x7f5fe80fb0e0 msgr2=0x7f5fe80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 -- 172.21.15.26:0/3402574553 shutdown_connections 2023-07-13T22:14:02.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.255+0000 7f5feedff700 1 -- 172.21.15.26:0/3402574553 wait complete. 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.256+0000 7f5feedff700 1 Processor -- start 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.256+0000 7f5feedff700 1 -- start start 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.256+0000 7f5feedff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.256+0000 7f5feedff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5feedff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8101680 0x7f5fe819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5feedff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819f5d0 con 0x7f5fe8101680 2023-07-13T22:14:02.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5feedff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819f710 con 0x7f5fe80ffb00 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5feedff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819f8a0 con 0x7f5fe8100cf0 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fed39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8101680 0x7f5fe819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34286/0 (socket says 172.21.15.26:34286) 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 -- 172.21.15.26:0/1886680850 learned_addr learned my addr 172.21.15.26:0/1886680850 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fe7fff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 -- 172.21.15.26:0/1886680850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 msgr2=0x7f5fe8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 -- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8101680 msgr2=0x7f5fe819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8101680 0x7f5fe819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 -- 172.21.15.26:0/1886680850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0095f0 con 0x7f5fe80ffb00 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fe7fff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:02.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.257+0000 7f5fecb9b700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005720 tx=0x7f5fdc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.258+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc01d070 con 0x7f5fe80ffb00 2023-07-13T22:14:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.258+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe819fb20 con 0x7f5fe80ffb00 2023-07-13T22:14:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.258+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe819fff0 con 0x7f5fe80ffb00 2023-07-13T22:14:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.258+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fdc021410 con 0x7f5fe80ffb00 2023-07-13T22:14:02.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.258+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc017650 con 0x7f5fe80ffb00 2023-07-13T22:14:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.259+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8192180 con 0x7f5fe80ffb00 2023-07-13T22:14:02.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.261+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5fdc00fac0 con 0x7f5fe80ffb00 2023-07-13T22:14:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.263+0000 7f5fe5ffb700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 0x7f5fd80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.263+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fdc0cdfd0 con 0x7f5fe80ffb00 2023-07-13T22:14:02.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.263+0000 7f5fe7fff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 0x7f5fd80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.263+0000 7f5fe7fff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 0x7f5fd80a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009cb0 tx=0x7f5fd4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:02.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.264+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5fdc097760 con 0x7f5fe80ffb00 2023-07-13T22:14:02.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.476+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fe802c9d0 con 0x7f5fe80ffb00 2023-07-13T22:14:02.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.478+0000 7f5fe5ffb700 1 -- 172.21.15.26:0/1886680850 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f5fdc09ad80 con 0x7f5fe80ffb00 2023-07-13T22:14:02.469 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:02.469 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:14:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.480+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 msgr2=0x7f5fd80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.480+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 0x7f5fd80a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009cb0 tx=0x7f5fd4009260 comp rx=0 tx=0).stop 2023-07-13T22:14:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.480+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 msgr2=0x7f5fe8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.480+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc005720 tx=0x7f5fdc009580 comp rx=0 tx=0).stop 2023-07-13T22:14:02.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 shutdown_connections 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5fd80a11a0 0x7f5fd80a3650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fe8101680 0x7f5fe819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe80ffb00 0x7f5fe8197a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 --2- 172.21.15.26:0/1886680850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fe8100cf0 0x7f5fe8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 >> 172.21.15.26:0/1886680850 conn(0x7f5fe80fb0e0 msgr2=0x7f5fe81065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 shutdown_connections 2023-07-13T22:14:02.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.481+0000 7f5feedff700 1 -- 172.21.15.26:0/1886680850 wait complete. 2023-07-13T22:14:02.489 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:14:02.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:14:02.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 -- 172.21.15.26:0/3938135745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798102e80 msgr2=0x7f4798105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 --2- 172.21.15.26:0/3938135745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798102e80 0x7f4798105260 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f478c009990 tx=0x7f478c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 -- 172.21.15.26:0/3938135745 shutdown_connections 2023-07-13T22:14:02.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 --2- 172.21.15.26:0/3938135745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f47981057a0 0x7f4798107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 --2- 172.21.15.26:0/3938135745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4798100560 0x7f4798102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 --2- 172.21.15.26:0/3938135745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798102e80 0x7f4798105260 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 -- 172.21.15.26:0/3938135745 >> 172.21.15.26:0/3938135745 conn(0x7f47980fa000 msgr2=0x7f47980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 -- 172.21.15.26:0/3938135745 shutdown_connections 2023-07-13T22:14:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.706+0000 7f479dfff700 1 -- 172.21.15.26:0/3938135745 wait complete. 2023-07-13T22:14:02.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.707+0000 7f479dfff700 1 Processor -- start 2023-07-13T22:14:02.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 -- start start 2023-07-13T22:14:02.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798102e80 0x7f4798197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47981057a0 0x7f479819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479819f5a0 con 0x7f4798100560 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479819f6e0 con 0x7f47981057a0 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f479dfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479819f870 con 0x7f4798102e80 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f47977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f47977fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45630/0 (socket says 172.21.15.26:45630) 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f47977fe700 1 -- 172.21.15.26:0/3014336124 learned_addr learned my addr 172.21.15.26:0/3014336124 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:02.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f4796ffd700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798102e80 0x7f4798197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.708+0000 7f4797fff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47981057a0 0x7f479819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 -- 172.21.15.26:0/3014336124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798102e80 msgr2=0x7f4798197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798102e80 0x7f4798197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 -- 172.21.15.26:0/3014336124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47981057a0 msgr2=0x7f479819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47981057a0 0x7f479819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 -- 172.21.15.26:0/3014336124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c0095f0 con 0x7f4798100560 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f47977fe700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f478800e9a0 tx=0x7f478800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:02.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788009790 con 0x7f4798100560 2023-07-13T22:14:02.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f478800cbd0 con 0x7f4798100560 2023-07-13T22:14:02.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479819fb50 con 0x7f4798100560 2023-07-13T22:14:02.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788018680 con 0x7f4798100560 2023-07-13T22:14:02.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.709+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47981a00b0 con 0x7f4798100560 2023-07-13T22:14:02.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.710+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798107360 con 0x7f4798100560 2023-07-13T22:14:02.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.712+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f478800cd40 con 0x7f4798100560 2023-07-13T22:14:02.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.715+0000 7f4794ff9700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 0x7f47800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:02.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.715+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4788014070 con 0x7f4798100560 2023-07-13T22:14:02.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.715+0000 7f4796ffd700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 0x7f47800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:02.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.715+0000 7f4796ffd700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 0x7f47800a3680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b680 tx=0x7f478c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:02.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.718+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4788051790 con 0x7f4798100560 2023-07-13T22:14:02.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.928+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47980611c0 con 0x7f4798100560 2023-07-13T22:14:02.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.929+0000 7f4794ff9700 1 -- 172.21.15.26:0/3014336124 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4788096280 con 0x7f4798100560 2023-07-13T22:14:02.919 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:14:02.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 msgr2=0x7f47800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 0x7f47800a3680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b680 tx=0x7f478c005fd0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 msgr2=0x7f4798197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f478800e9a0 tx=0x7f478800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 shutdown_connections 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f47800a11d0 0x7f47800a3680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4798100560 0x7f4798197a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47981057a0 0x7f479819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 --2- 172.21.15.26:0/3014336124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4798102e80 0x7f4798197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:02.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 >> 172.21.15.26:0/3014336124 conn(0x7f47980fa000 msgr2=0x7f47980febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:02.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 shutdown_connections 2023-07-13T22:14:02.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:02.931+0000 7f479dfff700 1 -- 172.21.15.26:0/3014336124 wait complete. 2023-07-13T22:14:02.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:02 smithi026 ceph-mon[95647]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:14:02.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:02 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1196526973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:02.938 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:02 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1886680850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:02.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:14:02.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:02 smithi121 ceph-mon[129187]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:14:02.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:02 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1196526973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:02.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:02 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1886680850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:03.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 -- 172.21.15.26:0/574415400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc0ffaf0 msgr2=0x7f76dc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 --2- 172.21.15.26:0/574415400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc0fff00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009990 tx=0x7f76cc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 -- 172.21.15.26:0/574415400 shutdown_connections 2023-07-13T22:14:03.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 --2- 172.21.15.26:0/574415400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc0fff00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 --2- 172.21.15.26:0/574415400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc101670 0x7f76dc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 --2- 172.21.15.26:0/574415400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 0x7f76dc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.159+0000 7f76e1dff700 1 -- 172.21.15.26:0/574415400 >> 172.21.15.26:0/574415400 conn(0x7f76dc0fb130 msgr2=0x7f76dc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.160+0000 7f76e1dff700 1 -- 172.21.15.26:0/574415400 shutdown_connections 2023-07-13T22:14:03.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.160+0000 7f76e1dff700 1 -- 172.21.15.26:0/574415400 wait complete. 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 Processor -- start 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 -- start start 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 0x7f76dc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc19f5d0 con 0x7f76dc101670 2023-07-13T22:14:03.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc19f710 con 0x7f76dc100ce0 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76e1dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc19f8a0 con 0x7f76dc0ffaf0 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45176/0 (socket says 172.21.15.26:45176) 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76dbfff700 1 -- 172.21.15.26:0/1840730561 learned_addr learned my addr 172.21.15.26:0/1840730561 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.161+0000 7f76db7fe700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76daffd700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 0x7f76dc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 -- 172.21.15.26:0/1840730561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc0ffaf0 msgr2=0x7f76dc197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 -- 172.21.15.26:0/1840730561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 msgr2=0x7f76dc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 0x7f76dc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 -- 172.21.15.26:0/1840730561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0095f0 con 0x7f76dc101670 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76dbfff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ef30 tx=0x7f76d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4018070 con 0x7f76dc101670 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76d400f040 con 0x7f76dc101670 2023-07-13T22:14:03.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc19fb20 con 0x7f76dc101670 2023-07-13T22:14:03.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d400cd00 con 0x7f76dc101670 2023-07-13T22:14:03.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.162+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc1a0030 con 0x7f76dc101670 2023-07-13T22:14:03.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.163+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc192180 con 0x7f76dc101670 2023-07-13T22:14:03.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.165+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f76d4003a00 con 0x7f76dc101670 2023-07-13T22:14:03.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.168+0000 7f76d8ff9700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 0x7f76c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.168+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f76d40cc440 con 0x7f76dc101670 2023-07-13T22:14:03.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.168+0000 7f76db7fe700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 0x7f76c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.168+0000 7f76db7fe700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 0x7f76c40a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00b2e0 tx=0x7f76cc005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:03.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.171+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f76d4095bd0 con 0x7f76dc101670 2023-07-13T22:14:03.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:02 smithi117 ceph-mon[129074]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2023-07-13T22:14:03.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:02 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1196526973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:03.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:02 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1886680850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:03.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.382+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76dc0611c0 con 0x7f76dc101670 2023-07-13T22:14:03.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.382+0000 7f76d8ff9700 1 -- 172.21.15.26:0/1840730561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76d40991f0 con 0x7f76dc101670 2023-07-13T22:14:03.373 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:14:03.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 msgr2=0x7f76c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 0x7f76c40a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00b2e0 tx=0x7f76cc005960 comp rx=0 tx=0).stop 2023-07-13T22:14:03.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 msgr2=0x7f76dc19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ef30 tx=0x7f76d400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 shutdown_connections 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f76c40a1150 0x7f76c40a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76dc101670 0x7f76dc19f000 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76dc0ffaf0 0x7f76dc197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 --2- 172.21.15.26:0/1840730561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76dc100ce0 0x7f76dc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 >> 172.21.15.26:0/1840730561 conn(0x7f76dc0fb130 msgr2=0x7f76dc106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:03.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 shutdown_connections 2023-07-13T22:14:03.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.385+0000 7f76e1dff700 1 -- 172.21.15.26:0/1840730561 wait complete. 2023-07-13T22:14:03.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:14:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 -- 172.21.15.26:0/1722789448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d40ffae0 msgr2=0x7f01d40ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1722789448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d40ffae0 0x7f01d40ffef0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009990 tx=0x7f01c4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 -- 172.21.15.26:0/1722789448 shutdown_connections 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1722789448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d40ffae0 0x7f01d40ffef0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1722789448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 0x7f01d4107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1722789448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d4100cd0 0x7f01d4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 -- 172.21.15.26:0/1722789448 >> 172.21.15.26:0/1722789448 conn(0x7f01d40fb0e0 msgr2=0x7f01d40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 -- 172.21.15.26:0/1722789448 shutdown_connections 2023-07-13T22:14:03.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.612+0000 7f01d9bff700 1 -- 172.21.15.26:0/1722789448 wait complete. 2023-07-13T22:14:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 Processor -- start 2023-07-13T22:14:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 -- start start 2023-07-13T22:14:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 0x7f01d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 0x7f01d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.613+0000 7f01d9bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40735e0 con 0x7f01d4100cd0 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01d9bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d4073750 con 0x7f01d40ffae0 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01d9bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d419f910 con 0x7f01d4101660 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45196/0 (socket says 172.21.15.26:45196) 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 -- 172.21.15.26:0/1376550313 learned_addr learned my addr 172.21.15.26:0/1376550313 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 0x7f01d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01d3fff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 0x7f01d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 -- 172.21.15.26:0/1376550313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 msgr2=0x7f01d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 0x7f01d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 -- 172.21.15.26:0/1376550313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 msgr2=0x7f01d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 0x7f01d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 -- 172.21.15.26:0/1376550313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc009520 con 0x7f01d4100cd0 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01d37fe700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 0x7f01d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.614+0000 7f01cbfff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00eb30 tx=0x7f01bc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.615+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc009880 con 0x7f01d4100cd0 2023-07-13T22:14:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.615+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01bc0044e0 con 0x7f01d4100cd0 2023-07-13T22:14:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.615+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc003e90 con 0x7f01d4100cd0 2023-07-13T22:14:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.615+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc009c40 con 0x7f01d4100cd0 2023-07-13T22:14:03.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.615+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c40095f0 con 0x7f01d4100cd0 2023-07-13T22:14:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.616+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d410b300 con 0x7f01d4100cd0 2023-07-13T22:14:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.618+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f01bc004000 con 0x7f01d4100cd0 2023-07-13T22:14:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.619+0000 7f01d17fa700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 0x7f01c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.619+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f01bc014070 con 0x7f01d4100cd0 2023-07-13T22:14:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.619+0000 7f01d37fe700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 0x7f01c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.620+0000 7f01d37fe700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 0x7f01c00a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b2e0 tx=0x7f01c4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.623+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f01bc096490 con 0x7f01d4100cd0 2023-07-13T22:14:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.831+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01d4106a10 con 0x7f01d4100cd0 2023-07-13T22:14:03.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.831+0000 7f01d17fa700 1 -- 172.21.15.26:0/1376550313 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01bc099ab0 con 0x7f01d4100cd0 2023-07-13T22:14:03.822 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:14:03.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 msgr2=0x7f01c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 0x7f01c00a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b2e0 tx=0x7f01c4005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 msgr2=0x7f01d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00eb30 tx=0x7f01bc00ee40 comp rx=0 tx=0).stop 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 shutdown_connections 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d4101660 0x7f01d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01d40ffae0 0x7f01d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01c00a11d0 0x7f01c00a3680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 --2- 172.21.15.26:0/1376550313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d4100cd0 0x7f01d4072900 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:03.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 >> 172.21.15.26:0/1376550313 conn(0x7f01d40fb0e0 msgr2=0x7f01d40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 shutdown_connections 2023-07-13T22:14:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:03.834+0000 7f01d9bff700 1 -- 172.21.15.26:0/1376550313 wait complete. 2023-07-13T22:14:03.843 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-13T22:14:03.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:14:03.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:03 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3014336124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:03.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:03 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1840730561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 -- 172.21.15.26:0/304412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 msgr2=0x7f59e40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 --2- 172.21.15.26:0/304412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e40fff10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009990 tx=0x7f59d4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 -- 172.21.15.26:0/304412 shutdown_connections 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 --2- 172.21.15.26:0/304412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e40fff10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 --2- 172.21.15.26:0/304412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4101680 0x7f59e4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 --2- 172.21.15.26:0/304412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4100cf0 0x7f59e4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 -- 172.21.15.26:0/304412 >> 172.21.15.26:0/304412 conn(0x7f59e40fb0e0 msgr2=0x7f59e40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 -- 172.21.15.26:0/304412 shutdown_connections 2023-07-13T22:14:04.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.057+0000 7f59e9dff700 1 -- 172.21.15.26:0/304412 wait complete. 2023-07-13T22:14:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 Processor -- start 2023-07-13T22:14:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 -- start start 2023-07-13T22:14:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4100cf0 0x7f59e4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 0x7f59e419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.058+0000 7f59e9dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419f630 con 0x7f59e40ffb00 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e9dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419f770 con 0x7f59e4100cf0 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e9dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e419f900 con 0x7f59e4101680 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45206/0 (socket says 172.21.15.26:45206) 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 -- 172.21.15.26:0/1656325832 learned_addr learned my addr 172.21.15.26:0/1656325832 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e2ffd700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4100cf0 0x7f59e4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e3fff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 0x7f59e419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 -- 172.21.15.26:0/1656325832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 msgr2=0x7f59e419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 0x7f59e419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 -- 172.21.15.26:0/1656325832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4100cf0 msgr2=0x7f59e4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4100cf0 0x7f59e4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 -- 172.21.15.26:0/1656325832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40095f0 con 0x7f59e40ffb00 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e3fff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 0x7f59e419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.059+0000 7f59e37fe700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f59d4004730 tx=0x7f59d4004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.060+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d401d070 con 0x7f59e40ffb00 2023-07-13T22:14:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.060+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59d4021410 con 0x7f59e40ffb00 2023-07-13T22:14:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.060+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e419fb80 con 0x7f59e40ffb00 2023-07-13T22:14:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.060+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4017650 con 0x7f59e40ffb00 2023-07-13T22:14:04.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.060+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e419ffa0 con 0x7f59e40ffb00 2023-07-13T22:14:04.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.061+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e410b320 con 0x7f59e40ffb00 2023-07-13T22:14:04.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.063+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f59d4021810 con 0x7f59e40ffb00 2023-07-13T22:14:04.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.065+0000 7f59e0ff9700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 0x7f59cc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.065+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59d40cde00 con 0x7f59e40ffb00 2023-07-13T22:14:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.065+0000 7f59e2ffd700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 0x7f59cc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.066+0000 7f59e2ffd700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 0x7f59cc0a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009c60 tx=0x7f59d8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:04.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.069+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f59d4097590 con 0x7f59e40ffb00 2023-07-13T22:14:04.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:03 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3014336124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:03 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1840730561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:03 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3014336124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:03 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1840730561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.328+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e410b510 con 0x7f59e40ffb00 2023-07-13T22:14:04.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.329+0000 7f59e0ff9700 1 -- 172.21.15.26:0/1656325832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f59d4095e60 con 0x7f59e40ffb00 2023-07-13T22:14:04.319 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:04.319 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 msgr2=0x7f59cc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 0x7f59cc0a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59d8009c60 tx=0x7f59d8009260 comp rx=0 tx=0).stop 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 msgr2=0x7f59e4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f59d4004730 tx=0x7f59d4004760 comp rx=0 tx=0).stop 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 shutdown_connections 2023-07-13T22:14:04.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59cc0a11a0 0x7f59cc0a3650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59e40ffb00 0x7f59e4197ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59e4101680 0x7f59e419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 --2- 172.21.15.26:0/1656325832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59e4100cf0 0x7f59e4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.331+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 >> 172.21.15.26:0/1656325832 conn(0x7f59e40fb0e0 msgr2=0x7f59e4106570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.332+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 shutdown_connections 2023-07-13T22:14:04.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.332+0000 7f59e9dff700 1 -- 172.21.15.26:0/1656325832 wait complete. 2023-07-13T22:14:04.324 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:14:04.340 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-13T22:14:04.340 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-13T22:14:04.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:14:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 -- 172.21.15.26:0/4277758927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec101680 msgr2=0x7f70ec107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 --2- 172.21.15.26:0/4277758927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec101680 0x7f70ec107f40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009f90 tx=0x7f70e400b330 comp rx=0 tx=0).stop 2023-07-13T22:14:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 -- 172.21.15.26:0/4277758927 shutdown_connections 2023-07-13T22:14:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 --2- 172.21.15.26:0/4277758927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec101680 0x7f70ec107f40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 --2- 172.21.15.26:0/4277758927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec0ffb00 0x7f70ec0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 --2- 172.21.15.26:0/4277758927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec100cf0 0x7f70ec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 -- 172.21.15.26:0/4277758927 >> 172.21.15.26:0/4277758927 conn(0x7f70ec0fb0e0 msgr2=0x7f70ec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.556+0000 7f70f37ff700 1 -- 172.21.15.26:0/4277758927 shutdown_connections 2023-07-13T22:14:04.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.557+0000 7f70f37ff700 1 -- 172.21.15.26:0/4277758927 wait complete. 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 Processor -- start 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 -- start start 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec100cf0 0x7f70ec195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec101680 0x7f70ec19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec19d4c0 con 0x7f70ec0ffb00 2023-07-13T22:14:04.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec19d600 con 0x7f70ec101680 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f37ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec19d790 con 0x7f70ec100cf0 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f159b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f159b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45220/0 (socket says 172.21.15.26:45220) 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f159b700 1 -- 172.21.15.26:0/3134957658 learned_addr learned my addr 172.21.15.26:0/3134957658 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.558+0000 7f70f0d9a700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec100cf0 0x7f70ec195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f1d9c700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec101680 0x7f70ec19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 -- 172.21.15.26:0/3134957658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec100cf0 msgr2=0x7f70ec195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec100cf0 0x7f70ec195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 -- 172.21.15.26:0/3134957658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec101680 msgr2=0x7f70ec19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec101680 0x7f70ec19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 -- 172.21.15.26:0/3134957658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4009c40 con 0x7f70ec0ffb00 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f159b700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8004220 con 0x7f70ec0ffb00 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e8004380 con 0x7f70ec0ffb00 2023-07-13T22:14:04.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec19da70 con 0x7f70ec0ffb00 2023-07-13T22:14:04.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8003be0 con 0x7f70ec0ffb00 2023-07-13T22:14:04.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.559+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec19dfd0 con 0x7f70ec0ffb00 2023-07-13T22:14:04.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.560+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec106ff0 con 0x7f70ec0ffb00 2023-07-13T22:14:04.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.562+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f70e8003d40 con 0x7f70ec0ffb00 2023-07-13T22:14:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.564+0000 7f70e27fc700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 0x7f70d80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.564+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f70e8014070 con 0x7f70ec0ffb00 2023-07-13T22:14:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.564+0000 7f70f0d9a700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 0x7f70d80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:04.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.564+0000 7f70f0d9a700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 0x7f70d80a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009c60 tx=0x7f70dc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:04.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.568+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f70e80960a0 con 0x7f70ec0ffb00 2023-07-13T22:14:04.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.774+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70ec060950 con 0x7f70d80a1220 2023-07-13T22:14:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.785+0000 7f70e27fc700 1 -- 172.21.15.26:0/3134957658 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345482 (secure 0 0 0) 0x7f70ec060950 con 0x7f70d80a1220 2023-07-13T22:14:04.779 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 msgr2=0x7f70d80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 0x7f70d80a36d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009c60 tx=0x7f70dc009260 comp rx=0 tx=0).stop 2023-07-13T22:14:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 msgr2=0x7f70ec195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800ed00 comp rx=0 tx=0).stop 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 shutdown_connections 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70d80a1220 0x7f70d80a36d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70ec0ffb00 0x7f70ec195940 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70ec101680 0x7f70ec19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 --2- 172.21.15.26:0/3134957658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70ec100cf0 0x7f70ec195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 >> 172.21.15.26:0/3134957658 conn(0x7f70ec0fb0e0 msgr2=0x7f70ec104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:04.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 shutdown_connections 2023-07-13T22:14:04.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:04.790+0000 7f70f37ff700 1 -- 172.21.15.26:0/3134957658 wait complete. 2023-07-13T22:14:04.783 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:14:04.799 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2023-07-13T22:14:03.445930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659521,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1105,"num_read_kb":680,"num_write":424,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182248,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":850,"ondisk_log_size":850,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":13312000,"data_stored":7281891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5342,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":74,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002984"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461705+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:13:51.461705+0000","last_peered":"2023-07-13T22:13:51.461705+0000","last_clean":"2023-07-13T22:13:51.461705+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:13:51.461705+0000","last_undegraded":"2023-07-13T22:13:51.461705+0000","last_fullsized":"2023-07-13T22:13:51.461705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:47.548617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455808+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:13:51.455808+0000","last_peered":"2023-07-13T22:13:51.455808+0000","last_clean":"2023-07-13T22:13:51.455808+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:13:51.455808+0000","last_undegraded":"2023-07-13T22:13:51.455808+0000","last_fullsized":"2023-07-13T22:13:51.455808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:21.973731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453820+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:13:51.453820+0000","last_peered":"2023-07-13T22:13:51.453820+0000","last_clean":"2023-07-13T22:13:51.453820+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:13:51.453820+0000","last_undegraded":"2023-07-13T22:13:51.453820+0000","last_fullsized":"2023-07-13T22:13:51.453820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:51.412468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453453+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:13:51.453453+0000","last_peered":"2023-07-13T22:13:51.453453+0000","last_clean":"2023-07-13T22:13:51.453453+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:13:51.453453+0000","last_undegraded":"2023-07-13T22:13:51.453453+0000","last_fullsized":"2023-07-13T22:13:51.453453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:33.084972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464668+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:13:51.464668+0000","last_peered":"2023-07-13T22:13:51.464668+0000","last_clean":"2023-07-13T22:13:51.464668+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:13:51.464668+0000","last_undegraded":"2023-07-13T22:13:51.464668+0000","last_fullsized":"2023-07-13T22:13:51.464668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:14:33.505950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453475+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:13:51.453475+0000","last_peered":"2023-07-13T22:13:51.453475+0000","last_clean":"2023-07-13T22:13:51.453475+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:13:51.453475+0000","last_undegraded":"2023-07-13T22:13:51.453475+0000","last_fullsized":"2023-07-13T22:13:51.453475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:04.653956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.444154+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:13:51.444154+0000","last_peered":"2023-07-13T22:13:51.444154+0000","last_clean":"2023-07-13T22:13:51.444154+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:13:51.444154+0000","last_undegraded":"2023-07-13T22:13:51.444154+0000","last_fullsized":"2023-07-13T22:13:51.444154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:06.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465428+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:13:51.465428+0000","last_peered":"2023-07-13T22:13:51.465428+0000","last_clean":"2023-07-13T22:13:51.465428+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:13:51.465428+0000","last_undegraded":"2023-07-13T22:13:51.465428+0000","last_fullsized":"2023-07-13T22:13:51.465428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:49.982151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464318+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:13:51.464318+0000","last_peered":"2023-07-13T22:13:51.464318+0000","last_clean":"2023-07-13T22:13:51.464318+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:13:51.464318+0000","last_undegraded":"2023-07-13T22:13:51.464318+0000","last_fullsized":"2023-07-13T22:13:51.464318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.458063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456141+0000","last_change":"2023-07-13T22:12:52.546759+0000","last_active":"2023-07-13T22:13:51.456141+0000","last_peered":"2023-07-13T22:13:51.456141+0000","last_clean":"2023-07-13T22:13:51.456141+0000","last_became_active":"2023-07-13T22:12:52.546378+0000","last_became_peered":"2023-07-13T22:12:52.546378+0000","last_unstale":"2023-07-13T22:13:51.456141+0000","last_undegraded":"2023-07-13T22:13:51.456141+0000","last_fullsized":"2023-07-13T22:13:51.456141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:38.494158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.450988+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:13:51.450988+0000","last_peered":"2023-07-13T22:13:51.450988+0000","last_clean":"2023-07-13T22:13:51.450988+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:13:51.450988+0000","last_undegraded":"2023-07-13T22:13:51.450988+0000","last_fullsized":"2023-07-13T22:13:51.450988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:47.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.454081+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:13:51.454081+0000","last_peered":"2023-07-13T22:13:51.454081+0000","last_clean":"2023-07-13T22:13:51.454081+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:13:51.454081+0000","last_undegraded":"2023-07-13T22:13:51.454081+0000","last_fullsized":"2023-07-13T22:13:51.454081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:55.880996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465068+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:13:51.465068+0000","last_peered":"2023-07-13T22:13:51.465068+0000","last_clean":"2023-07-13T22:13:51.465068+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:13:51.465068+0000","last_undegraded":"2023-07-13T22:13:51.465068+0000","last_fullsized":"2023-07-13T22:13:51.465068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:58.155432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457726+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:13:51.457726+0000","last_peered":"2023-07-13T22:13:51.457726+0000","last_clean":"2023-07-13T22:13:51.457726+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:13:51.457726+0000","last_undegraded":"2023-07-13T22:13:51.457726+0000","last_fullsized":"2023-07-13T22:13:51.457726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:57.803872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445655+0000","last_change":"2023-07-13T22:12:52.549238+0000","last_active":"2023-07-13T22:13:51.445655+0000","last_peered":"2023-07-13T22:13:51.445655+0000","last_clean":"2023-07-13T22:13:51.445655+0000","last_became_active":"2023-07-13T22:12:52.548923+0000","last_became_peered":"2023-07-13T22:12:52.548923+0000","last_unstale":"2023-07-13T22:13:51.445655+0000","last_undegraded":"2023-07-13T22:13:51.445655+0000","last_fullsized":"2023-07-13T22:13:51.445655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:08.925620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459735+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:13:51.459735+0000","last_peered":"2023-07-13T22:13:51.459735+0000","last_clean":"2023-07-13T22:13:51.459735+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:13:51.459735+0000","last_undegraded":"2023-07-13T22:13:51.459735+0000","last_fullsized":"2023-07-13T22:13:51.459735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:23.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461935+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:13:51.461935+0000","last_peered":"2023-07-13T22:13:51.461935+0000","last_clean":"2023-07-13T22:13:51.461935+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:13:51.461935+0000","last_undegraded":"2023-07-13T22:13:51.461935+0000","last_fullsized":"2023-07-13T22:13:51.461935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:37.216203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463124+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:13:51.463124+0000","last_peered":"2023-07-13T22:13:51.463124+0000","last_clean":"2023-07-13T22:13:51.463124+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:13:51.463124+0000","last_undegraded":"2023-07-13T22:13:51.463124+0000","last_fullsized":"2023-07-13T22:13:51.463124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:18.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456310+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:13:51.456310+0000","last_peered":"2023-07-13T22:13:51.456310+0000","last_clean":"2023-07-13T22:13:51.456310+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:13:51.456310+0000","last_undegraded":"2023-07-13T22:13:51.456310+0000","last_fullsized":"2023-07-13T22:13:51.456310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:56.130000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:57.616438+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:13:57.616438+0000","last_peered":"2023-07-13T22:13:57.616438+0000","last_clean":"2023-07-13T22:13:57.616438+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:13:57.616438+0000","last_undegraded":"2023-07-13T22:13:57.616438+0000","last_fullsized":"2023-07-13T22:13:57.616438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:28:47.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456796+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:13:51.456796+0000","last_peered":"2023-07-13T22:13:51.456796+0000","last_clean":"2023-07-13T22:13:51.456796+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:13:51.456796+0000","last_undegraded":"2023-07-13T22:13:51.456796+0000","last_fullsized":"2023-07-13T22:13:51.456796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:42:35.256971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791902+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:52.791902+0000","last_peered":"2023-07-13T22:12:52.791902+0000","last_clean":"2023-07-13T22:12:52.791902+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:52.791902+0000","last_undegraded":"2023-07-13T22:12:52.791902+0000","last_fullsized":"2023-07-13T22:12:52.791902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:29.131604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464536+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:13:51.464536+0000","last_peered":"2023-07-13T22:13:51.464536+0000","last_clean":"2023-07-13T22:13:51.464536+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:13:51.464536+0000","last_undegraded":"2023-07-13T22:13:51.464536+0000","last_fullsized":"2023-07-13T22:13:51.464536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:53.951535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.554410+0000","last_change":"2023-07-13T22:12:52.554410+0000","last_active":"2023-07-13T22:12:52.554410+0000","last_peered":"2023-07-13T22:12:52.554410+0000","last_clean":"2023-07-13T22:12:52.554410+0000","last_became_active":"2023-07-13T22:12:52.541864+0000","last_became_peered":"2023-07-13T22:12:52.541864+0000","last_unstale":"2023-07-13T22:12:52.554410+0000","last_undegraded":"2023-07-13T22:12:52.554410+0000","last_fullsized":"2023-07-13T22:12:52.554410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:48:29.339493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463743+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:13:51.463743+0000","last_peered":"2023-07-13T22:13:51.463743+0000","last_clean":"2023-07-13T22:13:51.463743+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:13:51.463743+0000","last_undegraded":"2023-07-13T22:13:51.463743+0000","last_fullsized":"2023-07-13T22:13:51.463743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:58.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542998+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:52.542998+0000","last_peered":"2023-07-13T22:12:52.542998+0000","last_clean":"2023-07-13T22:12:52.542998+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:52.542998+0000","last_undegraded":"2023-07-13T22:12:52.542998+0000","last_fullsized":"2023-07-13T22:12:52.542998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:22:28.850188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457222+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:13:51.457222+0000","last_peered":"2023-07-13T22:13:51.457222+0000","last_clean":"2023-07-13T22:13:51.457222+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:13:51.457222+0000","last_undegraded":"2023-07-13T22:13:51.457222+0000","last_fullsized":"2023-07-13T22:13:51.457222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:21.578829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":123,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572636+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:14:00.572636+0000","last_peered":"2023-07-13T22:14:00.572636+0000","last_clean":"2023-07-13T22:14:00.572636+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:14:00.572636+0000","last_undegraded":"2023-07-13T22:14:00.572636+0000","last_fullsized":"2023-07-13T22:14:00.572636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:56:16.055459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.441362+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:13:51.441362+0000","last_peered":"2023-07-13T22:13:51.441362+0000","last_clean":"2023-07-13T22:13:51.441362+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:13:51.441362+0000","last_undegraded":"2023-07-13T22:13:51.441362+0000","last_fullsized":"2023-07-13T22:13:51.441362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:28.566300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538988+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:52.538988+0000","last_peered":"2023-07-13T22:12:52.538988+0000","last_clean":"2023-07-13T22:12:52.538988+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:52.538988+0000","last_undegraded":"2023-07-13T22:12:52.538988+0000","last_fullsized":"2023-07-13T22:12:52.538988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:01:40.167436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465176+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:13:51.465176+0000","last_peered":"2023-07-13T22:13:51.465176+0000","last_clean":"2023-07-13T22:13:51.465176+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:13:51.465176+0000","last_undegraded":"2023-07-13T22:13:51.465176+0000","last_fullsized":"2023-07-13T22:13:51.465176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:03.123538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537975+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:52.537975+0000","last_peered":"2023-07-13T22:12:52.537975+0000","last_clean":"2023-07-13T22:12:52.537975+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:52.537975+0000","last_undegraded":"2023-07-13T22:12:52.537975+0000","last_fullsized":"2023-07-13T22:12:52.537975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451163+0000","last_change":"2023-07-13T22:12:52.546725+0000","last_active":"2023-07-13T22:13:51.451163+0000","last_peered":"2023-07-13T22:13:51.451163+0000","last_clean":"2023-07-13T22:13:51.451163+0000","last_became_active":"2023-07-13T22:12:52.546385+0000","last_became_peered":"2023-07-13T22:12:52.546385+0000","last_unstale":"2023-07-13T22:13:51.451163+0000","last_undegraded":"2023-07-13T22:13:51.451163+0000","last_fullsized":"2023-07-13T22:13:51.451163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:18.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534851+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:52.534851+0000","last_peered":"2023-07-13T22:12:52.534851+0000","last_clean":"2023-07-13T22:12:52.534851+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:52.534851+0000","last_undegraded":"2023-07-13T22:12:52.534851+0000","last_fullsized":"2023-07-13T22:12:52.534851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:06.618468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457170+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:13:51.457170+0000","last_peered":"2023-07-13T22:13:51.457170+0000","last_clean":"2023-07-13T22:13:51.457170+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:13:51.457170+0000","last_undegraded":"2023-07-13T22:13:51.457170+0000","last_fullsized":"2023-07-13T22:13:51.457170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:26.423367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.573478+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:14:01.573478+0000","last_peered":"2023-07-13T22:14:01.573478+0000","last_clean":"2023-07-13T22:14:01.573478+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:14:01.573478+0000","last_undegraded":"2023-07-13T22:14:01.573478+0000","last_fullsized":"2023-07-13T22:14:01.573478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:37.488363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459625+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:13:51.459625+0000","last_peered":"2023-07-13T22:13:51.459625+0000","last_clean":"2023-07-13T22:13:51.459625+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:13:51.459625+0000","last_undegraded":"2023-07-13T22:13:51.459625+0000","last_fullsized":"2023-07-13T22:13:51.459625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:47.438310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541598+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:52.541598+0000","last_peered":"2023-07-13T22:12:52.541598+0000","last_clean":"2023-07-13T22:12:52.541598+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:52.541598+0000","last_undegraded":"2023-07-13T22:12:52.541598+0000","last_fullsized":"2023-07-13T22:12:52.541598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:00.126919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458115+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:13:51.458115+0000","last_peered":"2023-07-13T22:13:51.458115+0000","last_clean":"2023-07-13T22:13:51.458115+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:13:51.458115+0000","last_undegraded":"2023-07-13T22:13:51.458115+0000","last_fullsized":"2023-07-13T22:13:51.458115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:50.338306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535374+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:52.535374+0000","last_peered":"2023-07-13T22:12:52.535374+0000","last_clean":"2023-07-13T22:12:52.535374+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:52.535374+0000","last_undegraded":"2023-07-13T22:12:52.535374+0000","last_fullsized":"2023-07-13T22:12:52.535374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:36.251918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462201+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:13:51.462201+0000","last_peered":"2023-07-13T22:13:51.462201+0000","last_clean":"2023-07-13T22:13:51.462201+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:13:51.462201+0000","last_undegraded":"2023-07-13T22:13:51.462201+0000","last_fullsized":"2023-07-13T22:13:51.462201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:23.829222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":257,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574528+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:14:01.574528+0000","last_peered":"2023-07-13T22:14:01.574528+0000","last_clean":"2023-07-13T22:14:01.574528+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:14:01.574528+0000","last_undegraded":"2023-07-13T22:14:01.574528+0000","last_fullsized":"2023-07-13T22:14:01.574528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:54:07.569446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61830,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":157,"num_write":36,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445202+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:13:51.445202+0000","last_peered":"2023-07-13T22:13:51.445202+0000","last_clean":"2023-07-13T22:13:51.445202+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:13:51.445202+0000","last_undegraded":"2023-07-13T22:13:51.445202+0000","last_fullsized":"2023-07-13T22:13:51.445202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:19.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540584+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:52.540584+0000","last_peered":"2023-07-13T22:12:52.540584+0000","last_clean":"2023-07-13T22:12:52.540584+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:52.540584+0000","last_undegraded":"2023-07-13T22:12:52.540584+0000","last_fullsized":"2023-07-13T22:12:52.540584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:23.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466045+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:13:51.466045+0000","last_peered":"2023-07-13T22:13:51.466045+0000","last_clean":"2023-07-13T22:13:51.466045+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:13:51.466045+0000","last_undegraded":"2023-07-13T22:13:51.466045+0000","last_fullsized":"2023-07-13T22:13:51.466045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:37.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534895+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:52.534895+0000","last_peered":"2023-07-13T22:12:52.534895+0000","last_clean":"2023-07-13T22:12:52.534895+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:52.534895+0000","last_undegraded":"2023-07-13T22:12:52.534895+0000","last_fullsized":"2023-07-13T22:12:52.534895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:03.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462642+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:13:51.462642+0000","last_peered":"2023-07-13T22:13:51.462642+0000","last_clean":"2023-07-13T22:13:51.462642+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:13:51.462642+0000","last_undegraded":"2023-07-13T22:13:51.462642+0000","last_fullsized":"2023-07-13T22:13:51.462642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:29.680386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574264+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:14:01.574264+0000","last_peered":"2023-07-13T22:14:01.574264+0000","last_clean":"2023-07-13T22:14:01.574264+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:14:01.574264+0000","last_undegraded":"2023-07-13T22:14:01.574264+0000","last_fullsized":"2023-07-13T22:14:01.574264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:24.354790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.449957+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:13:51.449957+0000","last_peered":"2023-07-13T22:13:51.449957+0000","last_clean":"2023-07-13T22:13:51.449957+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:13:51.449957+0000","last_undegraded":"2023-07-13T22:13:51.449957+0000","last_fullsized":"2023-07-13T22:13:51.449957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:39.397611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541703+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:52.541703+0000","last_peered":"2023-07-13T22:12:52.541703+0000","last_clean":"2023-07-13T22:12:52.541703+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:52.541703+0000","last_undegraded":"2023-07-13T22:12:52.541703+0000","last_fullsized":"2023-07-13T22:12:52.541703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:54.534152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456704+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:13:51.456704+0000","last_peered":"2023-07-13T22:13:51.456704+0000","last_clean":"2023-07-13T22:13:51.456704+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:13:51.456704+0000","last_undegraded":"2023-07-13T22:13:51.456704+0000","last_fullsized":"2023-07-13T22:13:51.456704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:18:37.092789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543636+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:52.543636+0000","last_peered":"2023-07-13T22:12:52.543636+0000","last_clean":"2023-07-13T22:12:52.543636+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:52.543636+0000","last_undegraded":"2023-07-13T22:12:52.543636+0000","last_fullsized":"2023-07-13T22:12:52.543636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:29:05.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456758+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:13:51.456758+0000","last_peered":"2023-07-13T22:13:51.456758+0000","last_clean":"2023-07-13T22:13:51.456758+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:13:51.456758+0000","last_undegraded":"2023-07-13T22:13:51.456758+0000","last_fullsized":"2023-07-13T22:13:51.456758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:53:20.274354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539898+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:52.539898+0000","last_peered":"2023-07-13T22:12:52.539898+0000","last_clean":"2023-07-13T22:12:52.539898+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:52.539898+0000","last_undegraded":"2023-07-13T22:12:52.539898+0000","last_fullsized":"2023-07-13T22:12:52.539898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:26.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465891+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:13:51.465891+0000","last_peered":"2023-07-13T22:13:51.465891+0000","last_clean":"2023-07-13T22:13:51.465891+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:13:51.465891+0000","last_undegraded":"2023-07-13T22:13:51.465891+0000","last_fullsized":"2023-07-13T22:13:51.465891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:02.628236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:56.615650+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:13:56.615650+0000","last_peered":"2023-07-13T22:13:56.615650+0000","last_clean":"2023-07-13T22:13:56.615650+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:13:56.615650+0000","last_undegraded":"2023-07-13T22:13:56.615650+0000","last_fullsized":"2023-07-13T22:13:56.615650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:22.807561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453734+0000","last_change":"2023-07-13T22:12:52.546608+0000","last_active":"2023-07-13T22:13:51.453734+0000","last_peered":"2023-07-13T22:13:51.453734+0000","last_clean":"2023-07-13T22:13:51.453734+0000","last_became_active":"2023-07-13T22:12:52.546240+0000","last_became_peered":"2023-07-13T22:12:52.546240+0000","last_unstale":"2023-07-13T22:13:51.453734+0000","last_undegraded":"2023-07-13T22:13:51.453734+0000","last_fullsized":"2023-07-13T22:13:51.453734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:11.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456809+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:13:51.456809+0000","last_peered":"2023-07-13T22:13:51.456809+0000","last_clean":"2023-07-13T22:13:51.456809+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:13:51.456809+0000","last_undegraded":"2023-07-13T22:13:51.456809+0000","last_fullsized":"2023-07-13T22:13:51.456809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:05.891842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572319+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:14:00.572319+0000","last_peered":"2023-07-13T22:14:00.572319+0000","last_clean":"2023-07-13T22:14:00.572319+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:14:00.572319+0000","last_undegraded":"2023-07-13T22:14:00.572319+0000","last_fullsized":"2023-07-13T22:14:00.572319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:09.930750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464940+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:13:51.464940+0000","last_peered":"2023-07-13T22:13:51.464940+0000","last_clean":"2023-07-13T22:13:51.464940+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:13:51.464940+0000","last_undegraded":"2023-07-13T22:13:51.464940+0000","last_fullsized":"2023-07-13T22:13:51.464940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:42:38.987941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539678+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:52.539678+0000","last_peered":"2023-07-13T22:12:52.539678+0000","last_clean":"2023-07-13T22:12:52.539678+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:52.539678+0000","last_undegraded":"2023-07-13T22:12:52.539678+0000","last_fullsized":"2023-07-13T22:12:52.539678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:49.259253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540813+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:52.540813+0000","last_peered":"2023-07-13T22:12:52.540813+0000","last_clean":"2023-07-13T22:12:52.540813+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:52.540813+0000","last_undegraded":"2023-07-13T22:12:52.540813+0000","last_fullsized":"2023-07-13T22:12:52.540813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:11:16.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461027+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:13:51.461027+0000","last_peered":"2023-07-13T22:13:51.461027+0000","last_clean":"2023-07-13T22:13:51.461027+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:13:51.461027+0000","last_undegraded":"2023-07-13T22:13:51.461027+0000","last_fullsized":"2023-07-13T22:13:51.461027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:45.725233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541086+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:52.541086+0000","last_peered":"2023-07-13T22:12:52.541086+0000","last_clean":"2023-07-13T22:12:52.541086+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:52.541086+0000","last_undegraded":"2023-07-13T22:12:52.541086+0000","last_fullsized":"2023-07-13T22:12:52.541086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:50.437984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543196+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:52.543196+0000","last_peered":"2023-07-13T22:12:52.543196+0000","last_clean":"2023-07-13T22:12:52.543196+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:52.543196+0000","last_undegraded":"2023-07-13T22:12:52.543196+0000","last_fullsized":"2023-07-13T22:12:52.543196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:37.229195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455688+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:13:51.455688+0000","last_peered":"2023-07-13T22:13:51.455688+0000","last_clean":"2023-07-13T22:13:51.455688+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:13:51.455688+0000","last_undegraded":"2023-07-13T22:13:51.455688+0000","last_fullsized":"2023-07-13T22:13:51.455688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:01.886536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540162+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:52.540162+0000","last_peered":"2023-07-13T22:12:52.540162+0000","last_clean":"2023-07-13T22:12:52.540162+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:52.540162+0000","last_undegraded":"2023-07-13T22:12:52.540162+0000","last_fullsized":"2023-07-13T22:12:52.540162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:27.067307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459700+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:13:51.459700+0000","last_peered":"2023-07-13T22:13:51.459700+0000","last_clean":"2023-07-13T22:13:51.459700+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:13:51.459700+0000","last_undegraded":"2023-07-13T22:13:51.459700+0000","last_fullsized":"2023-07-13T22:13:51.459700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:24.414406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:15.337161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466111+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:13:51.466111+0000","last_peered":"2023-07-13T22:13:51.466111+0000","last_clean":"2023-07-13T22:13:51.466111+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:13:51.466111+0000","last_undegraded":"2023-07-13T22:13:51.466111+0000","last_fullsized":"2023-07-13T22:13:51.466111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:39.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.551910+0000","last_change":"2023-07-13T22:12:52.551910+0000","last_active":"2023-07-13T22:12:52.551910+0000","last_peered":"2023-07-13T22:12:52.551910+0000","last_clean":"2023-07-13T22:12:52.551910+0000","last_became_active":"2023-07-13T22:12:52.548681+0000","last_became_peered":"2023-07-13T22:12:52.548681+0000","last_unstale":"2023-07-13T22:12:52.551910+0000","last_undegraded":"2023-07-13T22:12:52.551910+0000","last_fullsized":"2023-07-13T22:12:52.551910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:54:10.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.422697+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:13:51.422697+0000","last_peered":"2023-07-13T22:13:51.422697+0000","last_clean":"2023-07-13T22:13:51.422697+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:13:51.422697+0000","last_undegraded":"2023-07-13T22:13:51.422697+0000","last_fullsized":"2023-07-13T22:13:51.422697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:53.309826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791341+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:52.791341+0000","last_peered":"2023-07-13T22:12:52.791341+0000","last_clean":"2023-07-13T22:12:52.791341+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:52.791341+0000","last_undegraded":"2023-07-13T22:12:52.791341+0000","last_fullsized":"2023-07-13T22:12:52.791341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:31.189715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445763+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:13:51.445763+0000","last_peered":"2023-07-13T22:13:51.445763+0000","last_clean":"2023-07-13T22:13:51.445763+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:13:51.445763+0000","last_undegraded":"2023-07-13T22:13:51.445763+0000","last_fullsized":"2023-07-13T22:13:51.445763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:22.391535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540832+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:52.540832+0000","last_peered":"2023-07-13T22:12:52.540832+0000","last_clean":"2023-07-13T22:12:52.540832+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:52.540832+0000","last_undegraded":"2023-07-13T22:12:52.540832+0000","last_fullsized":"2023-07-13T22:12:52.540832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:34.701191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465349+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:13:51.465349+0000","last_peered":"2023-07-13T22:13:51.465349+0000","last_clean":"2023-07-13T22:13:51.465349+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:13:51.465349+0000","last_undegraded":"2023-07-13T22:13:51.465349+0000","last_fullsized":"2023-07-13T22:13:51.465349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:26:29.278158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542783+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:52.542783+0000","last_peered":"2023-07-13T22:12:52.542783+0000","last_clean":"2023-07-13T22:12:52.542783+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:52.542783+0000","last_undegraded":"2023-07-13T22:12:52.542783+0000","last_fullsized":"2023-07-13T22:12:52.542783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:04:20.180883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461401+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:13:51.461401+0000","last_peered":"2023-07-13T22:13:51.461401+0000","last_clean":"2023-07-13T22:13:51.461401+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:13:51.461401+0000","last_undegraded":"2023-07-13T22:13:51.461401+0000","last_fullsized":"2023-07-13T22:13:51.461401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:53.859029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":94,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.662195+0000","last_change":"2023-07-13T22:12:52.555796+0000","last_active":"2023-07-13T22:14:01.662195+0000","last_peered":"2023-07-13T22:14:01.662195+0000","last_clean":"2023-07-13T22:14:01.662195+0000","last_became_active":"2023-07-13T22:12:52.548664+0000","last_became_peered":"2023-07-13T22:12:52.548664+0000","last_unstale":"2023-07-13T22:14:01.662195+0000","last_undegraded":"2023-07-13T22:14:01.662195+0000","last_fullsized":"2023-07-13T22:14:01.662195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:37.846269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452434+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:13:51.452434+0000","last_peered":"2023-07-13T22:13:51.452434+0000","last_clean":"2023-07-13T22:13:51.452434+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:13:51.452434+0000","last_undegraded":"2023-07-13T22:13:51.452434+0000","last_fullsized":"2023-07-13T22:13:51.452434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:58.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535107+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:52.535107+0000","last_peered":"2023-07-13T22:12:52.535107+0000","last_clean":"2023-07-13T22:12:52.535107+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:52.535107+0000","last_undegraded":"2023-07-13T22:12:52.535107+0000","last_fullsized":"2023-07-13T22:12:52.535107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:29.092920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463990+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:13:51.463990+0000","last_peered":"2023-07-13T22:13:51.463990+0000","last_clean":"2023-07-13T22:13:51.463990+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:13:51.463990+0000","last_undegraded":"2023-07-13T22:13:51.463990+0000","last_fullsized":"2023-07-13T22:13:51.463990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:10.233101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538994+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:52.538994+0000","last_peered":"2023-07-13T22:12:52.538994+0000","last_clean":"2023-07-13T22:12:52.538994+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:52.538994+0000","last_undegraded":"2023-07-13T22:12:52.538994+0000","last_fullsized":"2023-07-13T22:12:52.538994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:31.553022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452262+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:13:51.452262+0000","last_peered":"2023-07-13T22:13:51.452262+0000","last_clean":"2023-07-13T22:13:51.452262+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:13:51.452262+0000","last_undegraded":"2023-07-13T22:13:51.452262+0000","last_fullsized":"2023-07-13T22:13:51.452262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:27.141696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448083+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:13:51.448083+0000","last_peered":"2023-07-13T22:13:51.448083+0000","last_clean":"2023-07-13T22:13:51.448083+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:13:51.448083+0000","last_undegraded":"2023-07-13T22:13:51.448083+0000","last_fullsized":"2023-07-13T22:13:51.448083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:12.051633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451991+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:13:51.451991+0000","last_peered":"2023-07-13T22:13:51.451991+0000","last_clean":"2023-07-13T22:13:51.451991+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:13:51.451991+0000","last_undegraded":"2023-07-13T22:13:51.451991+0000","last_fullsized":"2023-07-13T22:13:51.451991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:04:24.267514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448875+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:13:51.448875+0000","last_peered":"2023-07-13T22:13:51.448875+0000","last_clean":"2023-07-13T22:13:51.448875+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:13:51.448875+0000","last_undegraded":"2023-07-13T22:13:51.448875+0000","last_fullsized":"2023-07-13T22:13:51.448875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:21.951262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461338+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:13:51.461338+0000","last_peered":"2023-07-13T22:13:51.461338+0000","last_clean":"2023-07-13T22:13:51.461338+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:13:51.461338+0000","last_undegraded":"2023-07-13T22:13:51.461338+0000","last_fullsized":"2023-07-13T22:13:51.461338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:45.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461520+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:13:51.461520+0000","last_peered":"2023-07-13T22:13:51.461520+0000","last_clean":"2023-07-13T22:13:51.461520+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:13:51.461520+0000","last_undegraded":"2023-07-13T22:13:51.461520+0000","last_fullsized":"2023-07-13T22:13:51.461520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:36.995817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.447741+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:13:51.447741+0000","last_peered":"2023-07-13T22:13:51.447741+0000","last_clean":"2023-07-13T22:13:51.447741+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:13:51.447741+0000","last_undegraded":"2023-07-13T22:13:51.447741+0000","last_fullsized":"2023-07-13T22:13:51.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:14.395567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461321+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:13:51.461321+0000","last_peered":"2023-07-13T22:13:51.461321+0000","last_clean":"2023-07-13T22:13:51.461321+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:13:51.461321+0000","last_undegraded":"2023-07-13T22:13:51.461321+0000","last_fullsized":"2023-07-13T22:13:51.461321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:32.074362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459380+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:13:51.459380+0000","last_peered":"2023-07-13T22:13:51.459380+0000","last_clean":"2023-07-13T22:13:51.459380+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:13:51.459380+0000","last_undegraded":"2023-07-13T22:13:51.459380+0000","last_fullsized":"2023-07-13T22:13:51.459380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:16:17.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458725+0000","last_change":"2023-07-13T22:12:52.793042+0000","last_active":"2023-07-13T22:13:51.458725+0000","last_peered":"2023-07-13T22:13:51.458725+0000","last_clean":"2023-07-13T22:13:51.458725+0000","last_became_active":"2023-07-13T22:12:52.792625+0000","last_became_peered":"2023-07-13T22:12:52.792625+0000","last_unstale":"2023-07-13T22:13:51.458725+0000","last_undegraded":"2023-07-13T22:13:51.458725+0000","last_fullsized":"2023-07-13T22:13:51.458725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:37.356733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459347+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:13:51.459347+0000","last_peered":"2023-07-13T22:13:51.459347+0000","last_clean":"2023-07-13T22:13:51.459347+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:13:51.459347+0000","last_undegraded":"2023-07-13T22:13:51.459347+0000","last_fullsized":"2023-07-13T22:13:51.459347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:38.425408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466615+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:13:51.466615+0000","last_peered":"2023-07-13T22:13:51.466615+0000","last_clean":"2023-07-13T22:13:51.466615+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:13:51.466615+0000","last_undegraded":"2023-07-13T22:13:51.466615+0000","last_fullsized":"2023-07-13T22:13:51.466615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:38.511996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465842+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:13:51.465842+0000","last_peered":"2023-07-13T22:13:51.465842+0000","last_clean":"2023-07-13T22:13:51.465842+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:13:51.465842+0000","last_undegraded":"2023-07-13T22:13:51.465842+0000","last_fullsized":"2023-07-13T22:13:51.465842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:16.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458793+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:13:51.458793+0000","last_peered":"2023-07-13T22:13:51.458793+0000","last_clean":"2023-07-13T22:13:51.458793+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:13:51.458793+0000","last_undegraded":"2023-07-13T22:13:51.458793+0000","last_fullsized":"2023-07-13T22:13:51.458793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:02.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68907,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":979,"num_read_kb":571,"num_write":241,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1512,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":197765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":445151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":1.093,"15min":1.093},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":1.357,"15min":1.357},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.63500000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91900000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.75700000000000001}]},{"osd":8,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67900000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99299999999999999,"5min":1.335,"15min":1.335},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61299999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88900000000000001,"5min":1.23,"15min":1.23},"last":0.78100000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678054,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1092849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.54700000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.81299999999999994,"15min":0.83099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.9289999999999998,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.85499999999999998,"15min":0.89100000000000001},"min":{"1min":0.49399999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.47899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82499999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.90300000000000002,"15min":0.93400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5980000000000001,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.7050000000000001,"5min":5.234,"15min":5.234},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":1.117,"15min":1.117},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.497,"15min":0.497},"max":{"1min":0.91100000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94399999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96399999999999997}]},{"osd":11,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.754,"15min":0.752},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.56100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.89800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":1.109,"15min":1.109},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":1.101,"15min":1.101},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.0649999999999999,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.018,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.80300000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89000000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.747}]},{"osd":7,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.94199999999999995,"15min":0.95199999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.95599999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.581,"15min":1.581},"last":0.69499999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":0.999},"min":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.83499999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.78000000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84299999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":444660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.48199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82799999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.44600000000000001},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.47799999999999998,"15min":0.45400000000000001},"min":{"1min":0.372,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.624},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.746},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.59099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72699999999999998,"15min":0.70299999999999996},"min":{"1min":0.65600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.69499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83099999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94899999999999995,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84099999999999997,"15min":0.81899999999999995},"min":{"1min":0.623,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.01,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.85499999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1036649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28299999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":1.48,"15min":1.48},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74199999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.72799999999999998,"15min":0.67300000000000004},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":10.412000000000001,"5min":10.412000000000001,"15min":10.412000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.70399999999999996,"15min":0.63500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":7.4909999999999997,"5min":7.4909999999999997,"15min":7.4909999999999997},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86099999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":4.8090000000000002,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74099999999999999,"15min":0.72899999999999998},"min":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.755},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.82899999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93799999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.89100000000000001,"15min":0.84699999999999998},"min":{"1min":0.61299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92500000000000004,"5min":10.394,"15min":10.394},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.94999999999999996,"15min":0.83999999999999997},"min":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.3310000000000004,"5min":10.452999999999999,"15min":10.452999999999999},"last":0.871}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.998,"5min":1.016,"15min":1.016},"last":0.79200000000000004}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.496,"15min":0.499},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.54300000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.504,"5min":0.877,"15min":0.877},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.57599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.65600000000000003},"min":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81799999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85999999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.624,"15min":0.64600000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.67900000000000005},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.085,"15min":1.085},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88600000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.76300000000000001,"15min":0.80300000000000005},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84499999999999997,"5min":1.726,"15min":1.726},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.80200000000000005,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85899999999999999,"5min":1.573,"15min":1.573},"last":0.81499999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.044,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79700000000000004,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.86099999999999999},"min":{"1min":0.497,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83199999999999996,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84299999999999997,"15min":0.86099999999999999},"min":{"1min":0.63900000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91300000000000003,"5min":1.238,"15min":1.238},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98299999999999998,"5min":1.304,"15min":1.304},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.90000000000000002,"15min":0.94499999999999995},"min":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.91300000000000003,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.86299999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.56599999999999995,"15min":0.498},"min":{"1min":0.498,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54500000000000004,"15min":0.47899999999999998},"min":{"1min":0.503,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60499999999999998,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.55900000000000005},"min":{"1min":0.50600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92800000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.627}]},{"osd":4,"last update":"Thu Jul 13 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.42099999999999999},"min":{"1min":0.41999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.55800000000000005},"min":{"1min":0.51400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.71799999999999997},"min":{"1min":0.58299999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.872,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":7,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.52500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.56200000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":0.98299999999999998,"5min":6.6470000000000002,"15min":6.6470000000000002},"last":0.69999999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.623,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.83699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":446137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61699999999999999,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":0.89600000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64400000000000002,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64700000000000002,"15min":0.58399999999999996},"min":{"1min":0.53100000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64600000000000002,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.495},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.69799999999999995,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77900000000000003,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68799999999999994},"min":{"1min":0.45400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0740000000000001,"5min":1.331,"15min":1.331},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.79400000000000004},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.097,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.94899999999999995,"15min":0.98399999999999999},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.871,"15min":0.90400000000000003},"min":{"1min":0.499,"5min":0.443,"15min":0.443},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.83699999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95099999999999996,"15min":1.0169999999999999},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99199999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.753},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.91000000000000003,"15min":0.95399999999999996},"min":{"1min":0.53200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0529999999999999,"5min":1.925,"15min":1.925},"last":0.876}]},{"osd":10,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88900000000000001,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.88700000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3100000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60799999999999998,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.3819999999999997,"5min":6.3819999999999997,"15min":6.3819999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57199999999999995,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.875,"15min":0.875},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.57099999999999995,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.65100000000000002,"15min":0.625},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80800000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.48099999999999998,"15min":0.433},"min":{"1min":0.38400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.625}]},{"osd":6,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96799999999999997,"5min":6.4219999999999997,"15min":6.4219999999999997},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70399999999999996,"15min":0.67100000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84099999999999997},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":4.21,"15min":4.21},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.64100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.84099999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.45800000000000002},"min":{"1min":0.35199999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.853,"5min":7.8940000000000001,"15min":7.8940000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61699999999999999,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92300000000000004,"5min":13.968999999999999,"15min":13.968999999999999},"last":0.374}]},{"osd":1,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.978,"5min":7.0460000000000003,"15min":7.0460000000000003},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.80800000000000005,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.67200000000000004,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80600000000000005,"15min":0.67300000000000004},"min":{"1min":0.59899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9399999999999999,"5min":7.1319999999999997,"15min":7.1319999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.78100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.157,"5min":0.80700000000000005,"15min":0.69799999999999995},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.9950000000000001,"5min":6.9950000000000001,"15min":6.9950000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.82699999999999996,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":13.731999999999999,"5min":13.731999999999999,"15min":13.731999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.879,"15min":0.75800000000000001},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.827,"5min":13.891,"15min":13.891},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.93700000000000006,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":13.797000000000001,"5min":13.797000000000001,"15min":13.797000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1699999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.80900000000000005}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.99099999999999999,"15min":0.94499999999999995},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0310000000000001,"5min":7.2709999999999999,"15min":7.2709999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.034,"15min":0.95299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.3599999999999999,"5min":13.930999999999999,"15min":13.930999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88800000000000001},"min":{"1min":0.59899999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.046,"15min":1.01},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":7.085,"5min":7.085,"15min":7.085},"last":0.84699999999999998}]},{"osd":11,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.86699999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":445781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":1.004,"15min":1.004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89300000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.71799999999999997,"15min":0.67300000000000004},"min":{"1min":0.60099999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.2699999999999996,"5min":4.2699999999999996,"15min":4.2699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88600000000000001,"5min":1.01,"15min":1.01},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1556480,"data_stored":1036423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.139,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.622},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2330000000000001,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.55300000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83099999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65500000000000003,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.29699999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.66900000000000004,"5min":1.665,"15min":1.665},"last":0.503}]},{"osd":3,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80600000000000005,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.60199999999999998},"min":{"1min":0.56799999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.51000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":1.288,"15min":1.288},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93899999999999995,"5min":3.823,"15min":3.823},"last":0.751}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86599999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.874},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89600000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.77500000000000002},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96699999999999997,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.75800000000000001},"min":{"1min":0.60199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.72099999999999997,"15min":0.68600000000000005},"min":{"1min":0.65800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.64000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:14:04.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:14:04.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:04 smithi121 ceph-mon[129187]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-13T22:14:04.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:04 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1376550313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:04.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:04 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1656325832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/1134204055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a40ffb00 msgr2=0x7fb9a40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/1134204055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a40ffb00 0x7fb9a40fff10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009990 tx=0x7fb9a0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/1134204055 shutdown_connections 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/1134204055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a40ffb00 0x7fb9a40fff10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/1134204055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a4101680 0x7fb9a4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/1134204055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 0x7fb9a4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/1134204055 >> 172.21.15.26:0/1134204055 conn(0x7fb9a40fb0e0 msgr2=0x7fb9a40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/1134204055 shutdown_connections 2023-07-13T22:14:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.053+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/1134204055 wait complete. 2023-07-13T22:14:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.054+0000 7fb9ac1ff700 1 Processor -- start 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.054+0000 7fb9ac1ff700 1 -- start start 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a40ffb00 0x7fb9a4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 0x7fb9a4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419f5f0 con 0x7fb9a4101680 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419f730 con 0x7fb9a40ffb00 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9ac1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a419f8c0 con 0x7fb9a4100cf0 2023-07-13T22:14:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45236/0 (socket says 172.21.15.26:45236) 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 -- 172.21.15.26:0/2317367558 learned_addr learned my addr 172.21.15.26:0/2317367558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9a979a700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 0x7fb9a4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9a9f9b700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a40ffb00 0x7fb9a4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 -- 172.21.15.26:0/2317367558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 msgr2=0x7fb9a4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 0x7fb9a4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 -- 172.21.15.26:0/2317367558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a40ffb00 msgr2=0x7fb9a4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a40ffb00 0x7fb9a4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.055+0000 7fb9aa79c700 1 -- 172.21.15.26:0/2317367558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00095f0 con 0x7fb9a4101680 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb9aa79c700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007ee0 tx=0x7fb99c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00dd20 con 0x7fb9a4101680 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a419fba0 con 0x7fb9a4101680 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99c00f040 con 0x7fb9a4101680 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0048a0 con 0x7fb9a4101680 2023-07-13T22:14:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.056+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a41a00b0 con 0x7fb9a4101680 2023-07-13T22:14:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.057+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a41921e0 con 0x7fb9a4101680 2023-07-13T22:14:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.059+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb99c003a00 con 0x7fb9a4101680 2023-07-13T22:14:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.060+0000 7fb99affd700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 0x7fb9900a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.060+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb99c0cc170 con 0x7fb9a4101680 2023-07-13T22:14:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.061+0000 7fb9a9f9b700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 0x7fb9900a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.061+0000 7fb9a9f9b700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 0x7fb9900a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b2e0 tx=0x7fb9a0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.064+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb99c04e790 con 0x7fb9a4101680 2023-07-13T22:14:05.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:04 smithi117 ceph-mon[129074]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-13T22:14:05.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:04 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1376550313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:05.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:04 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1656325832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:05.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:04 smithi026 ceph-mon[95647]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2023-07-13T22:14:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1376550313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:05.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:04 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1656325832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:05.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.269+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9a4060950 con 0x7fb9900a11a0 2023-07-13T22:14:05.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.280+0000 7fb99affd700 1 -- 172.21.15.26:0/2317367558 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345482 (secure 0 0 0) 0x7fb9a4060950 con 0x7fb9900a11a0 2023-07-13T22:14:05.274 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 msgr2=0x7fb9900a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 0x7fb9900a3650 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000b2e0 tx=0x7fb9a0005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 msgr2=0x7fb9a419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb99c007ee0 tx=0x7fb99c00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 shutdown_connections 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9900a11a0 0x7fb9900a3650 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9a4101680 0x7fb9a419f020 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9a40ffb00 0x7fb9a4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 --2- 172.21.15.26:0/2317367558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9a4100cf0 0x7fb9a4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 >> 172.21.15.26:0/2317367558 conn(0x7fb9a40fb0e0 msgr2=0x7fb9a41065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 shutdown_connections 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.285+0000 7fb9ac1ff700 1 -- 172.21.15.26:0/2317367558 wait complete. 2023-07-13T22:14:05.277 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:14:05.293 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2023-07-13T22:14:03.445930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659521,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1105,"num_read_kb":680,"num_write":424,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182248,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":850,"ondisk_log_size":850,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336424,"kb_used_data":13000,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605400,"statfs":{"total":1151940427776,"available":1151595929600,"internally_reserved":0,"allocated":13312000,"data_stored":7281891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5342,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":74,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002984"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461705+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:13:51.461705+0000","last_peered":"2023-07-13T22:13:51.461705+0000","last_clean":"2023-07-13T22:13:51.461705+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:13:51.461705+0000","last_undegraded":"2023-07-13T22:13:51.461705+0000","last_fullsized":"2023-07-13T22:13:51.461705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:47.548617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455808+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:13:51.455808+0000","last_peered":"2023-07-13T22:13:51.455808+0000","last_clean":"2023-07-13T22:13:51.455808+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:13:51.455808+0000","last_undegraded":"2023-07-13T22:13:51.455808+0000","last_fullsized":"2023-07-13T22:13:51.455808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:21.973731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453820+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:13:51.453820+0000","last_peered":"2023-07-13T22:13:51.453820+0000","last_clean":"2023-07-13T22:13:51.453820+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:13:51.453820+0000","last_undegraded":"2023-07-13T22:13:51.453820+0000","last_fullsized":"2023-07-13T22:13:51.453820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:51.412468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453453+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:13:51.453453+0000","last_peered":"2023-07-13T22:13:51.453453+0000","last_clean":"2023-07-13T22:13:51.453453+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:13:51.453453+0000","last_undegraded":"2023-07-13T22:13:51.453453+0000","last_fullsized":"2023-07-13T22:13:51.453453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:33.084972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464668+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:13:51.464668+0000","last_peered":"2023-07-13T22:13:51.464668+0000","last_clean":"2023-07-13T22:13:51.464668+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:13:51.464668+0000","last_undegraded":"2023-07-13T22:13:51.464668+0000","last_fullsized":"2023-07-13T22:13:51.464668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:14:33.505950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453475+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:13:51.453475+0000","last_peered":"2023-07-13T22:13:51.453475+0000","last_clean":"2023-07-13T22:13:51.453475+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:13:51.453475+0000","last_undegraded":"2023-07-13T22:13:51.453475+0000","last_fullsized":"2023-07-13T22:13:51.453475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:04.653956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.444154+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:13:51.444154+0000","last_peered":"2023-07-13T22:13:51.444154+0000","last_clean":"2023-07-13T22:13:51.444154+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:13:51.444154+0000","last_undegraded":"2023-07-13T22:13:51.444154+0000","last_fullsized":"2023-07-13T22:13:51.444154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:06.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465428+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:13:51.465428+0000","last_peered":"2023-07-13T22:13:51.465428+0000","last_clean":"2023-07-13T22:13:51.465428+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:13:51.465428+0000","last_undegraded":"2023-07-13T22:13:51.465428+0000","last_fullsized":"2023-07-13T22:13:51.465428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:49.982151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464318+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:13:51.464318+0000","last_peered":"2023-07-13T22:13:51.464318+0000","last_clean":"2023-07-13T22:13:51.464318+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:13:51.464318+0000","last_undegraded":"2023-07-13T22:13:51.464318+0000","last_fullsized":"2023-07-13T22:13:51.464318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.458063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456141+0000","last_change":"2023-07-13T22:12:52.546759+0000","last_active":"2023-07-13T22:13:51.456141+0000","last_peered":"2023-07-13T22:13:51.456141+0000","last_clean":"2023-07-13T22:13:51.456141+0000","last_became_active":"2023-07-13T22:12:52.546378+0000","last_became_peered":"2023-07-13T22:12:52.546378+0000","last_unstale":"2023-07-13T22:13:51.456141+0000","last_undegraded":"2023-07-13T22:13:51.456141+0000","last_fullsized":"2023-07-13T22:13:51.456141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:38.494158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.450988+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:13:51.450988+0000","last_peered":"2023-07-13T22:13:51.450988+0000","last_clean":"2023-07-13T22:13:51.450988+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:13:51.450988+0000","last_undegraded":"2023-07-13T22:13:51.450988+0000","last_fullsized":"2023-07-13T22:13:51.450988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:47.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.454081+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:13:51.454081+0000","last_peered":"2023-07-13T22:13:51.454081+0000","last_clean":"2023-07-13T22:13:51.454081+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:13:51.454081+0000","last_undegraded":"2023-07-13T22:13:51.454081+0000","last_fullsized":"2023-07-13T22:13:51.454081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:55.880996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465068+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:13:51.465068+0000","last_peered":"2023-07-13T22:13:51.465068+0000","last_clean":"2023-07-13T22:13:51.465068+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:13:51.465068+0000","last_undegraded":"2023-07-13T22:13:51.465068+0000","last_fullsized":"2023-07-13T22:13:51.465068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:58.155432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457726+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:13:51.457726+0000","last_peered":"2023-07-13T22:13:51.457726+0000","last_clean":"2023-07-13T22:13:51.457726+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:13:51.457726+0000","last_undegraded":"2023-07-13T22:13:51.457726+0000","last_fullsized":"2023-07-13T22:13:51.457726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:57.803872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445655+0000","last_change":"2023-07-13T22:12:52.549238+0000","last_active":"2023-07-13T22:13:51.445655+0000","last_peered":"2023-07-13T22:13:51.445655+0000","last_clean":"2023-07-13T22:13:51.445655+0000","last_became_active":"2023-07-13T22:12:52.548923+0000","last_became_peered":"2023-07-13T22:12:52.548923+0000","last_unstale":"2023-07-13T22:13:51.445655+0000","last_undegraded":"2023-07-13T22:13:51.445655+0000","last_fullsized":"2023-07-13T22:13:51.445655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:08.925620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459735+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:13:51.459735+0000","last_peered":"2023-07-13T22:13:51.459735+0000","last_clean":"2023-07-13T22:13:51.459735+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:13:51.459735+0000","last_undegraded":"2023-07-13T22:13:51.459735+0000","last_fullsized":"2023-07-13T22:13:51.459735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:23.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461935+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:13:51.461935+0000","last_peered":"2023-07-13T22:13:51.461935+0000","last_clean":"2023-07-13T22:13:51.461935+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:13:51.461935+0000","last_undegraded":"2023-07-13T22:13:51.461935+0000","last_fullsized":"2023-07-13T22:13:51.461935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:37.216203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463124+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:13:51.463124+0000","last_peered":"2023-07-13T22:13:51.463124+0000","last_clean":"2023-07-13T22:13:51.463124+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:13:51.463124+0000","last_undegraded":"2023-07-13T22:13:51.463124+0000","last_fullsized":"2023-07-13T22:13:51.463124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:18.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456310+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:13:51.456310+0000","last_peered":"2023-07-13T22:13:51.456310+0000","last_clean":"2023-07-13T22:13:51.456310+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:13:51.456310+0000","last_undegraded":"2023-07-13T22:13:51.456310+0000","last_fullsized":"2023-07-13T22:13:51.456310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:56.130000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":125,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:57.616438+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:13:57.616438+0000","last_peered":"2023-07-13T22:13:57.616438+0000","last_clean":"2023-07-13T22:13:57.616438+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:13:57.616438+0000","last_undegraded":"2023-07-13T22:13:57.616438+0000","last_fullsized":"2023-07-13T22:13:57.616438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:28:47.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":94,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456796+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:13:51.456796+0000","last_peered":"2023-07-13T22:13:51.456796+0000","last_clean":"2023-07-13T22:13:51.456796+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:13:51.456796+0000","last_undegraded":"2023-07-13T22:13:51.456796+0000","last_fullsized":"2023-07-13T22:13:51.456796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:42:35.256971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791902+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:52.791902+0000","last_peered":"2023-07-13T22:12:52.791902+0000","last_clean":"2023-07-13T22:12:52.791902+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:52.791902+0000","last_undegraded":"2023-07-13T22:12:52.791902+0000","last_fullsized":"2023-07-13T22:12:52.791902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:29.131604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464536+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:13:51.464536+0000","last_peered":"2023-07-13T22:13:51.464536+0000","last_clean":"2023-07-13T22:13:51.464536+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:13:51.464536+0000","last_undegraded":"2023-07-13T22:13:51.464536+0000","last_fullsized":"2023-07-13T22:13:51.464536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:53.951535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.554410+0000","last_change":"2023-07-13T22:12:52.554410+0000","last_active":"2023-07-13T22:12:52.554410+0000","last_peered":"2023-07-13T22:12:52.554410+0000","last_clean":"2023-07-13T22:12:52.554410+0000","last_became_active":"2023-07-13T22:12:52.541864+0000","last_became_peered":"2023-07-13T22:12:52.541864+0000","last_unstale":"2023-07-13T22:12:52.554410+0000","last_undegraded":"2023-07-13T22:12:52.554410+0000","last_fullsized":"2023-07-13T22:12:52.554410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:48:29.339493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463743+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:13:51.463743+0000","last_peered":"2023-07-13T22:13:51.463743+0000","last_clean":"2023-07-13T22:13:51.463743+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:13:51.463743+0000","last_undegraded":"2023-07-13T22:13:51.463743+0000","last_fullsized":"2023-07-13T22:13:51.463743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:58.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542998+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:52.542998+0000","last_peered":"2023-07-13T22:12:52.542998+0000","last_clean":"2023-07-13T22:12:52.542998+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:52.542998+0000","last_undegraded":"2023-07-13T22:12:52.542998+0000","last_fullsized":"2023-07-13T22:12:52.542998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:22:28.850188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457222+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:13:51.457222+0000","last_peered":"2023-07-13T22:13:51.457222+0000","last_clean":"2023-07-13T22:13:51.457222+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:13:51.457222+0000","last_undegraded":"2023-07-13T22:13:51.457222+0000","last_fullsized":"2023-07-13T22:13:51.457222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:21.578829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":123,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572636+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:14:00.572636+0000","last_peered":"2023-07-13T22:14:00.572636+0000","last_clean":"2023-07-13T22:14:00.572636+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:14:00.572636+0000","last_undegraded":"2023-07-13T22:14:00.572636+0000","last_fullsized":"2023-07-13T22:14:00.572636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:56:16.055459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.441362+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:13:51.441362+0000","last_peered":"2023-07-13T22:13:51.441362+0000","last_clean":"2023-07-13T22:13:51.441362+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:13:51.441362+0000","last_undegraded":"2023-07-13T22:13:51.441362+0000","last_fullsized":"2023-07-13T22:13:51.441362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:28.566300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538988+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:52.538988+0000","last_peered":"2023-07-13T22:12:52.538988+0000","last_clean":"2023-07-13T22:12:52.538988+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:52.538988+0000","last_undegraded":"2023-07-13T22:12:52.538988+0000","last_fullsized":"2023-07-13T22:12:52.538988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:01:40.167436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465176+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:13:51.465176+0000","last_peered":"2023-07-13T22:13:51.465176+0000","last_clean":"2023-07-13T22:13:51.465176+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:13:51.465176+0000","last_undegraded":"2023-07-13T22:13:51.465176+0000","last_fullsized":"2023-07-13T22:13:51.465176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:03.123538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537975+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:52.537975+0000","last_peered":"2023-07-13T22:12:52.537975+0000","last_clean":"2023-07-13T22:12:52.537975+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:52.537975+0000","last_undegraded":"2023-07-13T22:12:52.537975+0000","last_fullsized":"2023-07-13T22:12:52.537975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451163+0000","last_change":"2023-07-13T22:12:52.546725+0000","last_active":"2023-07-13T22:13:51.451163+0000","last_peered":"2023-07-13T22:13:51.451163+0000","last_clean":"2023-07-13T22:13:51.451163+0000","last_became_active":"2023-07-13T22:12:52.546385+0000","last_became_peered":"2023-07-13T22:12:52.546385+0000","last_unstale":"2023-07-13T22:13:51.451163+0000","last_undegraded":"2023-07-13T22:13:51.451163+0000","last_fullsized":"2023-07-13T22:13:51.451163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:18.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534851+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:52.534851+0000","last_peered":"2023-07-13T22:12:52.534851+0000","last_clean":"2023-07-13T22:12:52.534851+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:52.534851+0000","last_undegraded":"2023-07-13T22:12:52.534851+0000","last_fullsized":"2023-07-13T22:12:52.534851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:06.618468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457170+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:13:51.457170+0000","last_peered":"2023-07-13T22:13:51.457170+0000","last_clean":"2023-07-13T22:13:51.457170+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:13:51.457170+0000","last_undegraded":"2023-07-13T22:13:51.457170+0000","last_fullsized":"2023-07-13T22:13:51.457170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:26.423367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.573478+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:14:01.573478+0000","last_peered":"2023-07-13T22:14:01.573478+0000","last_clean":"2023-07-13T22:14:01.573478+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:14:01.573478+0000","last_undegraded":"2023-07-13T22:14:01.573478+0000","last_fullsized":"2023-07-13T22:14:01.573478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:37.488363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459625+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:13:51.459625+0000","last_peered":"2023-07-13T22:13:51.459625+0000","last_clean":"2023-07-13T22:13:51.459625+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:13:51.459625+0000","last_undegraded":"2023-07-13T22:13:51.459625+0000","last_fullsized":"2023-07-13T22:13:51.459625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:47.438310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541598+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:52.541598+0000","last_peered":"2023-07-13T22:12:52.541598+0000","last_clean":"2023-07-13T22:12:52.541598+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:52.541598+0000","last_undegraded":"2023-07-13T22:12:52.541598+0000","last_fullsized":"2023-07-13T22:12:52.541598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:00.126919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458115+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:13:51.458115+0000","last_peered":"2023-07-13T22:13:51.458115+0000","last_clean":"2023-07-13T22:13:51.458115+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:13:51.458115+0000","last_undegraded":"2023-07-13T22:13:51.458115+0000","last_fullsized":"2023-07-13T22:13:51.458115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:50.338306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535374+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:52.535374+0000","last_peered":"2023-07-13T22:12:52.535374+0000","last_clean":"2023-07-13T22:12:52.535374+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:52.535374+0000","last_undegraded":"2023-07-13T22:12:52.535374+0000","last_fullsized":"2023-07-13T22:12:52.535374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:36.251918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462201+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:13:51.462201+0000","last_peered":"2023-07-13T22:13:51.462201+0000","last_clean":"2023-07-13T22:13:51.462201+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:13:51.462201+0000","last_undegraded":"2023-07-13T22:13:51.462201+0000","last_fullsized":"2023-07-13T22:13:51.462201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:23.829222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":257,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574528+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:14:01.574528+0000","last_peered":"2023-07-13T22:14:01.574528+0000","last_clean":"2023-07-13T22:14:01.574528+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:14:01.574528+0000","last_undegraded":"2023-07-13T22:14:01.574528+0000","last_fullsized":"2023-07-13T22:14:01.574528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:54:07.569446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61830,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":157,"num_write":36,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445202+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:13:51.445202+0000","last_peered":"2023-07-13T22:13:51.445202+0000","last_clean":"2023-07-13T22:13:51.445202+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:13:51.445202+0000","last_undegraded":"2023-07-13T22:13:51.445202+0000","last_fullsized":"2023-07-13T22:13:51.445202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:19.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540584+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:52.540584+0000","last_peered":"2023-07-13T22:12:52.540584+0000","last_clean":"2023-07-13T22:12:52.540584+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:52.540584+0000","last_undegraded":"2023-07-13T22:12:52.540584+0000","last_fullsized":"2023-07-13T22:12:52.540584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:23.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466045+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:13:51.466045+0000","last_peered":"2023-07-13T22:13:51.466045+0000","last_clean":"2023-07-13T22:13:51.466045+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:13:51.466045+0000","last_undegraded":"2023-07-13T22:13:51.466045+0000","last_fullsized":"2023-07-13T22:13:51.466045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:37.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534895+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:52.534895+0000","last_peered":"2023-07-13T22:12:52.534895+0000","last_clean":"2023-07-13T22:12:52.534895+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:52.534895+0000","last_undegraded":"2023-07-13T22:12:52.534895+0000","last_fullsized":"2023-07-13T22:12:52.534895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:03.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462642+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:13:51.462642+0000","last_peered":"2023-07-13T22:13:51.462642+0000","last_clean":"2023-07-13T22:13:51.462642+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:13:51.462642+0000","last_undegraded":"2023-07-13T22:13:51.462642+0000","last_fullsized":"2023-07-13T22:13:51.462642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:29.680386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574264+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:14:01.574264+0000","last_peered":"2023-07-13T22:14:01.574264+0000","last_clean":"2023-07-13T22:14:01.574264+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:14:01.574264+0000","last_undegraded":"2023-07-13T22:14:01.574264+0000","last_fullsized":"2023-07-13T22:14:01.574264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:24.354790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.449957+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:13:51.449957+0000","last_peered":"2023-07-13T22:13:51.449957+0000","last_clean":"2023-07-13T22:13:51.449957+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:13:51.449957+0000","last_undegraded":"2023-07-13T22:13:51.449957+0000","last_fullsized":"2023-07-13T22:13:51.449957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:39.397611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541703+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:52.541703+0000","last_peered":"2023-07-13T22:12:52.541703+0000","last_clean":"2023-07-13T22:12:52.541703+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:52.541703+0000","last_undegraded":"2023-07-13T22:12:52.541703+0000","last_fullsized":"2023-07-13T22:12:52.541703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:54.534152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456704+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:13:51.456704+0000","last_peered":"2023-07-13T22:13:51.456704+0000","last_clean":"2023-07-13T22:13:51.456704+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:13:51.456704+0000","last_undegraded":"2023-07-13T22:13:51.456704+0000","last_fullsized":"2023-07-13T22:13:51.456704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:18:37.092789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543636+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:52.543636+0000","last_peered":"2023-07-13T22:12:52.543636+0000","last_clean":"2023-07-13T22:12:52.543636+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:52.543636+0000","last_undegraded":"2023-07-13T22:12:52.543636+0000","last_fullsized":"2023-07-13T22:12:52.543636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:29:05.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456758+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:13:51.456758+0000","last_peered":"2023-07-13T22:13:51.456758+0000","last_clean":"2023-07-13T22:13:51.456758+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:13:51.456758+0000","last_undegraded":"2023-07-13T22:13:51.456758+0000","last_fullsized":"2023-07-13T22:13:51.456758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:53:20.274354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539898+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:52.539898+0000","last_peered":"2023-07-13T22:12:52.539898+0000","last_clean":"2023-07-13T22:12:52.539898+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:52.539898+0000","last_undegraded":"2023-07-13T22:12:52.539898+0000","last_fullsized":"2023-07-13T22:12:52.539898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:26.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465891+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:13:51.465891+0000","last_peered":"2023-07-13T22:13:51.465891+0000","last_clean":"2023-07-13T22:13:51.465891+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:13:51.465891+0000","last_undegraded":"2023-07-13T22:13:51.465891+0000","last_fullsized":"2023-07-13T22:13:51.465891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:02.628236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":116,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:56.615650+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:13:56.615650+0000","last_peered":"2023-07-13T22:13:56.615650+0000","last_clean":"2023-07-13T22:13:56.615650+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:13:56.615650+0000","last_undegraded":"2023-07-13T22:13:56.615650+0000","last_fullsized":"2023-07-13T22:13:56.615650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:22.807561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453734+0000","last_change":"2023-07-13T22:12:52.546608+0000","last_active":"2023-07-13T22:13:51.453734+0000","last_peered":"2023-07-13T22:13:51.453734+0000","last_clean":"2023-07-13T22:13:51.453734+0000","last_became_active":"2023-07-13T22:12:52.546240+0000","last_became_peered":"2023-07-13T22:12:52.546240+0000","last_unstale":"2023-07-13T22:13:51.453734+0000","last_undegraded":"2023-07-13T22:13:51.453734+0000","last_fullsized":"2023-07-13T22:13:51.453734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:11.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456809+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:13:51.456809+0000","last_peered":"2023-07-13T22:13:51.456809+0000","last_clean":"2023-07-13T22:13:51.456809+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:13:51.456809+0000","last_undegraded":"2023-07-13T22:13:51.456809+0000","last_fullsized":"2023-07-13T22:13:51.456809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:05.891842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572319+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:14:00.572319+0000","last_peered":"2023-07-13T22:14:00.572319+0000","last_clean":"2023-07-13T22:14:00.572319+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:14:00.572319+0000","last_undegraded":"2023-07-13T22:14:00.572319+0000","last_fullsized":"2023-07-13T22:14:00.572319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:09.930750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464940+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:13:51.464940+0000","last_peered":"2023-07-13T22:13:51.464940+0000","last_clean":"2023-07-13T22:13:51.464940+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:13:51.464940+0000","last_undegraded":"2023-07-13T22:13:51.464940+0000","last_fullsized":"2023-07-13T22:13:51.464940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:42:38.987941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539678+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:52.539678+0000","last_peered":"2023-07-13T22:12:52.539678+0000","last_clean":"2023-07-13T22:12:52.539678+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:52.539678+0000","last_undegraded":"2023-07-13T22:12:52.539678+0000","last_fullsized":"2023-07-13T22:12:52.539678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:49.259253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540813+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:52.540813+0000","last_peered":"2023-07-13T22:12:52.540813+0000","last_clean":"2023-07-13T22:12:52.540813+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:52.540813+0000","last_undegraded":"2023-07-13T22:12:52.540813+0000","last_fullsized":"2023-07-13T22:12:52.540813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:11:16.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461027+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:13:51.461027+0000","last_peered":"2023-07-13T22:13:51.461027+0000","last_clean":"2023-07-13T22:13:51.461027+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:13:51.461027+0000","last_undegraded":"2023-07-13T22:13:51.461027+0000","last_fullsized":"2023-07-13T22:13:51.461027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:45.725233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541086+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:52.541086+0000","last_peered":"2023-07-13T22:12:52.541086+0000","last_clean":"2023-07-13T22:12:52.541086+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:52.541086+0000","last_undegraded":"2023-07-13T22:12:52.541086+0000","last_fullsized":"2023-07-13T22:12:52.541086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:50.437984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543196+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:52.543196+0000","last_peered":"2023-07-13T22:12:52.543196+0000","last_clean":"2023-07-13T22:12:52.543196+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:52.543196+0000","last_undegraded":"2023-07-13T22:12:52.543196+0000","last_fullsized":"2023-07-13T22:12:52.543196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:37.229195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455688+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:13:51.455688+0000","last_peered":"2023-07-13T22:13:51.455688+0000","last_clean":"2023-07-13T22:13:51.455688+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:13:51.455688+0000","last_undegraded":"2023-07-13T22:13:51.455688+0000","last_fullsized":"2023-07-13T22:13:51.455688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:01.886536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'11","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540162+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:12:52.540162+0000","last_peered":"2023-07-13T22:12:52.540162+0000","last_clean":"2023-07-13T22:12:52.540162+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:12:52.540162+0000","last_undegraded":"2023-07-13T22:12:52.540162+0000","last_fullsized":"2023-07-13T22:12:52.540162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:27.067307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459700+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:13:51.459700+0000","last_peered":"2023-07-13T22:13:51.459700+0000","last_clean":"2023-07-13T22:13:51.459700+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:13:51.459700+0000","last_undegraded":"2023-07-13T22:13:51.459700+0000","last_fullsized":"2023-07-13T22:13:51.459700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:24.414406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:15.337161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466111+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:13:51.466111+0000","last_peered":"2023-07-13T22:13:51.466111+0000","last_clean":"2023-07-13T22:13:51.466111+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:13:51.466111+0000","last_undegraded":"2023-07-13T22:13:51.466111+0000","last_fullsized":"2023-07-13T22:13:51.466111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:39.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.551910+0000","last_change":"2023-07-13T22:12:52.551910+0000","last_active":"2023-07-13T22:12:52.551910+0000","last_peered":"2023-07-13T22:12:52.551910+0000","last_clean":"2023-07-13T22:12:52.551910+0000","last_became_active":"2023-07-13T22:12:52.548681+0000","last_became_peered":"2023-07-13T22:12:52.548681+0000","last_unstale":"2023-07-13T22:12:52.551910+0000","last_undegraded":"2023-07-13T22:12:52.551910+0000","last_fullsized":"2023-07-13T22:12:52.551910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:54:10.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.422697+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:13:51.422697+0000","last_peered":"2023-07-13T22:13:51.422697+0000","last_clean":"2023-07-13T22:13:51.422697+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:13:51.422697+0000","last_undegraded":"2023-07-13T22:13:51.422697+0000","last_fullsized":"2023-07-13T22:13:51.422697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:53.309826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791341+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:52.791341+0000","last_peered":"2023-07-13T22:12:52.791341+0000","last_clean":"2023-07-13T22:12:52.791341+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:52.791341+0000","last_undegraded":"2023-07-13T22:12:52.791341+0000","last_fullsized":"2023-07-13T22:12:52.791341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:31.189715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445763+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:13:51.445763+0000","last_peered":"2023-07-13T22:13:51.445763+0000","last_clean":"2023-07-13T22:13:51.445763+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:13:51.445763+0000","last_undegraded":"2023-07-13T22:13:51.445763+0000","last_fullsized":"2023-07-13T22:13:51.445763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:22.391535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540832+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:52.540832+0000","last_peered":"2023-07-13T22:12:52.540832+0000","last_clean":"2023-07-13T22:12:52.540832+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:52.540832+0000","last_undegraded":"2023-07-13T22:12:52.540832+0000","last_fullsized":"2023-07-13T22:12:52.540832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:34.701191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465349+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:13:51.465349+0000","last_peered":"2023-07-13T22:13:51.465349+0000","last_clean":"2023-07-13T22:13:51.465349+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:13:51.465349+0000","last_undegraded":"2023-07-13T22:13:51.465349+0000","last_fullsized":"2023-07-13T22:13:51.465349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:26:29.278158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542783+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:52.542783+0000","last_peered":"2023-07-13T22:12:52.542783+0000","last_clean":"2023-07-13T22:12:52.542783+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:52.542783+0000","last_undegraded":"2023-07-13T22:12:52.542783+0000","last_fullsized":"2023-07-13T22:12:52.542783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:04:20.180883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461401+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:13:51.461401+0000","last_peered":"2023-07-13T22:13:51.461401+0000","last_clean":"2023-07-13T22:13:51.461401+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:13:51.461401+0000","last_undegraded":"2023-07-13T22:13:51.461401+0000","last_fullsized":"2023-07-13T22:13:51.461401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:53.859029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":94,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.662195+0000","last_change":"2023-07-13T22:12:52.555796+0000","last_active":"2023-07-13T22:14:01.662195+0000","last_peered":"2023-07-13T22:14:01.662195+0000","last_clean":"2023-07-13T22:14:01.662195+0000","last_became_active":"2023-07-13T22:12:52.548664+0000","last_became_peered":"2023-07-13T22:12:52.548664+0000","last_unstale":"2023-07-13T22:14:01.662195+0000","last_undegraded":"2023-07-13T22:14:01.662195+0000","last_fullsized":"2023-07-13T22:14:01.662195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:37.846269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452434+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:13:51.452434+0000","last_peered":"2023-07-13T22:13:51.452434+0000","last_clean":"2023-07-13T22:13:51.452434+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:13:51.452434+0000","last_undegraded":"2023-07-13T22:13:51.452434+0000","last_fullsized":"2023-07-13T22:13:51.452434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:58.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535107+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:52.535107+0000","last_peered":"2023-07-13T22:12:52.535107+0000","last_clean":"2023-07-13T22:12:52.535107+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:52.535107+0000","last_undegraded":"2023-07-13T22:12:52.535107+0000","last_fullsized":"2023-07-13T22:12:52.535107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:29.092920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463990+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:13:51.463990+0000","last_peered":"2023-07-13T22:13:51.463990+0000","last_clean":"2023-07-13T22:13:51.463990+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:13:51.463990+0000","last_undegraded":"2023-07-13T22:13:51.463990+0000","last_fullsized":"2023-07-13T22:13:51.463990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:10.233101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538994+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:52.538994+0000","last_peered":"2023-07-13T22:12:52.538994+0000","last_clean":"2023-07-13T22:12:52.538994+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:52.538994+0000","last_undegraded":"2023-07-13T22:12:52.538994+0000","last_fullsized":"2023-07-13T22:12:52.538994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:31.553022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452262+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:13:51.452262+0000","last_peered":"2023-07-13T22:13:51.452262+0000","last_clean":"2023-07-13T22:13:51.452262+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:13:51.452262+0000","last_undegraded":"2023-07-13T22:13:51.452262+0000","last_fullsized":"2023-07-13T22:13:51.452262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:27.141696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448083+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:13:51.448083+0000","last_peered":"2023-07-13T22:13:51.448083+0000","last_clean":"2023-07-13T22:13:51.448083+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:13:51.448083+0000","last_undegraded":"2023-07-13T22:13:51.448083+0000","last_fullsized":"2023-07-13T22:13:51.448083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:12.051633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451991+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:13:51.451991+0000","last_peered":"2023-07-13T22:13:51.451991+0000","last_clean":"2023-07-13T22:13:51.451991+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:13:51.451991+0000","last_undegraded":"2023-07-13T22:13:51.451991+0000","last_fullsized":"2023-07-13T22:13:51.451991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:04:24.267514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448875+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:13:51.448875+0000","last_peered":"2023-07-13T22:13:51.448875+0000","last_clean":"2023-07-13T22:13:51.448875+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:13:51.448875+0000","last_undegraded":"2023-07-13T22:13:51.448875+0000","last_fullsized":"2023-07-13T22:13:51.448875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:21.951262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461338+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:13:51.461338+0000","last_peered":"2023-07-13T22:13:51.461338+0000","last_clean":"2023-07-13T22:13:51.461338+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:13:51.461338+0000","last_undegraded":"2023-07-13T22:13:51.461338+0000","last_fullsized":"2023-07-13T22:13:51.461338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:45.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461520+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:13:51.461520+0000","last_peered":"2023-07-13T22:13:51.461520+0000","last_clean":"2023-07-13T22:13:51.461520+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:13:51.461520+0000","last_undegraded":"2023-07-13T22:13:51.461520+0000","last_fullsized":"2023-07-13T22:13:51.461520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:36.995817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.447741+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:13:51.447741+0000","last_peered":"2023-07-13T22:13:51.447741+0000","last_clean":"2023-07-13T22:13:51.447741+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:13:51.447741+0000","last_undegraded":"2023-07-13T22:13:51.447741+0000","last_fullsized":"2023-07-13T22:13:51.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:14.395567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461321+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:13:51.461321+0000","last_peered":"2023-07-13T22:13:51.461321+0000","last_clean":"2023-07-13T22:13:51.461321+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:13:51.461321+0000","last_undegraded":"2023-07-13T22:13:51.461321+0000","last_fullsized":"2023-07-13T22:13:51.461321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:32.074362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459380+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:13:51.459380+0000","last_peered":"2023-07-13T22:13:51.459380+0000","last_clean":"2023-07-13T22:13:51.459380+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:13:51.459380+0000","last_undegraded":"2023-07-13T22:13:51.459380+0000","last_fullsized":"2023-07-13T22:13:51.459380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:16:17.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458725+0000","last_change":"2023-07-13T22:12:52.793042+0000","last_active":"2023-07-13T22:13:51.458725+0000","last_peered":"2023-07-13T22:13:51.458725+0000","last_clean":"2023-07-13T22:13:51.458725+0000","last_became_active":"2023-07-13T22:12:52.792625+0000","last_became_peered":"2023-07-13T22:12:52.792625+0000","last_unstale":"2023-07-13T22:13:51.458725+0000","last_undegraded":"2023-07-13T22:13:51.458725+0000","last_fullsized":"2023-07-13T22:13:51.458725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:37.356733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459347+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:13:51.459347+0000","last_peered":"2023-07-13T22:13:51.459347+0000","last_clean":"2023-07-13T22:13:51.459347+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:13:51.459347+0000","last_undegraded":"2023-07-13T22:13:51.459347+0000","last_fullsized":"2023-07-13T22:13:51.459347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:38.425408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466615+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:13:51.466615+0000","last_peered":"2023-07-13T22:13:51.466615+0000","last_clean":"2023-07-13T22:13:51.466615+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:13:51.466615+0000","last_undegraded":"2023-07-13T22:13:51.466615+0000","last_fullsized":"2023-07-13T22:13:51.466615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:38.511996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465842+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:13:51.465842+0000","last_peered":"2023-07-13T22:13:51.465842+0000","last_clean":"2023-07-13T22:13:51.465842+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:13:51.465842+0000","last_undegraded":"2023-07-13T22:13:51.465842+0000","last_fullsized":"2023-07-13T22:13:51.465842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:16.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458793+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:13:51.458793+0000","last_peered":"2023-07-13T22:13:51.458793+0000","last_clean":"2023-07-13T22:13:51.458793+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:13:51.458793+0000","last_undegraded":"2023-07-13T22:13:51.458793+0000","last_fullsized":"2023-07-13T22:13:51.458793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:02.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68907,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":979,"num_read_kb":571,"num_write":241,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1512,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":197765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":445151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":1.093,"15min":1.093},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":1.357,"15min":1.357},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.63500000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91900000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.75700000000000001}]},{"osd":8,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67900000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99299999999999999,"5min":1.335,"15min":1.335},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61299999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88900000000000001,"5min":1.23,"15min":1.23},"last":0.78100000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678054,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1092849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.54700000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.81299999999999994,"15min":0.83099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.9289999999999998,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.85499999999999998,"15min":0.89100000000000001},"min":{"1min":0.49399999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.47899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82499999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.90300000000000002,"15min":0.93400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5980000000000001,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.7050000000000001,"5min":5.234,"15min":5.234},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":1.117,"15min":1.117},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.497,"15min":0.497},"max":{"1min":0.91100000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94399999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96399999999999997}]},{"osd":11,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.754,"15min":0.752},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.56100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.89800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34000000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.5339999999999998,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.68100000000000005,"15min":0.68899999999999995},"min":{"1min":0.41199999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82699999999999996,"5min":1.109,"15min":1.109},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96499999999999997,"5min":1.101,"15min":1.101},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85199999999999998,"15min":0.85799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":3.0649999999999999,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.749},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.018,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.80300000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.87,"15min":0.875},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.90200000000000002,"15min":0.92600000000000005},"min":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.008,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.90600000000000003,"15min":0.93600000000000005},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89000000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":0.97599999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.747}]},{"osd":7,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.94199999999999995,"15min":0.95199999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.605,"5min":2.605,"15min":2.605},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.93899999999999995,"15min":0.95599999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.048,"5min":1.581,"15min":1.581},"last":0.69499999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":0.999},"min":{"1min":0.58599999999999997,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.96499999999999997,"15min":0.96699999999999997},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.83499999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.78000000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84299999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":444660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.48199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82799999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.44600000000000001},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.47799999999999998,"15min":0.45400000000000001},"min":{"1min":0.372,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39500000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.624},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.746},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996}]},{"osd":6,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.59099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72699999999999998,"15min":0.70299999999999996},"min":{"1min":0.65600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.69499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83099999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94899999999999995,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.84099999999999997,"15min":0.81899999999999995},"min":{"1min":0.623,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.01,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.85499999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.75800000000000001}]},{"osd":11,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.873}]}]},{"osd":7,"up_from":51,"seq":219043332146,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1036649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28299999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":1.48,"15min":1.48},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74199999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.72799999999999998,"15min":0.67300000000000004},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":10.412000000000001,"5min":10.412000000000001,"15min":10.412000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.70399999999999996,"15min":0.63500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":7.4909999999999997,"5min":7.4909999999999997,"15min":7.4909999999999997},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86099999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":4.8090000000000002,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74099999999999999,"15min":0.72899999999999998},"min":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.755},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.82899999999999996}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93799999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.84299999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.89100000000000001,"15min":0.84699999999999998},"min":{"1min":0.61299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92500000000000004,"5min":10.394,"15min":10.394},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.94999999999999996,"15min":0.83999999999999997},"min":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.3310000000000004,"5min":10.452999999999999,"15min":10.452999999999999},"last":0.871}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.998,"5min":1.016,"15min":1.016},"last":0.79200000000000004}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.88600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.496,"15min":0.499},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.54300000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.504,"5min":0.877,"15min":0.877},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.57599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.65600000000000003},"min":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81799999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85999999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.624,"15min":0.64600000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.67900000000000005},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.085,"15min":1.085},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88600000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.76300000000000001,"15min":0.80300000000000005},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84499999999999997,"5min":1.726,"15min":1.726},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.80200000000000005,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85899999999999999,"5min":1.573,"15min":1.573},"last":0.81499999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.044,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79700000000000004,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.86099999999999999},"min":{"1min":0.497,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83199999999999996,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84299999999999997,"15min":0.86099999999999999},"min":{"1min":0.63900000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91300000000000003,"5min":1.238,"15min":1.238},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98299999999999998,"5min":1.304,"15min":1.304},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.90000000000000002,"15min":0.94499999999999995},"min":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.91300000000000003,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.86299999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":502058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.40400000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55900000000000005,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88700000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.56599999999999995,"15min":0.498},"min":{"1min":0.498,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54500000000000004,"15min":0.47899999999999998},"min":{"1min":0.503,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60499999999999998,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.55900000000000005},"min":{"1min":0.50600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92800000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.627}]},{"osd":4,"last update":"Thu Jul 13 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.42099999999999999},"min":{"1min":0.41999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.55800000000000005},"min":{"1min":0.51400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.71799999999999997},"min":{"1min":0.58299999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.872,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75}]},{"osd":7,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.52500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.56200000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":0.98299999999999998,"5min":6.6470000000000002,"15min":6.6470000000000002},"last":0.69999999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.78300000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.623,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.94699999999999995}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.83699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":446137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61699999999999999,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":0.89600000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64400000000000002,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64700000000000002,"15min":0.58399999999999996},"min":{"1min":0.53100000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64600000000000002,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.495},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.69799999999999995,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77900000000000003,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68799999999999994},"min":{"1min":0.45400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0740000000000001,"5min":1.331,"15min":1.331},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.79400000000000004},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.097,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.94899999999999995,"15min":0.98399999999999999},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.871,"15min":0.90400000000000003},"min":{"1min":0.499,"5min":0.443,"15min":0.443},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.83699999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95099999999999996,"15min":1.0169999999999999},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99199999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.753},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.91000000000000003,"15min":0.95399999999999996},"min":{"1min":0.53200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0529999999999999,"5min":1.925,"15min":1.925},"last":0.876}]},{"osd":10,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88900000000000001,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.88700000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3100000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":500439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60799999999999998,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.3819999999999997,"5min":6.3819999999999997,"15min":6.3819999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57199999999999995,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.875,"15min":0.875},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.57099999999999995,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.65100000000000002,"15min":0.625},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80800000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.48099999999999998,"15min":0.433},"min":{"1min":0.38400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.625}]},{"osd":6,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96799999999999997,"5min":6.4219999999999997,"15min":6.4219999999999997},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70399999999999996,"15min":0.67100000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84099999999999997},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":4.21,"15min":4.21},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.64100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.84099999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.45800000000000002},"min":{"1min":0.35199999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.853,"5min":7.8940000000000001,"15min":7.8940000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61699999999999999,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92300000000000004,"5min":13.968999999999999,"15min":13.968999999999999},"last":0.374}]},{"osd":1,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.978,"5min":7.0460000000000003,"15min":7.0460000000000003},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.80800000000000005,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.67200000000000004,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80600000000000005,"15min":0.67300000000000004},"min":{"1min":0.59899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9399999999999999,"5min":7.1319999999999997,"15min":7.1319999999999997},"last":0.78300000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.78100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.157,"5min":0.80700000000000005,"15min":0.69799999999999995},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.9950000000000001,"5min":6.9950000000000001,"15min":6.9950000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.82699999999999996,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":13.731999999999999,"5min":13.731999999999999,"15min":13.731999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.879,"15min":0.75800000000000001},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.827,"5min":13.891,"15min":13.891},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.93700000000000006,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":13.797000000000001,"5min":13.797000000000001,"15min":13.797000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1699999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.80900000000000005}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.99099999999999999,"15min":0.94499999999999995},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0310000000000001,"5min":7.2709999999999999,"15min":7.2709999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.034,"15min":0.95299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.3599999999999999,"5min":13.930999999999999,"15min":13.930999999999999},"last":0.56499999999999995}]},{"osd":10,"last update":"Thu Jul 13 22:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88800000000000001},"min":{"1min":0.59899999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.046,"15min":1.01},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":7.085,"5min":7.085,"15min":7.085},"last":0.84699999999999998}]},{"osd":11,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.86699999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":445781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":1.004,"15min":1.004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89300000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.71799999999999997,"15min":0.67300000000000004},"min":{"1min":0.60099999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.2699999999999996,"5min":4.2699999999999996,"15min":4.2699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88600000000000001,"5min":1.01,"15min":1.01},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1556480,"data_stored":1036423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.139,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.622},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2330000000000001,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.55300000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83099999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65500000000000003,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.29699999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.66900000000000004,"5min":1.665,"15min":1.665},"last":0.503}]},{"osd":3,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80600000000000005,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.60199999999999998},"min":{"1min":0.56799999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.51000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":1.288,"15min":1.288},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93899999999999995,"5min":3.823,"15min":3.823},"last":0.751}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86599999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.874},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89600000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.77500000000000002},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96699999999999997,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.75800000000000001},"min":{"1min":0.60199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.72099999999999997,"15min":0.68600000000000005},"min":{"1min":0.65800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.64000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:14:05.301 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-13T22:14:05.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-13T22:14:05.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 -- 172.21.15.26:0/4054621994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 msgr2=0x7fee64105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 --2- 172.21.15.26:0/4054621994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64105260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee58009990 tx=0x7fee58009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 -- 172.21.15.26:0/4054621994 shutdown_connections 2023-07-13T22:14:05.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 --2- 172.21.15.26:0/4054621994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee64102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 --2- 172.21.15.26:0/4054621994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee64107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 --2- 172.21.15.26:0/4054621994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64105260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.547+0000 7fee695ff700 1 -- 172.21.15.26:0/4054621994 >> 172.21.15.26:0/4054621994 conn(0x7fee640fa000 msgr2=0x7fee640fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:05.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.548+0000 7fee695ff700 1 -- 172.21.15.26:0/4054621994 shutdown_connections 2023-07-13T22:14:05.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.548+0000 7fee695ff700 1 -- 172.21.15.26:0/4054621994 wait complete. 2023-07-13T22:14:05.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 Processor -- start 2023-07-13T22:14:05.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 -- start start 2023-07-13T22:14:05.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee6419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419f570 con 0x7fee64100560 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419f6b0 con 0x7fee64102e80 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee695ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419f840 con 0x7fee641057a0 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee62ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee627fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee62ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45258/0 (socket says 172.21.15.26:45258) 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee62ffd700 1 -- 172.21.15.26:0/1812002771 learned_addr learned my addr 172.21.15.26:0/1812002771 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:05.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.549+0000 7fee637fe700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee6419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 -- 172.21.15.26:0/1812002771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 msgr2=0x7fee6419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee6419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 -- 172.21.15.26:0/1812002771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 msgr2=0x7fee64197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 -- 172.21.15.26:0/1812002771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee580095f0 con 0x7fee64100560 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee637fe700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee6419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:05.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee62ffd700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9a0 tx=0x7fee5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54009790 con 0x7fee64100560 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee5400cbd0 con 0x7fee64100560 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6419fb20 con 0x7fee64100560 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54018680 con 0x7fee64100560 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.550+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee641a0080 con 0x7fee64100560 2023-07-13T22:14:05.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.552+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee64107360 con 0x7fee64100560 2023-07-13T22:14:05.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.553+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fee5400cd40 con 0x7fee64100560 2023-07-13T22:14:05.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.555+0000 7fee4bfff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 0x7fee4c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:05.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.555+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fee54014070 con 0x7fee64100560 2023-07-13T22:14:05.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.555+0000 7fee627fc700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 0x7fee4c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:05.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.556+0000 7fee627fc700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 0x7fee4c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b2e0 tx=0x7fee58005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.559+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fee54051790 con 0x7fee64100560 2023-07-13T22:14:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.764+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 --> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee64060950 con 0x7fee4c0a1220 2023-07-13T22:14:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.775+0000 7fee4bfff700 1 -- 172.21.15.26:0/1812002771 <== mgr.14148 v2:172.21.15.26:6800/4053809718 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345420 (secure 0 0 0) 0x7fee64060950 con 0x7fee4c0a1220 2023-07-13T22:14:05.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:05.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 msgr2=0x7fee4c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 0x7fee4c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fee5800b2e0 tx=0x7fee58005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 msgr2=0x7fee641979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9a0 tx=0x7fee5400ecb0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 shutdown_connections 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fee4c0a1220 0x7fee4c0a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee64100560 0x7fee641979f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee641057a0 0x7fee6419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 --2- 172.21.15.26:0/1812002771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64102e80 0x7fee64197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:05.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 >> 172.21.15.26:0/1812002771 conn(0x7fee640fa000 msgr2=0x7fee640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:05.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 shutdown_connections 2023-07-13T22:14:05.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:05.779+0000 7fee695ff700 1 -- 172.21.15.26:0/1812002771 wait complete. 2023-07-13T22:14:05.772 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-07-13T22:14:05.788 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-07-13T22:14:05.446456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659521,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1115,"num_read_kb":685,"num_write":428,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182248,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":852,"ondisk_log_size":852,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336432,"kb_used_data":13008,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605392,"statfs":{"total":1151940427776,"available":1151595921408,"internally_reserved":0,"allocated":13320192,"data_stored":7291163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4952,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":112,"num_read_kb":57,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002913"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461705+0000","last_change":"2023-07-13T22:12:18.985042+0000","last_active":"2023-07-13T22:13:51.461705+0000","last_peered":"2023-07-13T22:13:51.461705+0000","last_clean":"2023-07-13T22:13:51.461705+0000","last_became_active":"2023-07-13T22:12:18.984736+0000","last_became_peered":"2023-07-13T22:12:18.984736+0000","last_unstale":"2023-07-13T22:13:51.461705+0000","last_undegraded":"2023-07-13T22:13:51.461705+0000","last_fullsized":"2023-07-13T22:13:51.461705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:57:47.548617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455808+0000","last_change":"2023-07-13T22:12:18.976688+0000","last_active":"2023-07-13T22:13:51.455808+0000","last_peered":"2023-07-13T22:13:51.455808+0000","last_clean":"2023-07-13T22:13:51.455808+0000","last_became_active":"2023-07-13T22:12:18.976237+0000","last_became_peered":"2023-07-13T22:12:18.976237+0000","last_unstale":"2023-07-13T22:13:51.455808+0000","last_undegraded":"2023-07-13T22:13:51.455808+0000","last_fullsized":"2023-07-13T22:13:51.455808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:49:21.973731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453820+0000","last_change":"2023-07-13T22:12:18.970963+0000","last_active":"2023-07-13T22:13:51.453820+0000","last_peered":"2023-07-13T22:13:51.453820+0000","last_clean":"2023-07-13T22:13:51.453820+0000","last_became_active":"2023-07-13T22:12:18.970613+0000","last_became_peered":"2023-07-13T22:12:18.970613+0000","last_unstale":"2023-07-13T22:13:51.453820+0000","last_undegraded":"2023-07-13T22:13:51.453820+0000","last_fullsized":"2023-07-13T22:13:51.453820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:25:51.412468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453453+0000","last_change":"2023-07-13T22:12:18.969882+0000","last_active":"2023-07-13T22:13:51.453453+0000","last_peered":"2023-07-13T22:13:51.453453+0000","last_clean":"2023-07-13T22:13:51.453453+0000","last_became_active":"2023-07-13T22:12:18.969575+0000","last_became_peered":"2023-07-13T22:12:18.969575+0000","last_unstale":"2023-07-13T22:13:51.453453+0000","last_undegraded":"2023-07-13T22:13:51.453453+0000","last_fullsized":"2023-07-13T22:13:51.453453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:02:33.084972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464668+0000","last_change":"2023-07-13T22:12:18.976574+0000","last_active":"2023-07-13T22:13:51.464668+0000","last_peered":"2023-07-13T22:13:51.464668+0000","last_clean":"2023-07-13T22:13:51.464668+0000","last_became_active":"2023-07-13T22:12:18.976095+0000","last_became_peered":"2023-07-13T22:12:18.976095+0000","last_unstale":"2023-07-13T22:13:51.464668+0000","last_undegraded":"2023-07-13T22:13:51.464668+0000","last_fullsized":"2023-07-13T22:13:51.464668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:14:33.505950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453475+0000","last_change":"2023-07-13T22:12:18.965690+0000","last_active":"2023-07-13T22:13:51.453475+0000","last_peered":"2023-07-13T22:13:51.453475+0000","last_clean":"2023-07-13T22:13:51.453475+0000","last_became_active":"2023-07-13T22:12:18.965412+0000","last_became_peered":"2023-07-13T22:12:18.965412+0000","last_unstale":"2023-07-13T22:13:51.453475+0000","last_undegraded":"2023-07-13T22:13:51.453475+0000","last_fullsized":"2023-07-13T22:13:51.453475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:31:04.653956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.444154+0000","last_change":"2023-07-13T22:12:18.975741+0000","last_active":"2023-07-13T22:13:51.444154+0000","last_peered":"2023-07-13T22:13:51.444154+0000","last_clean":"2023-07-13T22:13:51.444154+0000","last_became_active":"2023-07-13T22:12:18.975370+0000","last_became_peered":"2023-07-13T22:12:18.975370+0000","last_unstale":"2023-07-13T22:13:51.444154+0000","last_undegraded":"2023-07-13T22:13:51.444154+0000","last_fullsized":"2023-07-13T22:13:51.444154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:06.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465428+0000","last_change":"2023-07-13T22:12:18.981212+0000","last_active":"2023-07-13T22:13:51.465428+0000","last_peered":"2023-07-13T22:13:51.465428+0000","last_clean":"2023-07-13T22:13:51.465428+0000","last_became_active":"2023-07-13T22:12:18.980669+0000","last_became_peered":"2023-07-13T22:12:18.980669+0000","last_unstale":"2023-07-13T22:13:51.465428+0000","last_undegraded":"2023-07-13T22:13:51.465428+0000","last_fullsized":"2023-07-13T22:13:51.465428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:17:49.982151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464318+0000","last_change":"2023-07-13T22:12:18.975109+0000","last_active":"2023-07-13T22:13:51.464318+0000","last_peered":"2023-07-13T22:13:51.464318+0000","last_clean":"2023-07-13T22:13:51.464318+0000","last_became_active":"2023-07-13T22:12:18.974793+0000","last_became_peered":"2023-07-13T22:12:18.974793+0000","last_unstale":"2023-07-13T22:13:51.464318+0000","last_undegraded":"2023-07-13T22:13:51.464318+0000","last_fullsized":"2023-07-13T22:13:51.464318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:19.458063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456141+0000","last_change":"2023-07-13T22:12:52.546759+0000","last_active":"2023-07-13T22:13:51.456141+0000","last_peered":"2023-07-13T22:13:51.456141+0000","last_clean":"2023-07-13T22:13:51.456141+0000","last_became_active":"2023-07-13T22:12:52.546378+0000","last_became_peered":"2023-07-13T22:12:52.546378+0000","last_unstale":"2023-07-13T22:13:51.456141+0000","last_undegraded":"2023-07-13T22:13:51.456141+0000","last_fullsized":"2023-07-13T22:13:51.456141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:00:38.494158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.450988+0000","last_change":"2023-07-13T22:12:18.972111+0000","last_active":"2023-07-13T22:13:51.450988+0000","last_peered":"2023-07-13T22:13:51.450988+0000","last_clean":"2023-07-13T22:13:51.450988+0000","last_became_active":"2023-07-13T22:12:18.971807+0000","last_became_peered":"2023-07-13T22:12:18.971807+0000","last_unstale":"2023-07-13T22:13:51.450988+0000","last_undegraded":"2023-07-13T22:13:51.450988+0000","last_fullsized":"2023-07-13T22:13:51.450988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:27:47.507942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.454081+0000","last_change":"2023-07-13T22:12:18.969050+0000","last_active":"2023-07-13T22:13:51.454081+0000","last_peered":"2023-07-13T22:13:51.454081+0000","last_clean":"2023-07-13T22:13:51.454081+0000","last_became_active":"2023-07-13T22:12:18.968736+0000","last_became_peered":"2023-07-13T22:12:18.968736+0000","last_unstale":"2023-07-13T22:13:51.454081+0000","last_undegraded":"2023-07-13T22:13:51.454081+0000","last_fullsized":"2023-07-13T22:13:51.454081+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:58:55.880996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465068+0000","last_change":"2023-07-13T22:12:18.975936+0000","last_active":"2023-07-13T22:13:51.465068+0000","last_peered":"2023-07-13T22:13:51.465068+0000","last_clean":"2023-07-13T22:13:51.465068+0000","last_became_active":"2023-07-13T22:12:18.975363+0000","last_became_peered":"2023-07-13T22:12:18.975363+0000","last_unstale":"2023-07-13T22:13:51.465068+0000","last_undegraded":"2023-07-13T22:13:51.465068+0000","last_fullsized":"2023-07-13T22:13:51.465068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:45:58.155432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457726+0000","last_change":"2023-07-13T22:12:18.972917+0000","last_active":"2023-07-13T22:13:51.457726+0000","last_peered":"2023-07-13T22:13:51.457726+0000","last_clean":"2023-07-13T22:13:51.457726+0000","last_became_active":"2023-07-13T22:12:18.972580+0000","last_became_peered":"2023-07-13T22:12:18.972580+0000","last_unstale":"2023-07-13T22:13:51.457726+0000","last_undegraded":"2023-07-13T22:13:51.457726+0000","last_fullsized":"2023-07-13T22:13:51.457726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:57:57.803872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445655+0000","last_change":"2023-07-13T22:12:52.549238+0000","last_active":"2023-07-13T22:13:51.445655+0000","last_peered":"2023-07-13T22:13:51.445655+0000","last_clean":"2023-07-13T22:13:51.445655+0000","last_became_active":"2023-07-13T22:12:52.548923+0000","last_became_peered":"2023-07-13T22:12:52.548923+0000","last_unstale":"2023-07-13T22:13:51.445655+0000","last_undegraded":"2023-07-13T22:13:51.445655+0000","last_fullsized":"2023-07-13T22:13:51.445655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:08.925620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459735+0000","last_change":"2023-07-13T22:12:18.981636+0000","last_active":"2023-07-13T22:13:51.459735+0000","last_peered":"2023-07-13T22:13:51.459735+0000","last_clean":"2023-07-13T22:13:51.459735+0000","last_became_active":"2023-07-13T22:12:18.981345+0000","last_became_peered":"2023-07-13T22:12:18.981345+0000","last_unstale":"2023-07-13T22:13:51.459735+0000","last_undegraded":"2023-07-13T22:13:51.459735+0000","last_fullsized":"2023-07-13T22:13:51.459735+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:46:23.412004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461935+0000","last_change":"2023-07-13T22:12:18.985136+0000","last_active":"2023-07-13T22:13:51.461935+0000","last_peered":"2023-07-13T22:13:51.461935+0000","last_clean":"2023-07-13T22:13:51.461935+0000","last_became_active":"2023-07-13T22:12:18.984841+0000","last_became_peered":"2023-07-13T22:12:18.984841+0000","last_unstale":"2023-07-13T22:13:51.461935+0000","last_undegraded":"2023-07-13T22:13:51.461935+0000","last_fullsized":"2023-07-13T22:13:51.461935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:49:37.216203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463124+0000","last_change":"2023-07-13T22:12:18.975956+0000","last_active":"2023-07-13T22:13:51.463124+0000","last_peered":"2023-07-13T22:13:51.463124+0000","last_clean":"2023-07-13T22:13:51.463124+0000","last_became_active":"2023-07-13T22:12:18.975580+0000","last_became_peered":"2023-07-13T22:12:18.975580+0000","last_unstale":"2023-07-13T22:13:51.463124+0000","last_undegraded":"2023-07-13T22:13:51.463124+0000","last_fullsized":"2023-07-13T22:13:51.463124+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:29:18.013029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456310+0000","last_change":"2023-07-13T22:12:18.983392+0000","last_active":"2023-07-13T22:13:51.456310+0000","last_peered":"2023-07-13T22:13:51.456310+0000","last_clean":"2023-07-13T22:13:51.456310+0000","last_became_active":"2023-07-13T22:12:18.983056+0000","last_became_peered":"2023-07-13T22:12:18.983056+0000","last_unstale":"2023-07-13T22:13:51.456310+0000","last_undegraded":"2023-07-13T22:13:51.456310+0000","last_fullsized":"2023-07-13T22:13:51.456310+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:55:56.130000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":130,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:02.623004+0000","last_change":"2023-07-13T22:12:17.986121+0000","last_active":"2023-07-13T22:14:02.623004+0000","last_peered":"2023-07-13T22:14:02.623004+0000","last_clean":"2023-07-13T22:14:02.623004+0000","last_became_active":"2023-07-13T22:12:17.985767+0000","last_became_peered":"2023-07-13T22:12:17.985767+0000","last_unstale":"2023-07-13T22:14:02.623004+0000","last_undegraded":"2023-07-13T22:14:02.623004+0000","last_fullsized":"2023-07-13T22:14:02.623004+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:28:47.898506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456796+0000","last_change":"2023-07-13T22:12:18.981082+0000","last_active":"2023-07-13T22:13:51.456796+0000","last_peered":"2023-07-13T22:13:51.456796+0000","last_clean":"2023-07-13T22:13:51.456796+0000","last_became_active":"2023-07-13T22:12:18.980775+0000","last_became_peered":"2023-07-13T22:12:18.980775+0000","last_unstale":"2023-07-13T22:13:51.456796+0000","last_undegraded":"2023-07-13T22:13:51.456796+0000","last_fullsized":"2023-07-13T22:13:51.456796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:42:35.256971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791902+0000","last_change":"2023-07-13T22:12:17.997371+0000","last_active":"2023-07-13T22:12:52.791902+0000","last_peered":"2023-07-13T22:12:52.791902+0000","last_clean":"2023-07-13T22:12:52.791902+0000","last_became_active":"2023-07-13T22:12:17.996953+0000","last_became_peered":"2023-07-13T22:12:17.996953+0000","last_unstale":"2023-07-13T22:12:52.791902+0000","last_undegraded":"2023-07-13T22:12:52.791902+0000","last_fullsized":"2023-07-13T22:12:52.791902+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:29.131604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464536+0000","last_change":"2023-07-13T22:12:18.972855+0000","last_active":"2023-07-13T22:13:51.464536+0000","last_peered":"2023-07-13T22:13:51.464536+0000","last_clean":"2023-07-13T22:13:51.464536+0000","last_became_active":"2023-07-13T22:12:18.972571+0000","last_became_peered":"2023-07-13T22:12:18.972571+0000","last_unstale":"2023-07-13T22:13:51.464536+0000","last_undegraded":"2023-07-13T22:13:51.464536+0000","last_fullsized":"2023-07-13T22:13:51.464536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:19:53.951535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.554410+0000","last_change":"2023-07-13T22:12:52.554410+0000","last_active":"2023-07-13T22:12:52.554410+0000","last_peered":"2023-07-13T22:12:52.554410+0000","last_clean":"2023-07-13T22:12:52.554410+0000","last_became_active":"2023-07-13T22:12:52.541864+0000","last_became_peered":"2023-07-13T22:12:52.541864+0000","last_unstale":"2023-07-13T22:12:52.554410+0000","last_undegraded":"2023-07-13T22:12:52.554410+0000","last_fullsized":"2023-07-13T22:12:52.554410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:48:29.339493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463743+0000","last_change":"2023-07-13T22:12:18.984981+0000","last_active":"2023-07-13T22:13:51.463743+0000","last_peered":"2023-07-13T22:13:51.463743+0000","last_clean":"2023-07-13T22:13:51.463743+0000","last_became_active":"2023-07-13T22:12:18.984696+0000","last_became_peered":"2023-07-13T22:12:18.984696+0000","last_unstale":"2023-07-13T22:13:51.463743+0000","last_undegraded":"2023-07-13T22:13:51.463743+0000","last_fullsized":"2023-07-13T22:13:51.463743+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:32:58.826285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542998+0000","last_change":"2023-07-13T22:12:17.987815+0000","last_active":"2023-07-13T22:12:52.542998+0000","last_peered":"2023-07-13T22:12:52.542998+0000","last_clean":"2023-07-13T22:12:52.542998+0000","last_became_active":"2023-07-13T22:12:17.987410+0000","last_became_peered":"2023-07-13T22:12:17.987410+0000","last_unstale":"2023-07-13T22:12:52.542998+0000","last_undegraded":"2023-07-13T22:12:52.542998+0000","last_fullsized":"2023-07-13T22:12:52.542998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:22:28.850188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457222+0000","last_change":"2023-07-13T22:12:18.977355+0000","last_active":"2023-07-13T22:13:51.457222+0000","last_peered":"2023-07-13T22:13:51.457222+0000","last_clean":"2023-07-13T22:13:51.457222+0000","last_became_active":"2023-07-13T22:12:18.977024+0000","last_became_peered":"2023-07-13T22:12:18.977024+0000","last_unstale":"2023-07-13T22:13:51.457222+0000","last_undegraded":"2023-07-13T22:13:51.457222+0000","last_fullsized":"2023-07-13T22:13:51.457222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:12:21.578829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'5","reported_seq":123,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572636+0000","last_change":"2023-07-13T22:12:17.988133+0000","last_active":"2023-07-13T22:14:00.572636+0000","last_peered":"2023-07-13T22:14:00.572636+0000","last_clean":"2023-07-13T22:14:00.572636+0000","last_became_active":"2023-07-13T22:12:17.987675+0000","last_became_peered":"2023-07-13T22:12:17.987675+0000","last_unstale":"2023-07-13T22:14:00.572636+0000","last_undegraded":"2023-07-13T22:14:00.572636+0000","last_fullsized":"2023-07-13T22:14:00.572636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:56:16.055459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.441362+0000","last_change":"2023-07-13T22:12:18.975279+0000","last_active":"2023-07-13T22:13:51.441362+0000","last_peered":"2023-07-13T22:13:51.441362+0000","last_clean":"2023-07-13T22:13:51.441362+0000","last_became_active":"2023-07-13T22:12:18.974736+0000","last_became_peered":"2023-07-13T22:12:18.974736+0000","last_unstale":"2023-07-13T22:13:51.441362+0000","last_undegraded":"2023-07-13T22:13:51.441362+0000","last_fullsized":"2023-07-13T22:13:51.441362+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:28.566300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538988+0000","last_change":"2023-07-13T22:12:17.993677+0000","last_active":"2023-07-13T22:12:52.538988+0000","last_peered":"2023-07-13T22:12:52.538988+0000","last_clean":"2023-07-13T22:12:52.538988+0000","last_became_active":"2023-07-13T22:12:17.993311+0000","last_became_peered":"2023-07-13T22:12:17.993311+0000","last_unstale":"2023-07-13T22:12:52.538988+0000","last_undegraded":"2023-07-13T22:12:52.538988+0000","last_fullsized":"2023-07-13T22:12:52.538988+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:01:40.167436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465176+0000","last_change":"2023-07-13T22:12:18.980907+0000","last_active":"2023-07-13T22:13:51.465176+0000","last_peered":"2023-07-13T22:13:51.465176+0000","last_clean":"2023-07-13T22:13:51.465176+0000","last_became_active":"2023-07-13T22:12:18.980623+0000","last_became_peered":"2023-07-13T22:12:18.980623+0000","last_unstale":"2023-07-13T22:13:51.465176+0000","last_undegraded":"2023-07-13T22:13:51.465176+0000","last_fullsized":"2023-07-13T22:13:51.465176+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:56:03.123538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.537975+0000","last_change":"2023-07-13T22:12:17.994505+0000","last_active":"2023-07-13T22:12:52.537975+0000","last_peered":"2023-07-13T22:12:52.537975+0000","last_clean":"2023-07-13T22:12:52.537975+0000","last_became_active":"2023-07-13T22:12:17.994163+0000","last_became_peered":"2023-07-13T22:12:17.994163+0000","last_unstale":"2023-07-13T22:12:52.537975+0000","last_undegraded":"2023-07-13T22:12:52.537975+0000","last_fullsized":"2023-07-13T22:12:52.537975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:32.938212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451163+0000","last_change":"2023-07-13T22:12:52.546725+0000","last_active":"2023-07-13T22:13:51.451163+0000","last_peered":"2023-07-13T22:13:51.451163+0000","last_clean":"2023-07-13T22:13:51.451163+0000","last_became_active":"2023-07-13T22:12:52.546385+0000","last_became_peered":"2023-07-13T22:12:52.546385+0000","last_unstale":"2023-07-13T22:13:51.451163+0000","last_undegraded":"2023-07-13T22:13:51.451163+0000","last_fullsized":"2023-07-13T22:13:51.451163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:44:18.074829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534851+0000","last_change":"2023-07-13T22:12:17.988068+0000","last_active":"2023-07-13T22:12:52.534851+0000","last_peered":"2023-07-13T22:12:52.534851+0000","last_clean":"2023-07-13T22:12:52.534851+0000","last_became_active":"2023-07-13T22:12:17.987735+0000","last_became_peered":"2023-07-13T22:12:17.987735+0000","last_unstale":"2023-07-13T22:12:52.534851+0000","last_undegraded":"2023-07-13T22:12:52.534851+0000","last_fullsized":"2023-07-13T22:12:52.534851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:23:06.618468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.457170+0000","last_change":"2023-07-13T22:12:18.976859+0000","last_active":"2023-07-13T22:13:51.457170+0000","last_peered":"2023-07-13T22:13:51.457170+0000","last_clean":"2023-07-13T22:13:51.457170+0000","last_became_active":"2023-07-13T22:12:18.976518+0000","last_became_peered":"2023-07-13T22:12:18.976518+0000","last_unstale":"2023-07-13T22:13:51.457170+0000","last_undegraded":"2023-07-13T22:13:51.457170+0000","last_fullsized":"2023-07-13T22:13:51.457170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:08:26.423367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'5","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.573478+0000","last_change":"2023-07-13T22:12:17.993562+0000","last_active":"2023-07-13T22:14:01.573478+0000","last_peered":"2023-07-13T22:14:01.573478+0000","last_clean":"2023-07-13T22:14:01.573478+0000","last_became_active":"2023-07-13T22:12:17.993225+0000","last_became_peered":"2023-07-13T22:12:17.993225+0000","last_unstale":"2023-07-13T22:14:01.573478+0000","last_undegraded":"2023-07-13T22:14:01.573478+0000","last_fullsized":"2023-07-13T22:14:01.573478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:35:37.488363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":194,"num_read_kb":194,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459625+0000","last_change":"2023-07-13T22:12:18.976728+0000","last_active":"2023-07-13T22:13:51.459625+0000","last_peered":"2023-07-13T22:13:51.459625+0000","last_clean":"2023-07-13T22:13:51.459625+0000","last_became_active":"2023-07-13T22:12:18.976406+0000","last_became_peered":"2023-07-13T22:12:18.976406+0000","last_unstale":"2023-07-13T22:13:51.459625+0000","last_undegraded":"2023-07-13T22:13:51.459625+0000","last_fullsized":"2023-07-13T22:13:51.459625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:36:47.438310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541598+0000","last_change":"2023-07-13T22:12:17.990604+0000","last_active":"2023-07-13T22:12:52.541598+0000","last_peered":"2023-07-13T22:12:52.541598+0000","last_clean":"2023-07-13T22:12:52.541598+0000","last_became_active":"2023-07-13T22:12:17.990339+0000","last_became_peered":"2023-07-13T22:12:17.990339+0000","last_unstale":"2023-07-13T22:12:52.541598+0000","last_undegraded":"2023-07-13T22:12:52.541598+0000","last_fullsized":"2023-07-13T22:12:52.541598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:00.126919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458115+0000","last_change":"2023-07-13T22:12:18.978514+0000","last_active":"2023-07-13T22:13:51.458115+0000","last_peered":"2023-07-13T22:13:51.458115+0000","last_clean":"2023-07-13T22:13:51.458115+0000","last_became_active":"2023-07-13T22:12:18.978125+0000","last_became_peered":"2023-07-13T22:12:18.978125+0000","last_unstale":"2023-07-13T22:13:51.458115+0000","last_undegraded":"2023-07-13T22:13:51.458115+0000","last_fullsized":"2023-07-13T22:13:51.458115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:31:50.338306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535374+0000","last_change":"2023-07-13T22:12:17.992442+0000","last_active":"2023-07-13T22:12:52.535374+0000","last_peered":"2023-07-13T22:12:52.535374+0000","last_clean":"2023-07-13T22:12:52.535374+0000","last_became_active":"2023-07-13T22:12:17.992081+0000","last_became_peered":"2023-07-13T22:12:17.992081+0000","last_unstale":"2023-07-13T22:12:52.535374+0000","last_undegraded":"2023-07-13T22:12:52.535374+0000","last_fullsized":"2023-07-13T22:12:52.535374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:05:36.251918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462201+0000","last_change":"2023-07-13T22:12:18.972839+0000","last_active":"2023-07-13T22:13:51.462201+0000","last_peered":"2023-07-13T22:13:51.462201+0000","last_clean":"2023-07-13T22:13:51.462201+0000","last_became_active":"2023-07-13T22:12:18.972555+0000","last_became_peered":"2023-07-13T22:12:18.972555+0000","last_unstale":"2023-07-13T22:13:51.462201+0000","last_undegraded":"2023-07-13T22:13:51.462201+0000","last_fullsized":"2023-07-13T22:13:51.462201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:50:23.829222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":257,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574528+0000","last_change":"2023-07-13T22:12:17.992789+0000","last_active":"2023-07-13T22:14:01.574528+0000","last_peered":"2023-07-13T22:14:01.574528+0000","last_clean":"2023-07-13T22:14:01.574528+0000","last_became_active":"2023-07-13T22:12:17.992467+0000","last_became_peered":"2023-07-13T22:12:17.992467+0000","last_unstale":"2023-07-13T22:14:01.574528+0000","last_undegraded":"2023-07-13T22:14:01.574528+0000","last_fullsized":"2023-07-13T22:14:01.574528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:54:07.569446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61830,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":157,"num_write":36,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445202+0000","last_change":"2023-07-13T22:12:18.971298+0000","last_active":"2023-07-13T22:13:51.445202+0000","last_peered":"2023-07-13T22:13:51.445202+0000","last_clean":"2023-07-13T22:13:51.445202+0000","last_became_active":"2023-07-13T22:12:18.970837+0000","last_became_peered":"2023-07-13T22:12:18.970837+0000","last_unstale":"2023-07-13T22:13:51.445202+0000","last_undegraded":"2023-07-13T22:13:51.445202+0000","last_fullsized":"2023-07-13T22:13:51.445202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:04:19.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540584+0000","last_change":"2023-07-13T22:12:17.992814+0000","last_active":"2023-07-13T22:12:52.540584+0000","last_peered":"2023-07-13T22:12:52.540584+0000","last_clean":"2023-07-13T22:12:52.540584+0000","last_became_active":"2023-07-13T22:12:17.992540+0000","last_became_peered":"2023-07-13T22:12:17.992540+0000","last_unstale":"2023-07-13T22:12:52.540584+0000","last_undegraded":"2023-07-13T22:12:52.540584+0000","last_fullsized":"2023-07-13T22:12:52.540584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:44:23.569177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466045+0000","last_change":"2023-07-13T22:12:18.976283+0000","last_active":"2023-07-13T22:13:51.466045+0000","last_peered":"2023-07-13T22:13:51.466045+0000","last_clean":"2023-07-13T22:13:51.466045+0000","last_became_active":"2023-07-13T22:12:18.975889+0000","last_became_peered":"2023-07-13T22:12:18.975889+0000","last_unstale":"2023-07-13T22:13:51.466045+0000","last_undegraded":"2023-07-13T22:13:51.466045+0000","last_fullsized":"2023-07-13T22:13:51.466045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:07:37.953436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.534895+0000","last_change":"2023-07-13T22:12:17.986191+0000","last_active":"2023-07-13T22:12:52.534895+0000","last_peered":"2023-07-13T22:12:52.534895+0000","last_clean":"2023-07-13T22:12:52.534895+0000","last_became_active":"2023-07-13T22:12:17.985837+0000","last_became_peered":"2023-07-13T22:12:17.985837+0000","last_unstale":"2023-07-13T22:12:52.534895+0000","last_undegraded":"2023-07-13T22:12:52.534895+0000","last_fullsized":"2023-07-13T22:12:52.534895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:45:03.241327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.462642+0000","last_change":"2023-07-13T22:12:18.985188+0000","last_active":"2023-07-13T22:13:51.462642+0000","last_peered":"2023-07-13T22:13:51.462642+0000","last_clean":"2023-07-13T22:13:51.462642+0000","last_became_active":"2023-07-13T22:12:18.984851+0000","last_became_peered":"2023-07-13T22:12:18.984851+0000","last_unstale":"2023-07-13T22:13:51.462642+0000","last_undegraded":"2023-07-13T22:13:51.462642+0000","last_fullsized":"2023-07-13T22:13:51.462642+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:50:29.680386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.574264+0000","last_change":"2023-07-13T22:12:17.985881+0000","last_active":"2023-07-13T22:14:01.574264+0000","last_peered":"2023-07-13T22:14:01.574264+0000","last_clean":"2023-07-13T22:14:01.574264+0000","last_became_active":"2023-07-13T22:12:17.985522+0000","last_became_peered":"2023-07-13T22:12:17.985522+0000","last_unstale":"2023-07-13T22:14:01.574264+0000","last_undegraded":"2023-07-13T22:14:01.574264+0000","last_fullsized":"2023-07-13T22:14:01.574264+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:04:24.354790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.449957+0000","last_change":"2023-07-13T22:12:18.976817+0000","last_active":"2023-07-13T22:13:51.449957+0000","last_peered":"2023-07-13T22:13:51.449957+0000","last_clean":"2023-07-13T22:13:51.449957+0000","last_became_active":"2023-07-13T22:12:18.975426+0000","last_became_peered":"2023-07-13T22:12:18.975426+0000","last_unstale":"2023-07-13T22:13:51.449957+0000","last_undegraded":"2023-07-13T22:13:51.449957+0000","last_fullsized":"2023-07-13T22:13:51.449957+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:43:39.397611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541703+0000","last_change":"2023-07-13T22:12:17.990502+0000","last_active":"2023-07-13T22:12:52.541703+0000","last_peered":"2023-07-13T22:12:52.541703+0000","last_clean":"2023-07-13T22:12:52.541703+0000","last_became_active":"2023-07-13T22:12:17.990162+0000","last_became_peered":"2023-07-13T22:12:17.990162+0000","last_unstale":"2023-07-13T22:12:52.541703+0000","last_undegraded":"2023-07-13T22:12:52.541703+0000","last_fullsized":"2023-07-13T22:12:52.541703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:58:54.534152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456704+0000","last_change":"2023-07-13T22:12:18.974751+0000","last_active":"2023-07-13T22:13:51.456704+0000","last_peered":"2023-07-13T22:13:51.456704+0000","last_clean":"2023-07-13T22:13:51.456704+0000","last_became_active":"2023-07-13T22:12:18.974409+0000","last_became_peered":"2023-07-13T22:12:18.974409+0000","last_unstale":"2023-07-13T22:13:51.456704+0000","last_undegraded":"2023-07-13T22:13:51.456704+0000","last_fullsized":"2023-07-13T22:13:51.456704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:18:37.092789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543636+0000","last_change":"2023-07-13T22:12:17.990285+0000","last_active":"2023-07-13T22:12:52.543636+0000","last_peered":"2023-07-13T22:12:52.543636+0000","last_clean":"2023-07-13T22:12:52.543636+0000","last_became_active":"2023-07-13T22:12:17.989943+0000","last_became_peered":"2023-07-13T22:12:17.989943+0000","last_unstale":"2023-07-13T22:12:52.543636+0000","last_undegraded":"2023-07-13T22:12:52.543636+0000","last_fullsized":"2023-07-13T22:12:52.543636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:29:05.526262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456758+0000","last_change":"2023-07-13T22:12:18.974950+0000","last_active":"2023-07-13T22:13:51.456758+0000","last_peered":"2023-07-13T22:13:51.456758+0000","last_clean":"2023-07-13T22:13:51.456758+0000","last_became_active":"2023-07-13T22:12:18.974649+0000","last_became_peered":"2023-07-13T22:12:18.974649+0000","last_unstale":"2023-07-13T22:13:51.456758+0000","last_undegraded":"2023-07-13T22:13:51.456758+0000","last_fullsized":"2023-07-13T22:13:51.456758+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:53:20.274354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539898+0000","last_change":"2023-07-13T22:12:17.993386+0000","last_active":"2023-07-13T22:12:52.539898+0000","last_peered":"2023-07-13T22:12:52.539898+0000","last_clean":"2023-07-13T22:12:52.539898+0000","last_became_active":"2023-07-13T22:12:17.992808+0000","last_became_peered":"2023-07-13T22:12:17.992808+0000","last_unstale":"2023-07-13T22:12:52.539898+0000","last_undegraded":"2023-07-13T22:12:52.539898+0000","last_fullsized":"2023-07-13T22:12:52.539898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:26.725096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465891+0000","last_change":"2023-07-13T22:12:18.974476+0000","last_active":"2023-07-13T22:13:51.465891+0000","last_peered":"2023-07-13T22:13:51.465891+0000","last_clean":"2023-07-13T22:13:51.465891+0000","last_became_active":"2023-07-13T22:12:18.974172+0000","last_became_peered":"2023-07-13T22:12:18.974172+0000","last_unstale":"2023-07-13T22:13:51.465891+0000","last_undegraded":"2023-07-13T22:13:51.465891+0000","last_fullsized":"2023-07-13T22:13:51.465891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:33:02.628236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.622309+0000","last_change":"2023-07-13T22:12:17.993896+0000","last_active":"2023-07-13T22:14:01.622309+0000","last_peered":"2023-07-13T22:14:01.622309+0000","last_clean":"2023-07-13T22:14:01.622309+0000","last_became_active":"2023-07-13T22:12:17.993592+0000","last_became_peered":"2023-07-13T22:12:17.993592+0000","last_unstale":"2023-07-13T22:14:01.622309+0000","last_undegraded":"2023-07-13T22:14:01.622309+0000","last_fullsized":"2023-07-13T22:14:01.622309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:00:22.807561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.453734+0000","last_change":"2023-07-13T22:12:52.546608+0000","last_active":"2023-07-13T22:13:51.453734+0000","last_peered":"2023-07-13T22:13:51.453734+0000","last_clean":"2023-07-13T22:13:51.453734+0000","last_became_active":"2023-07-13T22:12:52.546240+0000","last_became_peered":"2023-07-13T22:12:52.546240+0000","last_unstale":"2023-07-13T22:13:51.453734+0000","last_undegraded":"2023-07-13T22:13:51.453734+0000","last_fullsized":"2023-07-13T22:13:51.453734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:26:11.200607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.456809+0000","last_change":"2023-07-13T22:12:18.978806+0000","last_active":"2023-07-13T22:13:51.456809+0000","last_peered":"2023-07-13T22:13:51.456809+0000","last_clean":"2023-07-13T22:13:51.456809+0000","last_became_active":"2023-07-13T22:12:18.978504+0000","last_became_peered":"2023-07-13T22:12:18.978504+0000","last_unstale":"2023-07-13T22:13:51.456809+0000","last_undegraded":"2023-07-13T22:13:51.456809+0000","last_fullsized":"2023-07-13T22:13:51.456809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:16:05.891842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:00.572319+0000","last_change":"2023-07-13T22:12:17.993531+0000","last_active":"2023-07-13T22:14:00.572319+0000","last_peered":"2023-07-13T22:14:00.572319+0000","last_clean":"2023-07-13T22:14:00.572319+0000","last_became_active":"2023-07-13T22:12:17.992764+0000","last_became_peered":"2023-07-13T22:12:17.992764+0000","last_unstale":"2023-07-13T22:14:00.572319+0000","last_undegraded":"2023-07-13T22:14:00.572319+0000","last_fullsized":"2023-07-13T22:14:00.572319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:09.930750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.464940+0000","last_change":"2023-07-13T22:12:18.980592+0000","last_active":"2023-07-13T22:13:51.464940+0000","last_peered":"2023-07-13T22:13:51.464940+0000","last_clean":"2023-07-13T22:13:51.464940+0000","last_became_active":"2023-07-13T22:12:18.980270+0000","last_became_peered":"2023-07-13T22:12:18.980270+0000","last_unstale":"2023-07-13T22:13:51.464940+0000","last_undegraded":"2023-07-13T22:13:51.464940+0000","last_fullsized":"2023-07-13T22:13:51.464940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:42:38.987941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539678+0000","last_change":"2023-07-13T22:12:17.993285+0000","last_active":"2023-07-13T22:12:52.539678+0000","last_peered":"2023-07-13T22:12:52.539678+0000","last_clean":"2023-07-13T22:12:52.539678+0000","last_became_active":"2023-07-13T22:12:17.992905+0000","last_became_peered":"2023-07-13T22:12:17.992905+0000","last_unstale":"2023-07-13T22:12:52.539678+0000","last_undegraded":"2023-07-13T22:12:52.539678+0000","last_fullsized":"2023-07-13T22:12:52.539678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:01:49.259253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":334,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540813+0000","last_change":"2023-07-13T22:11:13.860239+0000","last_active":"2023-07-13T22:12:52.540813+0000","last_peered":"2023-07-13T22:12:52.540813+0000","last_clean":"2023-07-13T22:12:52.540813+0000","last_became_active":"2023-07-13T22:11:13.859701+0000","last_became_peered":"2023-07-13T22:11:13.859701+0000","last_unstale":"2023-07-13T22:12:52.540813+0000","last_undegraded":"2023-07-13T22:12:52.540813+0000","last_fullsized":"2023-07-13T22:12:52.540813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:08:09.976809+0000","last_clean_scrub_stamp":"2023-07-13T22:08:09.976809+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:11:16.076523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461027+0000","last_change":"2023-07-13T22:12:18.974360+0000","last_active":"2023-07-13T22:13:51.461027+0000","last_peered":"2023-07-13T22:13:51.461027+0000","last_clean":"2023-07-13T22:13:51.461027+0000","last_became_active":"2023-07-13T22:12:18.974010+0000","last_became_peered":"2023-07-13T22:12:18.974010+0000","last_unstale":"2023-07-13T22:13:51.461027+0000","last_undegraded":"2023-07-13T22:13:51.461027+0000","last_fullsized":"2023-07-13T22:13:51.461027+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:48:45.725233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.541086+0000","last_change":"2023-07-13T22:12:17.989027+0000","last_active":"2023-07-13T22:12:52.541086+0000","last_peered":"2023-07-13T22:12:52.541086+0000","last_clean":"2023-07-13T22:12:52.541086+0000","last_became_active":"2023-07-13T22:12:17.988320+0000","last_became_peered":"2023-07-13T22:12:17.988320+0000","last_unstale":"2023-07-13T22:12:52.541086+0000","last_undegraded":"2023-07-13T22:12:52.541086+0000","last_fullsized":"2023-07-13T22:12:52.541086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T09:13:50.437984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.543196+0000","last_change":"2023-07-13T22:12:17.993739+0000","last_active":"2023-07-13T22:12:52.543196+0000","last_peered":"2023-07-13T22:12:52.543196+0000","last_clean":"2023-07-13T22:12:52.543196+0000","last_became_active":"2023-07-13T22:12:17.993380+0000","last_became_peered":"2023-07-13T22:12:17.993380+0000","last_unstale":"2023-07-13T22:12:52.543196+0000","last_undegraded":"2023-07-13T22:12:52.543196+0000","last_fullsized":"2023-07-13T22:12:52.543196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:00:37.229195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.455688+0000","last_change":"2023-07-13T22:12:18.973320+0000","last_active":"2023-07-13T22:13:51.455688+0000","last_peered":"2023-07-13T22:13:51.455688+0000","last_clean":"2023-07-13T22:13:51.455688+0000","last_became_active":"2023-07-13T22:12:18.973000+0000","last_became_peered":"2023-07-13T22:12:18.973000+0000","last_unstale":"2023-07-13T22:13:51.455688+0000","last_undegraded":"2023-07-13T22:13:51.455688+0000","last_fullsized":"2023-07-13T22:13:51.455688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T10:06:01.886536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.626308+0000","last_change":"2023-07-13T22:12:17.984073+0000","last_active":"2023-07-13T22:13:51.626308+0000","last_peered":"2023-07-13T22:13:51.626308+0000","last_clean":"2023-07-13T22:13:51.626308+0000","last_became_active":"2023-07-13T22:12:17.983697+0000","last_became_peered":"2023-07-13T22:12:17.983697+0000","last_unstale":"2023-07-13T22:13:51.626308+0000","last_undegraded":"2023-07-13T22:13:51.626308+0000","last_fullsized":"2023-07-13T22:13:51.626308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:37:27.067307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459700+0000","last_change":"2023-07-13T22:12:18.972101+0000","last_active":"2023-07-13T22:13:51.459700+0000","last_peered":"2023-07-13T22:13:51.459700+0000","last_clean":"2023-07-13T22:13:51.459700+0000","last_became_active":"2023-07-13T22:12:18.971792+0000","last_became_peered":"2023-07-13T22:12:18.971792+0000","last_unstale":"2023-07-13T22:13:51.459700+0000","last_undegraded":"2023-07-13T22:13:51.459700+0000","last_fullsized":"2023-07-13T22:13:51.459700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:43:24.414406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.539157+0000","last_change":"2023-07-13T22:12:17.986800+0000","last_active":"2023-07-13T22:12:52.539157+0000","last_peered":"2023-07-13T22:12:52.539157+0000","last_clean":"2023-07-13T22:12:52.539157+0000","last_became_active":"2023-07-13T22:12:17.986485+0000","last_became_peered":"2023-07-13T22:12:17.986485+0000","last_unstale":"2023-07-13T22:12:52.539157+0000","last_undegraded":"2023-07-13T22:12:52.539157+0000","last_fullsized":"2023-07-13T22:12:52.539157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:18:15.337161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466111+0000","last_change":"2023-07-13T22:12:18.971600+0000","last_active":"2023-07-13T22:13:51.466111+0000","last_peered":"2023-07-13T22:13:51.466111+0000","last_clean":"2023-07-13T22:13:51.466111+0000","last_became_active":"2023-07-13T22:12:18.971309+0000","last_became_peered":"2023-07-13T22:12:18.971309+0000","last_unstale":"2023-07-13T22:13:51.466111+0000","last_undegraded":"2023-07-13T22:13:51.466111+0000","last_fullsized":"2023-07-13T22:13:51.466111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:37:39.269441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.551910+0000","last_change":"2023-07-13T22:12:52.551910+0000","last_active":"2023-07-13T22:12:52.551910+0000","last_peered":"2023-07-13T22:12:52.551910+0000","last_clean":"2023-07-13T22:12:52.551910+0000","last_became_active":"2023-07-13T22:12:52.548681+0000","last_became_peered":"2023-07-13T22:12:52.548681+0000","last_unstale":"2023-07-13T22:12:52.551910+0000","last_undegraded":"2023-07-13T22:12:52.551910+0000","last_fullsized":"2023-07-13T22:12:52.551910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:54:10.561539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.422697+0000","last_change":"2023-07-13T22:12:18.970801+0000","last_active":"2023-07-13T22:13:51.422697+0000","last_peered":"2023-07-13T22:13:51.422697+0000","last_clean":"2023-07-13T22:13:51.422697+0000","last_became_active":"2023-07-13T22:12:18.970476+0000","last_became_peered":"2023-07-13T22:12:18.970476+0000","last_unstale":"2023-07-13T22:13:51.422697+0000","last_undegraded":"2023-07-13T22:13:51.422697+0000","last_fullsized":"2023-07-13T22:13:51.422697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:53.309826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.791341+0000","last_change":"2023-07-13T22:12:17.987906+0000","last_active":"2023-07-13T22:12:52.791341+0000","last_peered":"2023-07-13T22:12:52.791341+0000","last_clean":"2023-07-13T22:12:52.791341+0000","last_became_active":"2023-07-13T22:12:17.987595+0000","last_became_peered":"2023-07-13T22:12:17.987595+0000","last_unstale":"2023-07-13T22:12:52.791341+0000","last_undegraded":"2023-07-13T22:12:52.791341+0000","last_fullsized":"2023-07-13T22:12:52.791341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:10:31.189715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.445763+0000","last_change":"2023-07-13T22:12:18.985162+0000","last_active":"2023-07-13T22:13:51.445763+0000","last_peered":"2023-07-13T22:13:51.445763+0000","last_clean":"2023-07-13T22:13:51.445763+0000","last_became_active":"2023-07-13T22:12:18.984816+0000","last_became_peered":"2023-07-13T22:12:18.984816+0000","last_unstale":"2023-07-13T22:13:51.445763+0000","last_undegraded":"2023-07-13T22:13:51.445763+0000","last_fullsized":"2023-07-13T22:13:51.445763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:42:22.391535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.540913+0000","last_change":"2023-07-13T22:12:17.992955+0000","last_active":"2023-07-13T22:12:52.540913+0000","last_peered":"2023-07-13T22:12:52.540913+0000","last_clean":"2023-07-13T22:12:52.540913+0000","last_became_active":"2023-07-13T22:12:17.992583+0000","last_became_peered":"2023-07-13T22:12:17.992583+0000","last_unstale":"2023-07-13T22:12:52.540913+0000","last_undegraded":"2023-07-13T22:12:52.540913+0000","last_fullsized":"2023-07-13T22:12:52.540913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:44:34.701191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465349+0000","last_change":"2023-07-13T22:12:18.969078+0000","last_active":"2023-07-13T22:13:51.465349+0000","last_peered":"2023-07-13T22:13:51.465349+0000","last_clean":"2023-07-13T22:13:51.465349+0000","last_became_active":"2023-07-13T22:12:18.968717+0000","last_became_peered":"2023-07-13T22:12:18.968717+0000","last_unstale":"2023-07-13T22:13:51.465349+0000","last_undegraded":"2023-07-13T22:13:51.465349+0000","last_fullsized":"2023-07-13T22:13:51.465349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T23:26:29.278158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.542783+0000","last_change":"2023-07-13T22:12:17.987584+0000","last_active":"2023-07-13T22:12:52.542783+0000","last_peered":"2023-07-13T22:12:52.542783+0000","last_clean":"2023-07-13T22:12:52.542783+0000","last_became_active":"2023-07-13T22:12:17.987089+0000","last_became_peered":"2023-07-13T22:12:17.987089+0000","last_unstale":"2023-07-13T22:12:52.542783+0000","last_undegraded":"2023-07-13T22:12:52.542783+0000","last_fullsized":"2023-07-13T22:12:52.542783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:04:20.180883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461401+0000","last_change":"2023-07-13T22:12:18.973422+0000","last_active":"2023-07-13T22:13:51.461401+0000","last_peered":"2023-07-13T22:13:51.461401+0000","last_clean":"2023-07-13T22:13:51.461401+0000","last_became_active":"2023-07-13T22:12:18.973036+0000","last_became_peered":"2023-07-13T22:12:18.973036+0000","last_unstale":"2023-07-13T22:13:51.461401+0000","last_undegraded":"2023-07-13T22:13:51.461401+0000","last_fullsized":"2023-07-13T22:13:51.461401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:35:53.859029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'8","reported_seq":94,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:14:01.662195+0000","last_change":"2023-07-13T22:12:52.555796+0000","last_active":"2023-07-13T22:14:01.662195+0000","last_peered":"2023-07-13T22:14:01.662195+0000","last_clean":"2023-07-13T22:14:01.662195+0000","last_became_active":"2023-07-13T22:12:52.548664+0000","last_became_peered":"2023-07-13T22:12:52.548664+0000","last_unstale":"2023-07-13T22:14:01.662195+0000","last_undegraded":"2023-07-13T22:14:01.662195+0000","last_fullsized":"2023-07-13T22:14:01.662195+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:58:37.846269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1969,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1512,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452434+0000","last_change":"2023-07-13T22:12:18.975722+0000","last_active":"2023-07-13T22:13:51.452434+0000","last_peered":"2023-07-13T22:13:51.452434+0000","last_clean":"2023-07-13T22:13:51.452434+0000","last_became_active":"2023-07-13T22:12:18.975181+0000","last_became_peered":"2023-07-13T22:12:18.975181+0000","last_unstale":"2023-07-13T22:13:51.452434+0000","last_undegraded":"2023-07-13T22:13:51.452434+0000","last_fullsized":"2023-07-13T22:13:51.452434+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-14T22:31:58.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.535107+0000","last_change":"2023-07-13T22:12:17.988759+0000","last_active":"2023-07-13T22:12:52.535107+0000","last_peered":"2023-07-13T22:12:52.535107+0000","last_clean":"2023-07-13T22:12:52.535107+0000","last_became_active":"2023-07-13T22:12:17.988463+0000","last_became_peered":"2023-07-13T22:12:17.988463+0000","last_unstale":"2023-07-13T22:12:52.535107+0000","last_undegraded":"2023-07-13T22:12:52.535107+0000","last_fullsized":"2023-07-13T22:12:52.535107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:45:29.092920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.463990+0000","last_change":"2023-07-13T22:12:18.978688+0000","last_active":"2023-07-13T22:13:51.463990+0000","last_peered":"2023-07-13T22:13:51.463990+0000","last_clean":"2023-07-13T22:13:51.463990+0000","last_became_active":"2023-07-13T22:12:18.978349+0000","last_became_peered":"2023-07-13T22:12:18.978349+0000","last_unstale":"2023-07-13T22:13:51.463990+0000","last_undegraded":"2023-07-13T22:13:51.463990+0000","last_fullsized":"2023-07-13T22:13:51.463990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:53:10.233101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:12:52.538994+0000","last_change":"2023-07-13T22:12:17.985957+0000","last_active":"2023-07-13T22:12:52.538994+0000","last_peered":"2023-07-13T22:12:52.538994+0000","last_clean":"2023-07-13T22:12:52.538994+0000","last_became_active":"2023-07-13T22:12:17.985590+0000","last_became_peered":"2023-07-13T22:12:17.985590+0000","last_unstale":"2023-07-13T22:12:52.538994+0000","last_undegraded":"2023-07-13T22:12:52.538994+0000","last_fullsized":"2023-07-13T22:12:52.538994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:16.962599+0000","last_clean_scrub_stamp":"2023-07-13T22:12:16.962599+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T07:17:31.553022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.452262+0000","last_change":"2023-07-13T22:12:18.969124+0000","last_active":"2023-07-13T22:13:51.452262+0000","last_peered":"2023-07-13T22:13:51.452262+0000","last_clean":"2023-07-13T22:13:51.452262+0000","last_became_active":"2023-07-13T22:12:18.968795+0000","last_became_peered":"2023-07-13T22:12:18.968795+0000","last_unstale":"2023-07-13T22:13:51.452262+0000","last_undegraded":"2023-07-13T22:13:51.452262+0000","last_fullsized":"2023-07-13T22:13:51.452262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:35:27.141696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448083+0000","last_change":"2023-07-13T22:12:18.977773+0000","last_active":"2023-07-13T22:13:51.448083+0000","last_peered":"2023-07-13T22:13:51.448083+0000","last_clean":"2023-07-13T22:13:51.448083+0000","last_became_active":"2023-07-13T22:12:18.977485+0000","last_became_peered":"2023-07-13T22:12:18.977485+0000","last_unstale":"2023-07-13T22:13:51.448083+0000","last_undegraded":"2023-07-13T22:13:51.448083+0000","last_fullsized":"2023-07-13T22:13:51.448083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:40:12.051633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.451991+0000","last_change":"2023-07-13T22:12:18.974631+0000","last_active":"2023-07-13T22:13:51.451991+0000","last_peered":"2023-07-13T22:13:51.451991+0000","last_clean":"2023-07-13T22:13:51.451991+0000","last_became_active":"2023-07-13T22:12:18.974319+0000","last_became_peered":"2023-07-13T22:12:18.974319+0000","last_unstale":"2023-07-13T22:13:51.451991+0000","last_undegraded":"2023-07-13T22:13:51.451991+0000","last_fullsized":"2023-07-13T22:13:51.451991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:04:24.267514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.448875+0000","last_change":"2023-07-13T22:12:18.979829+0000","last_active":"2023-07-13T22:13:51.448875+0000","last_peered":"2023-07-13T22:13:51.448875+0000","last_clean":"2023-07-13T22:13:51.448875+0000","last_became_active":"2023-07-13T22:12:18.979595+0000","last_became_peered":"2023-07-13T22:12:18.979595+0000","last_unstale":"2023-07-13T22:13:51.448875+0000","last_undegraded":"2023-07-13T22:13:51.448875+0000","last_fullsized":"2023-07-13T22:13:51.448875+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T03:24:21.951262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461338+0000","last_change":"2023-07-13T22:12:18.976527+0000","last_active":"2023-07-13T22:13:51.461338+0000","last_peered":"2023-07-13T22:13:51.461338+0000","last_clean":"2023-07-13T22:13:51.461338+0000","last_became_active":"2023-07-13T22:12:18.976211+0000","last_became_peered":"2023-07-13T22:12:18.976211+0000","last_unstale":"2023-07-13T22:13:51.461338+0000","last_undegraded":"2023-07-13T22:13:51.461338+0000","last_fullsized":"2023-07-13T22:13:51.461338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:43:45.476908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461520+0000","last_change":"2023-07-13T22:12:18.976395+0000","last_active":"2023-07-13T22:13:51.461520+0000","last_peered":"2023-07-13T22:13:51.461520+0000","last_clean":"2023-07-13T22:13:51.461520+0000","last_became_active":"2023-07-13T22:12:18.976049+0000","last_became_peered":"2023-07-13T22:12:18.976049+0000","last_unstale":"2023-07-13T22:13:51.461520+0000","last_undegraded":"2023-07-13T22:13:51.461520+0000","last_fullsized":"2023-07-13T22:13:51.461520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:01:36.995817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.447741+0000","last_change":"2023-07-13T22:12:18.975169+0000","last_active":"2023-07-13T22:13:51.447741+0000","last_peered":"2023-07-13T22:13:51.447741+0000","last_clean":"2023-07-13T22:13:51.447741+0000","last_became_active":"2023-07-13T22:12:18.974861+0000","last_became_peered":"2023-07-13T22:12:18.974861+0000","last_unstale":"2023-07-13T22:13:51.447741+0000","last_undegraded":"2023-07-13T22:13:51.447741+0000","last_fullsized":"2023-07-13T22:13:51.447741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:09:14.395567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.461321+0000","last_change":"2023-07-13T22:12:18.972071+0000","last_active":"2023-07-13T22:13:51.461321+0000","last_peered":"2023-07-13T22:13:51.461321+0000","last_clean":"2023-07-13T22:13:51.461321+0000","last_became_active":"2023-07-13T22:12:18.971646+0000","last_became_peered":"2023-07-13T22:12:18.971646+0000","last_unstale":"2023-07-13T22:13:51.461321+0000","last_undegraded":"2023-07-13T22:13:51.461321+0000","last_fullsized":"2023-07-13T22:13:51.461321+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T06:48:32.074362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459380+0000","last_change":"2023-07-13T22:12:18.969930+0000","last_active":"2023-07-13T22:13:51.459380+0000","last_peered":"2023-07-13T22:13:51.459380+0000","last_clean":"2023-07-13T22:13:51.459380+0000","last_became_active":"2023-07-13T22:12:18.969606+0000","last_became_peered":"2023-07-13T22:12:18.969606+0000","last_unstale":"2023-07-13T22:13:51.459380+0000","last_undegraded":"2023-07-13T22:13:51.459380+0000","last_fullsized":"2023-07-13T22:13:51.459380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T02:16:17.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458725+0000","last_change":"2023-07-13T22:12:52.793042+0000","last_active":"2023-07-13T22:13:51.458725+0000","last_peered":"2023-07-13T22:13:51.458725+0000","last_clean":"2023-07-13T22:13:51.458725+0000","last_became_active":"2023-07-13T22:12:52.792625+0000","last_became_peered":"2023-07-13T22:12:52.792625+0000","last_unstale":"2023-07-13T22:13:51.458725+0000","last_undegraded":"2023-07-13T22:13:51.458725+0000","last_fullsized":"2023-07-13T22:13:51.458725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T04:54:37.356733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.459347+0000","last_change":"2023-07-13T22:12:18.972113+0000","last_active":"2023-07-13T22:13:51.459347+0000","last_peered":"2023-07-13T22:13:51.459347+0000","last_clean":"2023-07-13T22:13:51.459347+0000","last_became_active":"2023-07-13T22:12:18.971836+0000","last_became_peered":"2023-07-13T22:12:18.971836+0000","last_unstale":"2023-07-13T22:13:51.459347+0000","last_undegraded":"2023-07-13T22:13:51.459347+0000","last_fullsized":"2023-07-13T22:13:51.459347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T05:16:38.425408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.466615+0000","last_change":"2023-07-13T22:12:18.980738+0000","last_active":"2023-07-13T22:13:51.466615+0000","last_peered":"2023-07-13T22:13:51.466615+0000","last_clean":"2023-07-13T22:13:51.466615+0000","last_became_active":"2023-07-13T22:12:18.980406+0000","last_became_peered":"2023-07-13T22:12:18.980406+0000","last_unstale":"2023-07-13T22:13:51.466615+0000","last_undegraded":"2023-07-13T22:13:51.466615+0000","last_fullsized":"2023-07-13T22:13:51.466615+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T01:33:38.511996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.465842+0000","last_change":"2023-07-13T22:12:18.979136+0000","last_active":"2023-07-13T22:13:51.465842+0000","last_peered":"2023-07-13T22:13:51.465842+0000","last_clean":"2023-07-13T22:13:51.465842+0000","last_became_active":"2023-07-13T22:12:18.978838+0000","last_became_peered":"2023-07-13T22:12:18.978838+0000","last_unstale":"2023-07-13T22:13:51.465842+0000","last_undegraded":"2023-07-13T22:13:51.465842+0000","last_fullsized":"2023-07-13T22:13:51.465842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T08:27:16.307630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-13T22:13:51.458793+0000","last_change":"2023-07-13T22:12:18.973431+0000","last_active":"2023-07-13T22:13:51.458793+0000","last_peered":"2023-07-13T22:13:51.458793+0000","last_clean":"2023-07-13T22:13:51.458793+0000","last_became_active":"2023-07-13T22:12:18.973093+0000","last_became_peered":"2023-07-13T22:12:18.973093+0000","last_unstale":"2023-07-13T22:13:51.458793+0000","last_undegraded":"2023-07-13T22:13:51.458793+0000","last_fullsized":"2023-07-13T22:13:51.458793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-13T22:12:17.964943+0000","last_clean_scrub_stamp":"2023-07-13T22:12:17.964943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-15T00:39:02.195085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68907,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":989,"num_read_kb":576,"num_write":245,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1512,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":206721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":445151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.39500000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.501},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.67900000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.46999999999999997,"15min":0.46600000000000003},"min":{"1min":0.39800000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.86799999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.80600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":5,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":1.093,"15min":1.093},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56999999999999995,"15min":0.56499999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.70499999999999996,"15min":0.71199999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.86199999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.437},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.49099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90900000000000003,"5min":1.357,"15min":1.357},"last":0.56399999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.63500000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91900000000000004,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.75700000000000001}]},{"osd":8,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.56399999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94899999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.59699999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.96899999999999997,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.67900000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.99299999999999999,"5min":1.335,"15min":1.335},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.61299999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88900000000000001,"5min":1.23,"15min":1.23},"last":0.78100000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678054,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1622016,"data_stored":1092849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45200000000000001,"15min":0.45400000000000001},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.54700000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48399999999999999,"15min":0.48599999999999999},"min":{"1min":0.34999999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84399999999999997,"5min":0.88,"15min":0.88},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78000000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.78300000000000003}]},{"osd":3,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.81299999999999994,"15min":0.83099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.9289999999999998,"5min":5.3220000000000001,"15min":5.3220000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.85499999999999998,"15min":0.89100000000000001},"min":{"1min":0.49399999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.56200000000000006},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.746,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54700000000000004,"15min":0.53700000000000003},"min":{"1min":0.47899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82499999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85199999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.90300000000000002,"15min":0.93400000000000005},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5980000000000001,"5min":5.2800000000000002,"15min":5.2800000000000002},"last":0.754},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93899999999999995,"15min":0.96099999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.7050000000000001,"5min":5.234,"15min":5.234},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.624},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57399999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.745,"15min":0.749},"min":{"1min":0.47099999999999997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":1.117,"15min":1.117},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.497,"15min":0.497},"max":{"1min":0.91100000000000003,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92900000000000005}]},{"osd":9,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.806,"5min":2.806,"15min":2.806},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80500000000000005,"15min":0.81100000000000005},"min":{"1min":0.63300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.94399999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.96399999999999997}]},{"osd":11,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.754,"15min":0.752},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.56100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.89800000000000002}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":444282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.749,"5min":3.5339999999999998,"15min":3.5339999999999998},"last":0.442},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.61099999999999999,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66800000000000004,"15min":0.68500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83799999999999997,"5min":1.109,"15min":1.109},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71299999999999997,"15min":0.72699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.75600000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.60899999999999999,"5min":0.504,"15min":0.504},"max":{"1min":0.91200000000000003,"5min":1.101,"15min":1.101},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.81699999999999995,"15min":0.84599999999999997},"min":{"1min":0.53600000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91900000000000004,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.67300000000000004,"15min":0.69799999999999995},"min":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.88200000000000001,"5min":3.0649999999999999,"15min":3.0649999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.69399999999999995,"15min":0.73599999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92200000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Thu Jul 13 22:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.64400000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.93700000000000006,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.85999999999999999,"15min":0.91200000000000003},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.89000000000000001,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.85099999999999998,"15min":0.91700000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95899999999999996,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.89800000000000002,"15min":0.95599999999999996},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0640000000000001,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.89400000000000002,"15min":0.93600000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95299999999999996,"5min":2.605,"15min":2.605},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.92900000000000005,"5min":1.581,"15min":1.581},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jul 13 22:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":0.98799999999999999},"min":{"1min":0.64000000000000001,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.024,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.93300000000000005,"15min":0.95599999999999996},"min":{"1min":0.61199999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.998,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.59899999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.878},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91600000000000004,"15min":0.91400000000000003},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.52,"5min":2.52,"15min":2.52},"last":0.82599999999999996}]},{"osd":11,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85499999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":444660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.48199999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.82799999999999996,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.77800000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.442}]},{"osd":1,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.58999999999999997},"min":{"1min":0.47199999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.628,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85799999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.496}]},{"osd":2,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.45800000000000002,"15min":0.44600000000000001},"min":{"1min":0.42399999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.73599999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.47799999999999998,"15min":0.45400000000000001},"min":{"1min":0.372,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63400000000000001,"15min":0.63600000000000001},"min":{"1min":0.39600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.52000000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.61199999999999999,"15min":0.57999999999999996},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88600000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":0.89300000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.629}]},{"osd":5,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59299999999999997,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84399999999999997,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.77000000000000002,"15min":0.74099999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7489999999999997,"5min":5.7489999999999997,"15min":5.7489999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.74099999999999999,"15min":0.72599999999999998},"min":{"1min":0.59099999999999997,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93700000000000006,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jul 13 22:13:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.72699999999999998,"15min":0.70299999999999996},"min":{"1min":0.65600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.73299999999999998,"15min":0.70899999999999996},"min":{"1min":0.69499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.86699999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94899999999999995,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.876,"5min":0.84099999999999997,"15min":0.81899999999999995},"min":{"1min":0.623,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.01,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.752}]},{"osd":10,"last update":"Thu Jul 13 22:13:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.59699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.77100000000000002}]},{"osd":11,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83799999999999997}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28460,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716692,"statfs":{"total":95995035648,"available":95965892608,"internally_reserved":0,"allocated":1544192,"data_stored":1036649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56200000000000006,"15min":0.55700000000000005},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.032,"15min":1.032},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80500000000000005,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.435}]},{"osd":1,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28299999999999997,"5min":0.251,"15min":0.251},"max":{"1min":0.66900000000000004,"5min":1.48,"15min":1.48},"last":0.377},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.32400000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.74199999999999999,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.317}]},{"osd":2,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83499999999999996,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77200000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":0.72799999999999998,"15min":0.67300000000000004},"min":{"1min":0.42799999999999999,"5min":0.375,"15min":0.375},"max":{"1min":10.412000000000001,"5min":10.412000000000001,"15min":10.412000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.70399999999999996,"15min":0.63500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":7.4909999999999997,"5min":7.4909999999999997,"15min":7.4909999999999997},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.71499999999999997,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.437},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.35999999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.86099999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.70399999999999996,"15min":0.67400000000000004},"min":{"1min":0.46300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":4.8090000000000002,"5min":4.8090000000000002,"15min":4.8090000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.74099999999999999,"15min":0.72899999999999998},"min":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.8940000000000001,"5min":2.8940000000000001,"15min":2.8940000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.81599999999999995,"15min":0.77800000000000002},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":4.8570000000000002,"5min":4.8570000000000002,"15min":4.8570000000000002},"last":0.51200000000000001}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71199999999999997,"15min":0.70399999999999996},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93799999999999994,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97799999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.66000000000000003}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.89100000000000001,"15min":0.84699999999999998},"min":{"1min":0.61299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.92500000000000004,"5min":10.394,"15min":10.394},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":0.94999999999999996,"15min":0.83999999999999997},"min":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":7.3310000000000004,"5min":10.452999999999999,"15min":10.452999999999999},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.47899999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76400000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.998,"5min":1.016,"15min":1.016},"last":0.69499999999999995}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78000000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":443957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.496,"15min":0.499},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.54300000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.504,"5min":0.877,"15min":0.877},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.65900000000000003,"15min":0.68899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.57599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.76700000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67100000000000004,"15min":0.65600000000000003},"min":{"1min":0.51500000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81799999999999995,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85999999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.624,"15min":0.64600000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76400000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.65400000000000003,"15min":0.67900000000000005},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":0.81100000000000005,"5min":1.085,"15min":1.085},"last":0.70999999999999996}]},{"osd":4,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.49399999999999999,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88600000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.76300000000000001,"15min":0.80300000000000005},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.84499999999999997,"5min":1.726,"15min":1.726},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.80200000000000005,"5min":1.369,"15min":1.369},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85899999999999999,"5min":1.573,"15min":1.573},"last":0.81499999999999995}]},{"osd":7,"last update":"Thu Jul 13 22:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.044,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79700000000000004,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68999999999999995}]},{"osd":8,"last update":"Thu Jul 13 22:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.82599999999999996,"15min":0.86099999999999999},"min":{"1min":0.497,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83199999999999996,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.84299999999999997,"15min":0.86099999999999999},"min":{"1min":0.63900000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91300000000000003,"5min":1.238,"15min":1.238},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.91300000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98299999999999998,"5min":1.304,"15min":1.304},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.90000000000000002,"15min":0.94499999999999995},"min":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":0.91300000000000003,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.86299999999999999}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91200000000000003,"15min":0.92000000000000004},"min":{"1min":0.73099999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.90600000000000003,"15min":0.91000000000000003},"min":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":995328,"data_stored":506694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:14:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.42699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.0189999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.0529999999999999},"last":0.443}]},{"osd":2,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.56599999999999995,"15min":0.498},"min":{"1min":0.498,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.54500000000000004,"15min":0.47899999999999998},"min":{"1min":0.503,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jul 13 22:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.60499999999999998,"15min":0.53100000000000003},"min":{"1min":0.42299999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.95599999999999996,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.63,"15min":0.55900000000000005},"min":{"1min":0.50600000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92800000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jul 13 22:13:08 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.38800000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.67200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.42099999999999999},"min":{"1min":0.41999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77500000000000002,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47699999999999998}]},{"osd":5,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.58299999999999996,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59699999999999998,"15min":0.55800000000000005},"min":{"1min":0.51400000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86199999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jul 13 22:13:51 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.53500000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.94699999999999995,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73799999999999999,"15min":0.71799999999999997},"min":{"1min":0.58299999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.872,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71199999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.52500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.438,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73499999999999999}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.56200000000000006,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.748},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":0.98299999999999998,"5min":6.6470000000000002,"15min":6.6470000000000002},"last":0.81999999999999995}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.60599999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.85299999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:13:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73499999999999999,"15min":0.72299999999999998},"min":{"1min":0.66000000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.754},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.623,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0549999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.89000000000000001}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80200000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":446137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.61699999999999999,"15min":0.53900000000000003},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":0.89600000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64400000000000002,"15min":0.56599999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.79100000000000004,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64700000000000002,"15min":0.58399999999999996},"min":{"1min":0.53100000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97299999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.64600000000000002,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.495},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Thu Jul 13 22:13:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66000000000000003,"15min":0.60399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.69799999999999995,"15min":0.629},"min":{"1min":0.42799999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":3.077,"5min":3.077,"15min":3.077},"last":0.57799999999999996}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.77900000000000003,"15min":0.746},"min":{"1min":0.60199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.952,"5min":2.952,"15min":2.952},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.70099999999999996,"15min":0.68799999999999994},"min":{"1min":0.45400000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72699999999999998,"15min":0.70099999999999996},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0740000000000001,"5min":1.331,"15min":1.331},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.79400000000000004},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.097,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.89700000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.48599999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86799999999999999,"15min":0.85299999999999998},"min":{"1min":0.73299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.62}]},{"osd":8,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.94899999999999995,"15min":0.98399999999999999},"min":{"1min":0.65500000000000003,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.871,"15min":0.90400000000000003},"min":{"1min":0.499,"5min":0.443,"15min":0.443},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.83699999999999997}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.95099999999999996,"15min":1.0169999999999999},"min":{"1min":0.51100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.99199999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.753},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.91000000000000003,"15min":0.95399999999999996},"min":{"1min":0.53200000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0529999999999999,"5min":1.925,"15min":1.925},"last":0.876}]},{"osd":10,"last update":"Thu Jul 13 22:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88900000000000001,"15min":0.89300000000000002},"min":{"1min":0.60999999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.65400000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.86099999999999999,"15min":0.84699999999999998},"min":{"1min":0.69899999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.89100000000000001,"15min":0.88700000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.3100000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.81100000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":1003520,"data_stored":505075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.60799999999999998,"15min":0.56200000000000006},"min":{"1min":0.38700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":6.3819999999999997,"5min":6.3819999999999997,"15min":6.3819999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53600000000000003,"15min":0.52500000000000002},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.57199999999999995,"15min":0.51600000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84799999999999998,"5min":0.875,"15min":0.875},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.57099999999999995,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83199999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.5}]},{"osd":3,"last update":"Thu Jul 13 22:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.65100000000000002,"15min":0.625},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.78500000000000003,"5min":4.2469999999999999,"15min":4.2469999999999999},"last":0.627},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80800000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.41499999999999998},"min":{"1min":0.315,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.48099999999999998,"15min":0.433},"min":{"1min":0.38400000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jul 13 22:13:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.67700000000000005},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96799999999999997,"5min":6.4219999999999997,"15min":6.4219999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.70399999999999996,"15min":0.67100000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.1120000000000001,"5min":3.1120000000000001,"15min":3.1120000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jul 13 22:12:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88700000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46500000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70599999999999996}]},{"osd":9,"last update":"Thu Jul 13 22:13:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.81200000000000006,"15min":0.84099999999999997},"min":{"1min":0.498,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":4.21,"15min":4.21},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88100000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Thu Jul 13 22:13:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.71199999999999997},"min":{"1min":0.53300000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.64100000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.89400000000000002}]},{"osd":11,"last update":"Thu Jul 13 22:13:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.872}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":443505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.45800000000000002},"min":{"1min":0.35199999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.853,"5min":7.8940000000000001,"15min":7.8940000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61699999999999999,"15min":0.48499999999999999},"min":{"1min":0.378,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92300000000000004,"5min":13.968999999999999,"15min":13.968999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.978,"5min":7.0460000000000003,"15min":7.0460000000000003},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.80800000000000005,"15min":0.64900000000000002},"min":{"1min":0.49399999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":13.851000000000001,"15min":13.851000000000001},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.67200000000000004,"15min":0.63200000000000001},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.7440000000000002,"5min":2.7440000000000002,"15min":2.7440000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.80600000000000005,"15min":0.67300000000000004},"min":{"1min":0.59899999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.9399999999999999,"5min":7.1319999999999997,"15min":7.1319999999999997},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":0.78100000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":7.2370000000000001,"5min":7.2370000000000001,"15min":7.2370000000000001},"last":0.754},{"interface":"front","average":{"1min":1.157,"5min":0.80700000000000005,"15min":0.69799999999999995},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.9950000000000001,"5min":6.9950000000000001,"15min":6.9950000000000001},"last":0.88}]},{"osd":5,"last update":"Thu Jul 13 22:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.82699999999999996,"15min":0.69499999999999995},"min":{"1min":0.499,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":13.731999999999999,"5min":13.731999999999999,"15min":13.731999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.47599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jul 13 22:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.67600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.879,"15min":0.75800000000000001},"min":{"1min":0.52300000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.827,"5min":13.891,"15min":13.891},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":0.93700000000000006,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":13.797000000000001,"5min":13.797000000000001,"15min":13.797000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":0.93300000000000005,"15min":0.873},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":7.1699999999999999,"5min":7.1699999999999999,"15min":7.1699999999999999},"last":0.84699999999999998}]},{"osd":8,"last update":"Thu Jul 13 22:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.874}]},{"osd":9,"last update":"Thu Jul 13 22:13:53 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.99099999999999999,"15min":0.94499999999999995},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0310000000000001,"5min":7.2709999999999999,"15min":7.2709999999999999},"last":0.624},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.034,"15min":0.95299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.3599999999999999,"5min":13.930999999999999,"15min":13.930999999999999},"last":0.60399999999999998}]},{"osd":10,"last update":"Thu Jul 13 22:13:17 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.88800000000000001},"min":{"1min":0.59899999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":2.6349999999999998,"5min":2.6349999999999998,"15min":2.6349999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.266,"5min":1.046,"15min":1.01},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":7.085,"5min":7.085,"15min":7.085},"last":0.90500000000000003}]},{"osd":11,"last update":"Thu Jul 13 22:13:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.7120000000000002,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.2030000000000003,"5min":7.2030000000000003,"15min":7.2030000000000003},"last":0.70899999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":445781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Jul 13 22:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24399999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.377,"15min":0.371},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60999999999999999,"15min":0.56499999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83899999999999997,"5min":3.6659999999999999,"15min":3.6659999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58199999999999996,"15min":0.55000000000000004},"min":{"1min":0.436,"5min":0.376,"15min":0.376},"max":{"1min":0.82299999999999995,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jul 13 22:13:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.70599999999999996,"5min":1.004,"15min":1.004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jul 13 22:13:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.55200000000000005},"min":{"1min":0.40999999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jul 13 22:13:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89300000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.71799999999999997,"15min":0.67300000000000004},"min":{"1min":0.60099999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jul 13 22:13:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.49299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76400000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.57199999999999995,"15min":0.54300000000000004},"min":{"1min":0.39700000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83899999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65300000000000002}]},{"osd":8,"last update":"Thu Jul 13 22:13:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.57199999999999995,"5min":0.438,"15min":0.438},"max":{"1min":0.92700000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":9,"last update":"Thu Jul 13 22:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.2699999999999996,"5min":4.2699999999999996,"15min":4.2699999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77100000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.77100000000000002}]},{"osd":10,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.76300000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.88600000000000001,"5min":1.01,"15min":1.01},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.85499999999999998,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":11,"last update":"Thu Jul 13 22:13:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.69299999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28472,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93716680,"statfs":{"total":95995035648,"available":95965880320,"internally_reserved":0,"allocated":1556480,"data_stored":1036423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.139,"5min":1.7170000000000001,"15min":1.7170000000000001},"last":0.622},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60499999999999998,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.2330000000000001,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.64100000000000001}]},{"osd":1,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.60599999999999998,"15min":0.56599999999999995},"min":{"1min":0.55300000000000005,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.83199999999999996,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83099999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80300000000000005}]},{"osd":2,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.434,"15min":0.39700000000000002},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65500000000000003,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.42399999999999999,"15min":0.39600000000000002},"min":{"1min":0.29699999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.66900000000000004,"5min":1.665,"15min":1.665},"last":0.503}]},{"osd":3,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.59399999999999997},"min":{"1min":0.39500000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":2.9590000000000001,"15min":2.9590000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41099999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89700000000000002,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Jul 13 22:13:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80600000000000005,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64400000000000002,"15min":0.60199999999999998},"min":{"1min":0.56799999999999995,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90200000000000002,"5min":2.9159999999999999,"15min":2.9159999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.624},"min":{"1min":0.51000000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79000000000000004,"5min":1.288,"15min":1.288},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.93899999999999995,"5min":3.823,"15min":3.823},"last":0.751}]},{"osd":7,"last update":"Thu Jul 13 22:13:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.68300000000000005,"15min":0.66000000000000003},"min":{"1min":0.46600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.875,"5min":2.875,"15min":2.875},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Jul 13 22:13:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59699999999999998,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86599999999999999,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.874},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.89600000000000002}]},{"osd":9,"last update":"Thu Jul 13 22:13:50 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.77500000000000002},"min":{"1min":0.57899999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96699999999999997,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77500000000000002,"15min":0.75800000000000001},"min":{"1min":0.60199999999999998,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":3.1139999999999999,"15min":3.1139999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Thu Jul 13 22:13:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.72099999999999997,"15min":0.68600000000000005},"min":{"1min":0.65800000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.64000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.84199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-13T22:14:05.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:14:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.041+0000 7f6a73dff700 1 -- 172.21.15.26:0/334788697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 msgr2=0x7f6a6c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.041+0000 7f6a73dff700 1 --2- 172.21.15.26:0/334788697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c101140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0098f0 tx=0x7f6a5c009c00 comp rx=0 tx=0).stop 2023-07-13T22:14:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 -- 172.21.15.26:0/334788697 shutdown_connections 2023-07-13T22:14:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 --2- 172.21.15.26:0/334788697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c101680 0x7f6a6c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 --2- 172.21.15.26:0/334788697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 --2- 172.21.15.26:0/334788697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c101140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 -- 172.21.15.26:0/334788697 >> 172.21.15.26:0/334788697 conn(0x7f6a6c0fb0e0 msgr2=0x7f6a6c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 -- 172.21.15.26:0/334788697 shutdown_connections 2023-07-13T22:14:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.042+0000 7f6a73dff700 1 -- 172.21.15.26:0/334788697 wait complete. 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 Processor -- start 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 -- start start 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c101680 0x7f6a6c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19f600 con 0x7f6a6c0ffb00 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19f740 con 0x7f6a6c100cf0 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a73dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c19f8d0 con 0x7f6a6c101680 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a71b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a71b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45276/0 (socket says 172.21.15.26:45276) 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a71b9b700 1 -- 172.21.15.26:0/995342469 learned_addr learned my addr 172.21.15.26:0/995342469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a7139a700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.043+0000 7f6a7239c700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c101680 0x7f6a6c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 -- 172.21.15.26:0/995342469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c101680 msgr2=0x7f6a6c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c101680 0x7f6a6c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 -- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 msgr2=0x7f6a6c197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 -- 172.21.15.26:0/995342469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64009d00 con 0x7f6a6c100cf0 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a71b9b700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a7139a700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c197fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009230 tx=0x7f6a5c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c01c070 con 0x7f6a6c100cf0 2023-07-13T22:14:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.044+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c009520 con 0x7f6a6c100cf0 2023-07-13T22:14:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.045+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c19fe40 con 0x7f6a6c100cf0 2023-07-13T22:14:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.045+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5c00f7f0 con 0x7f6a6c100cf0 2023-07-13T22:14:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.045+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c003970 con 0x7f6a6c100cf0 2023-07-13T22:14:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.045+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c1921e0 con 0x7f6a6c100cf0 2023-07-13T22:14:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.048+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6a5c020030 con 0x7f6a6c100cf0 2023-07-13T22:14:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.049+0000 7f6a62ffd700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 0x7f6a580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.049+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a5c0ce230 con 0x7f6a6c100cf0 2023-07-13T22:14:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.049+0000 7f6a71b9b700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 0x7f6a580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.050+0000 7f6a71b9b700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 0x7f6a580a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a68005950 tx=0x7f6a68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.051+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a5c0979c0 con 0x7f6a6c100cf0 2023-07-13T22:14:06.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.263+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a6c02caa0 con 0x7f6a6c100cf0 2023-07-13T22:14:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.265+0000 7f6a62ffd700 1 -- 172.21.15.26:0/995342469 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f6a5c09afe0 con 0x7f6a6c100cf0 2023-07-13T22:14:06.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:06.256 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:14:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 msgr2=0x7f6a580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 0x7f6a580a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a68005950 tx=0x7f6a68009260 comp rx=0 tx=0).stop 2023-07-13T22:14:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 msgr2=0x7f6a6c197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c197fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009230 tx=0x7f6a5c00b730 comp rx=0 tx=0).stop 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 shutdown_connections 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a580a1220 0x7f6a580a36d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a6c0ffb00 0x7f6a6c197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a6c101680 0x7f6a6c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 --2- 172.21.15.26:0/995342469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c100cf0 0x7f6a6c197fc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 >> 172.21.15.26:0/995342469 conn(0x7f6a6c0fb0e0 msgr2=0x7f6a6c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:06.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 shutdown_connections 2023-07-13T22:14:06.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.268+0000 7f6a73dff700 1 -- 172.21.15.26:0/995342469 wait complete. 2023-07-13T22:14:06.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:14:06.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:14:06.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/981670423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ffae0 msgr2=0x7f0f080ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/981670423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ffae0 0x7f0f080ffef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009990 tx=0x7f0ef8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/981670423 shutdown_connections 2023-07-13T22:14:06.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/981670423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ffae0 0x7f0f080ffef0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/981670423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f08101660 0x7f0f08107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/981670423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08100cd0 0x7f0f08101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.494+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/981670423 >> 172.21.15.26:0/981670423 conn(0x7f0f080fb0e0 msgr2=0x7f0f080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.495+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/981670423 shutdown_connections 2023-07-13T22:14:06.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.495+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/981670423 wait complete. 2023-07-13T22:14:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 Processor -- start 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 -- start start 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f080ffae0 0x7f0f08197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 0x7f0f0819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0819f610 con 0x7f0f08100cd0 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0819f750 con 0x7f0f080ffae0 2023-07-13T22:14:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0e3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0819f8e0 con 0x7f0f08101660 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45288/0 (socket says 172.21.15.26:45288) 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f077fe700 1 -- 172.21.15.26:0/4184567338 learned_addr learned my addr 172.21.15.26:0/4184567338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f07fff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f080ffae0 0x7f0f08197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.496+0000 7f0f0c99c700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 0x7f0f0819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 -- 172.21.15.26:0/4184567338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 msgr2=0x7f0f0819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 0x7f0f0819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 -- 172.21.15.26:0/4184567338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f080ffae0 msgr2=0x7f0f08197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f080ffae0 0x7f0f08197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 -- 172.21.15.26:0/4184567338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80095f0 con 0x7f0f08100cd0 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f0c99c700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 0x7f0f0819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f077fe700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e9a0 tx=0x7f0efc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc009790 con 0x7f0f08100cd0 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0efc00cbd0 con 0x7f0f08100cd0 2023-07-13T22:14:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc018680 con 0x7f0f08100cd0 2023-07-13T22:14:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.497+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0819fbc0 con 0x7f0f08100cd0 2023-07-13T22:14:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.498+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f081a0080 con 0x7f0f08100cd0 2023-07-13T22:14:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.499+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0810b300 con 0x7f0f08100cd0 2023-07-13T22:14:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.500+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0efc00cd40 con 0x7f0f08100cd0 2023-07-13T22:14:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.502+0000 7f0f057fa700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 0x7f0ef00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.502+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0efc014070 con 0x7f0f08100cd0 2023-07-13T22:14:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.502+0000 7f0f07fff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 0x7f0ef00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.503+0000 7f0f07fff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 0x7f0ef00a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b2e0 tx=0x7f0ef8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.505+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0efc051790 con 0x7f0f08100cd0 2023-07-13T22:14:06.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.717+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f081a0630 con 0x7f0f08100cd0 2023-07-13T22:14:06.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.718+0000 7f0f057fa700 1 -- 172.21.15.26:0/4184567338 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0efc096300 con 0x7f0f08100cd0 2023-07-13T22:14:06.709 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:14:06.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.720+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 msgr2=0x7f0ef00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.720+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 0x7f0ef00a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800b2e0 tx=0x7f0ef8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.720+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 msgr2=0x7f0f08197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.720+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e9a0 tx=0x7f0efc00ecb0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 shutdown_connections 2023-07-13T22:14:06.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f08101660 0x7f0f0819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f080ffae0 0x7f0f08197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0ef00a1150 0x7f0ef00a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 --2- 172.21.15.26:0/4184567338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f08100cd0 0x7f0f08197fd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 >> 172.21.15.26:0/4184567338 conn(0x7f0f080fb0e0 msgr2=0x7f0f080fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:06.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 shutdown_connections 2023-07-13T22:14:06.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.721+0000 7f0f0e3ff700 1 -- 172.21.15.26:0/4184567338 wait complete. 2023-07-13T22:14:06.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:14:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 -- 172.21.15.26:0/3374604141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 msgr2=0x7f4524102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 --2- 172.21.15.26:0/3374604141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f4524102940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4514009990 tx=0x7f4514009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 -- 172.21.15.26:0/3374604141 shutdown_connections 2023-07-13T22:14:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 --2- 172.21.15.26:0/3374604141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f4524102940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 --2- 172.21.15.26:0/3374604141 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45241057a0 0x7f4524107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 --2- 172.21.15.26:0/3374604141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4524102e80 0x7f4524105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 -- 172.21.15.26:0/3374604141 >> 172.21.15.26:0/3374604141 conn(0x7f45240fa000 msgr2=0x7f45240fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 -- 172.21.15.26:0/3374604141 shutdown_connections 2023-07-13T22:14:06.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.947+0000 7f452a1ff700 1 -- 172.21.15.26:0/3374604141 wait complete. 2023-07-13T22:14:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 Processor -- start 2023-07-13T22:14:06.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 -- start start 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4524102e80 0x7f45240fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45241057a0 0x7f45240fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45241996f0 con 0x7f4524100560 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45240fef90 con 0x7f45241057a0 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.948+0000 7f452a1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45240ff150 con 0x7f4524102e80 2023-07-13T22:14:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45298/0 (socket says 172.21.15.26:45298) 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 -- 172.21.15.26:0/4239524374 learned_addr learned my addr 172.21.15.26:0/4239524374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f4523fff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45241057a0 0x7f45240fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f4522ffd700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4524102e80 0x7f45240fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 -- 172.21.15.26:0/4239524374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4524102e80 msgr2=0x7f45240fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4524102e80 0x7f45240fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 -- 172.21.15.26:0/4239524374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45241057a0 msgr2=0x7f45240fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:06.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45241057a0 0x7f45240fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:06.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 -- 172.21.15.26:0/4239524374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4518009520 con 0x7f4524100560 2023-07-13T22:14:06.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.949+0000 7f45237fe700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4514004740 tx=0x7f4514009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.950+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451401d070 con 0x7f4524100560 2023-07-13T22:14:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.950+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4514004830 con 0x7f4524100560 2023-07-13T22:14:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.950+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45140053c0 con 0x7f4524100560 2023-07-13T22:14:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.950+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45140095f0 con 0x7f4524100560 2023-07-13T22:14:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.950+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45241a4280 con 0x7f4524100560 2023-07-13T22:14:06.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.951+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452410b2e0 con 0x7f4524100560 2023-07-13T22:14:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.953+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4514005560 con 0x7f4524100560 2023-07-13T22:14:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.954+0000 7f4520ff9700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 0x7f450c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:06.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.954+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45140cdee0 con 0x7f4524100560 2023-07-13T22:14:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.954+0000 7f4522ffd700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 0x7f450c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.955+0000 7f4522ffd700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 0x7f450c0a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4518009df0 tx=0x7f4518009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:06.956+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f45140976d0 con 0x7f4524100560 2023-07-13T22:14:06.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:06 smithi121 ceph-mon[129187]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:06.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:06 smithi121 ceph-mon[129187]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:06.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:06 smithi121 ceph-mon[129187]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:06.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:06 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/995342469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:06.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:06 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4184567338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.165+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45241a4830 con 0x7f4524100560 2023-07-13T22:14:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.165+0000 7f4520ff9700 1 -- 172.21.15.26:0/4239524374 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f451409acf0 con 0x7f4524100560 2023-07-13T22:14:07.156 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:14:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 msgr2=0x7f450c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 0x7f450c0a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4518009df0 tx=0x7f4518009260 comp rx=0 tx=0).stop 2023-07-13T22:14:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 msgr2=0x7f45240ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4514004740 tx=0x7f4514009580 comp rx=0 tx=0).stop 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 shutdown_connections 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f450c0a1150 0x7f450c0a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4524100560 0x7f45240ffed0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45241057a0 0x7f45240fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 --2- 172.21.15.26:0/4239524374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4524102e80 0x7f45240fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 >> 172.21.15.26:0/4239524374 conn(0x7f45240fa000 msgr2=0x7f4524101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 shutdown_connections 2023-07-13T22:14:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.168+0000 7f452a1ff700 1 -- 172.21.15.26:0/4239524374 wait complete. 2023-07-13T22:14:07.174 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:06 smithi026 ceph-mon[95647]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:06 smithi026 ceph-mon[95647]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:06 smithi026 ceph-mon[95647]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:07.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:06 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/995342469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:07.175 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:06 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4184567338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:14:07.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:06 smithi117 ceph-mon[129074]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:06 smithi117 ceph-mon[129074]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:06 smithi117 ceph-mon[129074]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:07.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:06 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/995342469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:07.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:06 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4184567338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 -- 172.21.15.26:0/2403704331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec0ffae0 msgr2=0x7f7aec0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 --2- 172.21.15.26:0/2403704331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0ffef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80098f0 tx=0x7f7ae8009c00 comp rx=0 tx=0).stop 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 -- 172.21.15.26:0/2403704331 shutdown_connections 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 --2- 172.21.15.26:0/2403704331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec101660 0x7f7aec107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 --2- 172.21.15.26:0/2403704331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0ffef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 --2- 172.21.15.26:0/2403704331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec100cd0 0x7f7aec101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 -- 172.21.15.26:0/2403704331 >> 172.21.15.26:0/2403704331 conn(0x7f7aec0fb0e0 msgr2=0x7f7aec0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 -- 172.21.15.26:0/2403704331 shutdown_connections 2023-07-13T22:14:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.395+0000 7f7af43ff700 1 -- 172.21.15.26:0/2403704331 wait complete. 2023-07-13T22:14:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.396+0000 7f7af43ff700 1 Processor -- start 2023-07-13T22:14:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.396+0000 7f7af43ff700 1 -- start start 2023-07-13T22:14:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec101660 0x7f7aec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0735e0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec073750 con 0x7f7aec101660 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af43ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec19f950 con 0x7f7aec100cd0 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af199a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45320/0 (socket says 172.21.15.26:45320) 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 -- 172.21.15.26:0/949207577 learned_addr learned my addr 172.21.15.26:0/949207577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af199a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57882/0 (socket says 172.21.15.26:57882) 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af299c700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec101660 0x7f7aec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 -- 172.21.15.26:0/949207577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 msgr2=0x7f7aec072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 -- 172.21.15.26:0/949207577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec101660 msgr2=0x7f7aec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec101660 0x7f7aec072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.397+0000 7f7af219b700 1 -- 172.21.15.26:0/949207577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0095f0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7af199a700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7af219b700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8005e90 tx=0x7f7ae800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801c070 con 0x7f7aec0ffae0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae800f7f0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae8009520 con 0x7f7aec0ffae0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8003990 con 0x7f7aec0ffae0 2023-07-13T22:14:07.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.398+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec19fec0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.399+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec10b300 con 0x7f7aec0ffae0 2023-07-13T22:14:07.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.401+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7ae8020030 con 0x7f7aec0ffae0 2023-07-13T22:14:07.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.403+0000 7f7ae37fe700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 0x7f7ad80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.403+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ae80cd810 con 0x7f7aec0ffae0 2023-07-13T22:14:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.404+0000 7f7af199a700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 0x7f7ad80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.404+0000 7f7af199a700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 0x7f7ad80a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005fd0 tx=0x7f7adc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:07.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.407+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7ae8096fa0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.617+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7aec106a90 con 0x7f7aec0ffae0 2023-07-13T22:14:07.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.618+0000 7f7ae37fe700 1 -- 172.21.15.26:0/949207577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ae809a5c0 con 0x7f7aec0ffae0 2023-07-13T22:14:07.608 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.620+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 msgr2=0x7f7ad80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.620+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 0x7f7ad80a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005fd0 tx=0x7f7adc009310 comp rx=0 tx=0).stop 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.620+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 msgr2=0x7f7aec0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.620+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8005e90 tx=0x7f7ae800b730 comp rx=0 tx=0).stop 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.620+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 shutdown_connections 2023-07-13T22:14:07.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ad80a11d0 0x7f7ad80a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aec0ffae0 0x7f7aec0742c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7aec101660 0x7f7aec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 --2- 172.21.15.26:0/949207577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7aec100cd0 0x7f7aec072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 >> 172.21.15.26:0/949207577 conn(0x7f7aec0fb0e0 msgr2=0x7f7aec0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 shutdown_connections 2023-07-13T22:14:07.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.621+0000 7f7af43ff700 1 -- 172.21.15.26:0/949207577 wait complete. 2023-07-13T22:14:07.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:14:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 -- 172.21.15.26:0/504806802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc100cf0 msgr2=0x7f3cfc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 --2- 172.21.15.26:0/504806802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc101140 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009990 tx=0x7f3cec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 -- 172.21.15.26:0/504806802 shutdown_connections 2023-07-13T22:14:07.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 --2- 172.21.15.26:0/504806802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc101680 0x7f3cfc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 --2- 172.21.15.26:0/504806802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 --2- 172.21.15.26:0/504806802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc101140 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.843+0000 7f3d045ff700 1 -- 172.21.15.26:0/504806802 >> 172.21.15.26:0/504806802 conn(0x7f3cfc0fb0e0 msgr2=0x7f3cfc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.844+0000 7f3d045ff700 1 -- 172.21.15.26:0/504806802 shutdown_connections 2023-07-13T22:14:07.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.844+0000 7f3d045ff700 1 -- 172.21.15.26:0/504806802 wait complete. 2023-07-13T22:14:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 Processor -- start 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 -- start start 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 0x7f3cfc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc19f590 con 0x7f3cfc101680 2023-07-13T22:14:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc19f6d0 con 0x7f3cfc0ffb00 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d045ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc19f860 con 0x7f3cfc100cf0 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d01b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d01b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57908/0 (socket says 172.21.15.26:57908) 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d02b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 0x7f3cfc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.845+0000 7f3d01b9a700 1 -- 172.21.15.26:0/2149071769 learned_addr learned my addr 172.21.15.26:0/2149071769 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d0239b700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 -- 172.21.15.26:0/2149071769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 msgr2=0x7f3cfc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 -- 172.21.15.26:0/2149071769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 msgr2=0x7f3cfc197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 -- 172.21.15.26:0/2149071769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0095f0 con 0x7f3cfc101680 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d01b9a700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d0239b700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d02b9c700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 0x7f3cfc19efc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400efc0 tx=0x7f3cf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4018070 con 0x7f3cfc101680 2023-07-13T22:14:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc19fae0 con 0x7f3cfc101680 2023-07-13T22:14:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cf400f040 con 0x7f3cfc101680 2023-07-13T22:14:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.846+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400cd40 con 0x7f3cfc101680 2023-07-13T22:14:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.847+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc19fff0 con 0x7f3cfc101680 2023-07-13T22:14:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.847+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc10b320 con 0x7f3cfc101680 2023-07-13T22:14:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.850+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3cf4003a00 con 0x7f3cfc101680 2023-07-13T22:14:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.852+0000 7f3cf37fe700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 0x7f3ce80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.852+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3cf40cc4c0 con 0x7f3cfc101680 2023-07-13T22:14:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.852+0000 7f3d0239b700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 0x7f3ce80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.853+0000 7f3d0239b700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 0x7f3ce80a3650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc100b50 tx=0x7f3cf8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:07.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:07.856+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3cf4095cb0 con 0x7f3cfc101680 2023-07-13T22:14:07.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:07 smithi026 ceph-mon[95647]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4239524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:07 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/949207577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:07 smithi121 ceph-mon[129187]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:07.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:07 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4239524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:07.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:07 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/949207577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.111+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cfc10b4e0 con 0x7f3cfc101680 2023-07-13T22:14:08.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.112+0000 7f3cf37fe700 1 -- 172.21.15.26:0/2149071769 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f3cf40992d0 con 0x7f3cfc101680 2023-07-13T22:14:08.102 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:08.103 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 msgr2=0x7f3ce80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 0x7f3ce80a3650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc100b50 tx=0x7f3cf8009260 comp rx=0 tx=0).stop 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 msgr2=0x7f3cfc19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 0x7f3cfc19efc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400efc0 tx=0x7f3cf400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 shutdown_connections 2023-07-13T22:14:08.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3ce80a11a0 0x7f3ce80a3650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cfc101680 0x7f3cfc19efc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3cfc0ffb00 0x7f3cfc197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 --2- 172.21.15.26:0/2149071769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cfc100cf0 0x7f3cfc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.114+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 >> 172.21.15.26:0/2149071769 conn(0x7f3cfc0fb0e0 msgr2=0x7f3cfc106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.115+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 shutdown_connections 2023-07-13T22:14:08.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.115+0000 7f3d045ff700 1 -- 172.21.15.26:0/2149071769 wait complete. 2023-07-13T22:14:08.107 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:14:08.124 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:14:08.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:07 smithi117 ceph-mon[129074]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-13T22:14:08.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:07 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4239524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:14:08.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:07 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/949207577' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:14:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 -- 172.21.15.26:0/3261681541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 msgr2=0x7f1248101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 --2- 172.21.15.26:0/3261681541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248101140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1238009990 tx=0x7f1238009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 -- 172.21.15.26:0/3261681541 shutdown_connections 2023-07-13T22:14:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 --2- 172.21.15.26:0/3261681541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 0x7f1248107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 --2- 172.21.15.26:0/3261681541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f12480fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 --2- 172.21.15.26:0/3261681541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248101140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 -- 172.21.15.26:0/3261681541 >> 172.21.15.26:0/3261681541 conn(0x7f12480fb0e0 msgr2=0x7f12480fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.337+0000 7f12503ff700 1 -- 172.21.15.26:0/3261681541 shutdown_connections 2023-07-13T22:14:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.338+0000 7f12503ff700 1 -- 172.21.15.26:0/3261681541 wait complete. 2023-07-13T22:14:08.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 Processor -- start 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 -- start start 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f1248197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 0x7f124819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819f5c0 con 0x7f1248100cf0 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819f700 con 0x7f12480ffb00 2023-07-13T22:14:08.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f12503ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124819f890 con 0x7f1248101680 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f124d99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f124d99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45350/0 (socket says 172.21.15.26:45350) 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f124d99a700 1 -- 172.21.15.26:0/228541586 learned_addr learned my addr 172.21.15.26:0/228541586 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f124e19b700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f1248197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.339+0000 7f124e99c700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 0x7f124819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 -- 172.21.15.26:0/228541586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 msgr2=0x7f124819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 0x7f124819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 -- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 msgr2=0x7f1248197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 -- 172.21.15.26:0/228541586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009520 con 0x7f12480ffb00 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f124e19b700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f1248197a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f124400e9c0 tx=0x7f124400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400cb70 con 0x7f12480ffb00 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12380095f0 con 0x7f12480ffb00 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f124400ee60 con 0x7f12480ffb00 2023-07-13T22:14:08.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.340+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244018680 con 0x7f12480ffb00 2023-07-13T22:14:08.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.341+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124819fef0 con 0x7f12480ffb00 2023-07-13T22:14:08.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.342+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12481921e0 con 0x7f12480ffb00 2023-07-13T22:14:08.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.344+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1244010430 con 0x7f12480ffb00 2023-07-13T22:14:08.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.345+0000 7f123f7fe700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 0x7f12340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.345+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1244014070 con 0x7f12480ffb00 2023-07-13T22:14:08.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.345+0000 7f124d99a700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 0x7f12340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.346+0000 7f124d99a700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 0x7f12340a3650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f123800b2e0 tx=0x7f1238005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.347+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f12440963d0 con 0x7f12480ffb00 2023-07-13T22:14:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.558+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f124802cac0 con 0x7f12480ffb00 2023-07-13T22:14:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.560+0000 7f123f7fe700 1 -- 172.21.15.26:0/228541586 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f124400ce60 con 0x7f12480ffb00 2023-07-13T22:14:08.550 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:08.551 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:14:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 msgr2=0x7f12340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 0x7f12340a3650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f123800b2e0 tx=0x7f1238005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 msgr2=0x7f1248197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f1248197a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f124400e9c0 tx=0x7f124400ecd0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 shutdown_connections 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1248101680 0x7f124819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12480ffb00 0x7f1248197a40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f12340a11a0 0x7f12340a3650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 --2- 172.21.15.26:0/228541586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1248100cf0 0x7f1248197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 >> 172.21.15.26:0/228541586 conn(0x7f12480fb0e0 msgr2=0x7f12481065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 shutdown_connections 2023-07-13T22:14:08.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.562+0000 7f12503ff700 1 -- 172.21.15.26:0/228541586 wait complete. 2023-07-13T22:14:08.571 DEBUG:tasks.fwd_scrub:Yielding 2023-07-13T22:14:08.572 INFO:teuthology.run_tasks:Running task workunit... 2023-07-13T22:14:08.585 INFO:tasks.workunit:Pulling workunits from ref 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:14:08.585 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-13T22:14:08.586 INFO:tasks.workunit:timeout=3h 2023-07-13T22:14:08.586 INFO:tasks.workunit:cleanup=True 2023-07-13T22:14:08.586 DEBUG:teuthology.orchestra.run.smithi026:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-13T22:14:08.587 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-13T22:14:08.587 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:14:08.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:14:08.632 INFO:teuthology.orchestra.run.smithi026.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-07-13 22:12:26.971276892 +0000 2023-07-13T22:14:08.633 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-07-13 22:12:26.971276892 +0000 2023-07-13T22:14:08.634 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-07-13 22:13:59.717486731 +0000 2023-07-13T22:14:08.634 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-07-13 22:12:26.971276892 +0000 2023-07-13T22:14:08.634 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-13T22:14:08.634 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-13T22:14:08.783 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3fc0089cc1082313a98853d48b892acdc1385b8b 2023-07-13T22:14:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.819+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/44992524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 msgr2=0x7f7f08101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.819+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/44992524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08101130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009990 tx=0x7f7efc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/44992524 shutdown_connections 2023-07-13T22:14:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/44992524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f08101670 0x7f7f08107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/44992524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f080ffaf0 0x7f7f080fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/44992524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08101130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/44992524 >> 172.21.15.26:0/44992524 conn(0x7f7f080fb130 msgr2=0x7f7f080fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/44992524 shutdown_connections 2023-07-13T22:14:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.820+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/44992524 wait complete. 2023-07-13T22:14:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 Processor -- start 2023-07-13T22:14:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 -- start start 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffaf0 0x7f7f080742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080735e0 con 0x7f7f08100ce0 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08073750 con 0x7f7f080ffaf0 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.821+0000 7f7f0d9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0819f8a0 con 0x7f7f08101670 2023-07-13T22:14:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57938/0 (socket says 172.21.15.26:57938) 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45364/0 (socket says 172.21.15.26:45364) 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 -- 172.21.15.26:0/128324681 learned_addr learned my addr 172.21.15.26:0/128324681 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffaf0 0x7f7f080742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 -- 172.21.15.26:0/128324681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffaf0 msgr2=0x7f7f080742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffaf0 0x7f7f080742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 -- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 msgr2=0x7f7f08072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 -- 172.21.15.26:0/128324681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f00009c40 con 0x7f7f08101670 2023-07-13T22:14:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f067fc700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.822+0000 7f7f077fe700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c370 tx=0x7f7f0000c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.823+0000 7f7f0c9fd700 1 -- 172.21.15.26:0/128324681 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00017070 con 0x7f7f08101670 2023-07-13T22:14:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.823+0000 7f7f0c9fd700 1 -- 172.21.15.26:0/128324681 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f0000f040 con 0x7f7f08101670 2023-07-13T22:14:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.823+0000 7f7f0c9fd700 1 -- 172.21.15.26:0/128324681 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000e050 con 0x7f7f08101670 2023-07-13T22:14:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.823+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0095f0 con 0x7f7f08101670 2023-07-13T22:14:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.823+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0819fe50 con 0x7f7f08101670 2023-07-13T22:14:08.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.828+0000 7f7f0c9fd700 1 -- 172.21.15.26:0/128324681 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7f0000cb60 con 0x7f7f08101670 2023-07-13T22:14:08.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.828+0000 7f7f21382b80 1 Processor -- start 2023-07-13T22:14:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 -- start start 2023-07-13T22:14:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef4191c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4191e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef41821d30 con 0x55ef4191cea0 2023-07-13T22:14:08.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef41874ac0 con 0x55ef4191dcf0 2023-07-13T22:14:08.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f21382b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4186df50 con 0x55ef4191c550 2023-07-13T22:14:08.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef4191c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45376/0 (socket says 172.21.15.26:45376) 2023-07-13T22:14:08.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f06ffd700 1 -- 172.21.15.26:0/557425908 learned_addr learned my addr 172.21.15.26:0/557425908 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.829+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef4191c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57940/0 (socket says 172.21.15.26:57940) 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f067fc700 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4191e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 -- 172.21.15.26:0/557425908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 msgr2=0x55ef4191c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef4191c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 -- 172.21.15.26:0/557425908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 msgr2=0x55ef4191e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4191e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 -- 172.21.15.26:0/557425908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef4169be60 con 0x55ef4191cea0 2023-07-13T22:14:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f067fc700 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4191e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f0c9fd700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 0x7f7ef00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009d30 tx=0x7f7ef8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1820fc0bdd2c33 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7f0c9fd700 1 -- 172.21.15.26:0/128324681 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f000cc260 con 0x7f7f08101670 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7eedffb700 1 -- 172.21.15.26:0/557425908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8004220 con 0x55ef4191cea0 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7eedffb700 1 -- 172.21.15.26:0/557425908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ef8004380 con 0x55ef4191cea0 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.830+0000 7f7eedffb700 1 -- 172.21.15.26:0/557425908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80198d0 con 0x55ef4191cea0 2023-07-13T22:14:08.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.831+0000 7f7f067fc700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 0x7f7ef00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.831+0000 7f7f21382b80 1 -- 172.21.15.26:0/557425908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 msgr2=0x55ef4191d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.831+0000 7f7f21382b80 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009d30 tx=0x7f7ef8011720 comp rx=0 tx=0).stop 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.831+0000 7f7f067fc700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 0x7f7ef00a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005cd0 tx=0x7f7efc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 -- 172.21.15.26:0/557425908 shutdown_connections 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4191e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef4191c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 --2- 172.21.15.26:0/557425908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191cea0 0x55ef4191d760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 -- 172.21.15.26:0/557425908 >> 172.21.15.26:0/557425908 conn(0x55ef41911e20 msgr2=0x55ef41917250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 -- 172.21.15.26:0/557425908 shutdown_connections 2023-07-13T22:14:08.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 -- 172.21.15.26:0/557425908 wait complete. 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.832+0000 7f7f21382b80 1 -- start start 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 Processor -- start 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191cea0 0x55ef41926680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4192d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4180db40 con 0x55ef4191c550 2023-07-13T22:14:08.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4180d840 con 0x55ef4191dcf0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef416f19a0 con 0x55ef4191cea0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.833+0000 7f7f21382b80 10 client.0 _ll_register_callbacks cb 0x55ef4180b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45390/0 (socket says 172.21.15.26:45390) 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 -- 172.21.15.26:0/2177013813 learned_addr learned my addr 172.21.15.26:0/2177013813 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f077fe700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4192d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f067fc700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191cea0 0x55ef41926680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 -- 172.21.15.26:0/2177013813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191cea0 msgr2=0x55ef41926680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191cea0 0x55ef41926680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 -- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 msgr2=0x55ef4192d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4192d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 -- 172.21.15.26:0/2177013813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef8009960 con 0x55ef4191c550 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ef801b040 tx=0x7f7ef8005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56cd7a803d50c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.834+0000 7f7ee6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8005040 con 0x55ef4191c550 2023-07-13T22:14:08.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef417c7570 con 0x55ef4191c550 2023-07-13T22:14:08.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7ef80051a0 con 0x55ef4191c550 2023-07-13T22:14:08.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ef8005360 con 0x55ef4191c550 2023-07-13T22:14:08.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7f21382b80 10 client.15348 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:08.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ef41684ce0 con 0x55ef4191c550 2023-07-13T22:14:08.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef417c7570 con 0x55ef4191c550 2023-07-13T22:14:08.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.835+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7ef8021040 con 0x55ef4191c550 2023-07-13T22:14:08.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 10 client.15348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 20 client.15348 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef41931df0 0x55ef41934200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 4 client.15348 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef41866f30 con 0x55ef41931df0 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.836+0000 7f7f067fc700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef41931df0 0x55ef41934200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.837+0000 7f7f067fc700 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef41931df0 0x55ef41934200 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a4a01869837b85 server_cookie=7123bf44ff37553e in_seq=0 out_seq=0 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.837+0000 7f7ee6ffd700 10 client.15348 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.837+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/2177013813 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef41866f30 con 0x55ef41931df0 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.837+0000 7f7ee6ffd700 10 client.15348 handle_command_reply: tid=0 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 client.15348 shutdown 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 msgr2=0x55ef41926140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ef801b040 tx=0x7f7ef8005d60 comp rx=0 tx=0).stop 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 shutdown_connections 2023-07-13T22:14:08.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef41926140 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef41931df0 0x55ef41934200 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191dcf0 0x55ef4192d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 --2- 172.21.15.26:0/2177013813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191cea0 0x55ef41926680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7ee6ffd700 0 client.15348 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 >> 172.21.15.26:0/2177013813 conn(0x55ef41911e20 msgr2=0x55ef41916930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 shutdown_connections 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 1 -- 172.21.15.26:0/2177013813 wait complete. 2023-07-13T22:14:08.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.838+0000 7f7f21382b80 20 client.15348 trim_cache size 0 max 16384 2023-07-13T22:14:08.835 INFO:tasks.workunit.client.0.smithi026.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-13T22:14:08.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.856+0000 7f7f21382b80 1 Processor -- start 2023-07-13T22:14:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.856+0000 7f7f21382b80 1 -- start start 2023-07-13T22:14:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef41920b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 0x55ef419210d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef41923df0 con 0x55ef4191dcf0 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef41921700 con 0x55ef4191cea0 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f21382b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef41931b10 con 0x55ef4191c550 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 0x55ef419210d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef41920b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60532/0 (socket says 172.21.15.26:60532) 2023-07-13T22:14:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef41920b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57952/0 (socket says 172.21.15.26:57952) 2023-07-13T22:14:08.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 -- 172.21.15.26:0/608262450 learned_addr learned my addr 172.21.15.26:0/608262450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 -- 172.21.15.26:0/608262450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 msgr2=0x55ef41920b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef41920b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 -- 172.21.15.26:0/608262450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 msgr2=0x55ef419210d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 0x55ef419210d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.857+0000 7f7f067fc700 1 -- 172.21.15.26:0/608262450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef419251d0 con 0x55ef4191cea0 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.858+0000 7f7f077fe700 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 0x55ef419210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.858+0000 7f7f067fc700 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009990 tx=0x7f7efc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7401dfb27e04c056 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.858+0000 7f7eee7fc700 1 -- 172.21.15.26:0/608262450 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc037a60 con 0x55ef4191cea0 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.858+0000 7f7eee7fc700 1 -- 172.21.15.26:0/608262450 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7efc037530 con 0x55ef4191cea0 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.858+0000 7f7eee7fc700 1 -- 172.21.15.26:0/608262450 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0049a0 con 0x55ef4191cea0 2023-07-13T22:14:08.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- 172.21.15.26:0/608262450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 msgr2=0x55ef4192d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009990 tx=0x7f7efc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- 172.21.15.26:0/608262450 shutdown_connections 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191dcf0 0x55ef419210d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191c550 0x55ef41920b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 --2- 172.21.15.26:0/608262450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192d680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- 172.21.15.26:0/608262450 >> 172.21.15.26:0/608262450 conn(0x55ef41911e20 msgr2=0x55ef4192dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- 172.21.15.26:0/608262450 shutdown_connections 2023-07-13T22:14:08.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- 172.21.15.26:0/608262450 wait complete. 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.859+0000 7f7f21382b80 1 -- start start 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 Processor -- start 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef4192ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4192c6e0 con 0x55ef4191c550 2023-07-13T22:14:08.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4192c850 con 0x55ef4191cea0 2023-07-13T22:14:08.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef4192ca10 con 0x55ef4191dcf0 2023-07-13T22:14:08.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f21382b80 10 client.0 _ll_register_callbacks cb 0x7f7f000097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f067fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57966/0 (socket says 172.21.15.26:57966) 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f077fe700 1 -- 172.21.15.26:0/672490044 learned_addr learned my addr 172.21.15.26:0/672490044 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.860+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef4192ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 -- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 msgr2=0x55ef4192b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 -- 172.21.15.26:0/672490044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 msgr2=0x55ef4192ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef4192ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 -- 172.21.15.26:0/672490044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f00082e70 con 0x55ef4191dcf0 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef4192ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f077fe700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c9d0 tx=0x7f7f00084390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7798931fa565aa2c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7ee6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000092e0 con 0x55ef4191dcf0 2023-07-13T22:14:08.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ef8004380 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f00099620 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f000b6020 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f21382b80 10 client.25366 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.861+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f7efc030040 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ef8004380 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7f00098b10 con 0x55ef4191dcf0 2023-07-13T22:14:08.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 10 client.25366 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 20 client.25366 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef419884d0 0x55ef4198a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 4 client.25366 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ef418290d0 con 0x55ef419884d0 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.862+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef419884d0 0x55ef4198a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.863+0000 7f7f06ffd700 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef419884d0 0x55ef4198a8e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ff56f0ef5033bf server_cookie=d6ad6f48ce18f97 in_seq=0 out_seq=0 2023-07-13T22:14:08.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.863+0000 7f7ee6ffd700 10 client.25366 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7ee6ffd700 1 -- 172.21.15.26:0/672490044 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ef418290d0 con 0x55ef419884d0 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7ee6ffd700 10 client.25366 handle_command_reply: tid=0 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 client.25366 shutdown 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 msgr2=0x55ef4192c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c9d0 tx=0x7f7f00084390 comp rx=0 tx=0).stop 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 shutdown_connections 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ef4191c550 0x55ef4192ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ef419884d0 0x55ef4198a8e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ef4191dcf0 0x55ef4192c1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 --2- 172.21.15.26:0/672490044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef4191cea0 0x55ef4192b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 >> 172.21.15.26:0/672490044 conn(0x55ef41911e20 msgr2=0x55ef41920b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7ee6ffd700 0 client.25366 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.864+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 shutdown_connections 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f21382b80 1 -- 172.21.15.26:0/672490044 wait complete. 2023-07-13T22:14:08.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f21382b80 20 client.25366 trim_cache size 0 max 16384 2023-07-13T22:14:08.862 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:14:08.862 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:14:08.862 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "86d5de01-5b81-44a0-940e-5048c9363cc0", 2023-07-13T22:14:08.862 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:14:08.862 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:14:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 msgr2=0x7f7ef00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 0x7f7ef00a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005cd0 tx=0x7f7efc005fb0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 msgr2=0x7f7f08072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c370 tx=0x7f7f0000c680 comp rx=0 tx=0).stop 2023-07-13T22:14:08.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 shutdown_connections 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f08101670 0x7f7f08072e40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f080ffaf0 0x7f7f080742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.865+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ef00a11d0 0x7f7ef00a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.866+0000 7f7f0d9ff700 1 --2- 172.21.15.26:0/128324681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f08100ce0 0x7f7f08072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.866+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 >> 172.21.15.26:0/128324681 conn(0x7f7f080fb130 msgr2=0x7f7f080fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.866+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 shutdown_connections 2023-07-13T22:14:08.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:08.866+0000 7f7f0d9ff700 1 -- 172.21.15.26:0/128324681 wait complete. 2023-07-13T22:14:08.872 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:08 smithi026 ceph-mon[95647]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-13T22:14:08.872 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:08 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2149071769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:08.872 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:08 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/228541586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:08.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:14:08.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:08 smithi121 ceph-mon[129187]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-13T22:14:08.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:08 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2149071769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:08.995 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:08 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/228541586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:09.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 -- 172.21.15.26:0/678474410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 msgr2=0x7f44f40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/678474410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f40fff00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009990 tx=0x7f44f0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 -- 172.21.15.26:0/678474410 shutdown_connections 2023-07-13T22:14:09.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/678474410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f40fff00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/678474410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4101670 0x7f44f4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/678474410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4100ce0 0x7f44f4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 -- 172.21.15.26:0/678474410 >> 172.21.15.26:0/678474410 conn(0x7f44f40fb130 msgr2=0x7f44f40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 -- 172.21.15.26:0/678474410 shutdown_connections 2023-07-13T22:14:09.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.137+0000 7f44fc3ff700 1 -- 172.21.15.26:0/678474410 wait complete. 2023-07-13T22:14:09.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 Processor -- start 2023-07-13T22:14:09.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 -- start start 2023-07-13T22:14:09.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 0x7f44f4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4101670 0x7f44f419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f419f5c0 con 0x7f44f40ffaf0 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f419f700 con 0x7f44f4101670 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.138+0000 7f44fc3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f419f890 con 0x7f44f4100ce0 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45434/0 (socket says 172.21.15.26:45434) 2023-07-13T22:14:09.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 -- 172.21.15.26:0/1841416836 learned_addr learned my addr 172.21.15.26:0/1841416836 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa99c700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4101670 0x7f44f419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44f999a700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 0x7f44f4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 -- 172.21.15.26:0/1841416836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 msgr2=0x7f44f4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 0x7f44f4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 -- 172.21.15.26:0/1841416836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4101670 msgr2=0x7f44f419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4101670 0x7f44f419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 -- 172.21.15.26:0/1841416836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f00095f0 con 0x7f44f40ffaf0 2023-07-13T22:14:09.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44f999a700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 0x7f44f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44f00059d0 tx=0x7f44f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44eb7fe700 1 -- 172.21.15.26:0/1841416836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f001d070 con 0x7f44f40ffaf0 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f419fb10 con 0x7f44f40ffaf0 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44eb7fe700 1 -- 172.21.15.26:0/1841416836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44f0004020 con 0x7f44f40ffaf0 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.139+0000 7f44eb7fe700 1 -- 172.21.15.26:0/1841416836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0022d00 con 0x7f44f40ffaf0 2023-07-13T22:14:09.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.140+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f419ff90 con 0x7f44f40ffaf0 2023-07-13T22:14:09.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.143+0000 7f44eb7fe700 1 -- 172.21.15.26:0/1841416836 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44f0022470 con 0x7f44f40ffaf0 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.144+0000 7f450fde5b80 1 Processor -- start 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.144+0000 7f450fde5b80 1 -- start start 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.144+0000 7f44eb7fe700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 0x7f44e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.144+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.144+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 0x5570971366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x5570971370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f450fde5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709703acb0 con 0x5570971354d0 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f450fde5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709708da40 con 0x557097136c70 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f450fde5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557097086ed0 con 0x557097135e20 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44eb7fe700 1 -- 172.21.15.26:0/1841416836 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44f00cddd0 con 0x7f44f40ffaf0 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa99c700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 0x7f44e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45436/0 (socket says 172.21.15.26:45436) 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa19b700 1 -- 172.21.15.26:0/3143196621 learned_addr learned my addr 172.21.15.26:0/3143196621 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 0x5570971366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa99c700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x5570971370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 -- 172.21.15.26:0/3143196621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 msgr2=0x5570971370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x5570971370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 -- 172.21.15.26:0/3143196621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 msgr2=0x5570971358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44f999a700 1 -- 172.21.15.26:0/3143196621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557096eb4de0 con 0x557097135e20 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.145+0000 7f44fa19b700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.146+0000 7f44fa99c700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 0x7f44e00a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009c10 tx=0x7f44ec004040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.146+0000 7f44f999a700 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 0x5570971366e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44e400bcd0 tx=0x7f44e400f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a3bc7b4cc81362 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.146+0000 7f44e8bfe700 1 -- 172.21.15.26:0/3143196621 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400fbe0 con 0x557097135e20 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.146+0000 7f44e8bfe700 1 -- 172.21.15.26:0/3143196621 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e400fd40 con 0x557097135e20 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.146+0000 7f44e8bfe700 1 -- 172.21.15.26:0/3143196621 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4010610 con 0x557097135e20 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- 172.21.15.26:0/3143196621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 msgr2=0x5570971366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 0x5570971366e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44e400bcd0 tx=0x7f44e400f360 comp rx=0 tx=0).stop 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- 172.21.15.26:0/3143196621 shutdown_connections 2023-07-13T22:14:09.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x5570971358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x5570971370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 --2- 172.21.15.26:0/3143196621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097135e20 0x5570971366e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- 172.21.15.26:0/3143196621 >> 172.21.15.26:0/3143196621 conn(0x55709712ada0 msgr2=0x5570971301d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- 172.21.15.26:0/3143196621 shutdown_connections 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- 172.21.15.26:0/3143196621 wait complete. 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.147+0000 7f450fde5b80 1 -- start start 2023-07-13T22:14:09.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 Processor -- start 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 0x55709713f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097136c70 0x5570971466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557097026ac0 con 0x5570971354d0 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570970267c0 con 0x557097135e20 2023-07-13T22:14:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557096f0a920 con 0x557097136c70 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f450fde5b80 10 client.0 _ll_register_callbacks cb 0x557097024760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45448/0 (socket says 172.21.15.26:45448) 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f44fa19b700 1 -- 172.21.15.26:0/523572598 learned_addr learned my addr 172.21.15.26:0/523572598 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f44fa99c700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097136c70 0x5570971466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.148+0000 7f44f999a700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 0x55709713f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 -- 172.21.15.26:0/523572598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097136c70 msgr2=0x5570971466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097136c70 0x5570971466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 -- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 msgr2=0x55709713f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 0x55709713f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 -- 172.21.15.26:0/523572598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f419ff90 con 0x5570971354d0 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44f999a700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 0x55709713f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44fa19b700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005cd0 tx=0x7f44f0084960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1088bc403fdbbec2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f009b220 con 0x5570971354d0 2023-07-13T22:14:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44f009f250 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557096fe04f0 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f00a35e0 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f450fde5b80 10 client.15366 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.149+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f44e400b900 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557096fe04f0 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f44f009a710 con 0x5570971354d0 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 10 client.15366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 20 client.15366 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55709714aaf0 0x55709714cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 4 client.15366 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55709707feb0 con 0x55709714aaf0 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.150+0000 7f44f999a700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55709714aaf0 0x55709714cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.151+0000 7f44f999a700 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55709714aaf0 0x55709714cf00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363e9755b5897bcd server_cookie=7b4333bf04e978e4 in_seq=0 out_seq=0 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.151+0000 7f44ddffb700 10 client.15366 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.151+0000 7f44ddffb700 1 -- 172.21.15.26:0/523572598 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55709707feb0 con 0x55709714aaf0 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f44ddffb700 10 client.15366 handle_command_reply: tid=0 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 client.15366 shutdown 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 msgr2=0x55709713f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005cd0 tx=0x7f44f0084960 comp rx=0 tx=0).stop 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 shutdown_connections 2023-07-13T22:14:09.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5570971354d0 0x55709713f150 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55709714aaf0 0x55709714cf00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557097136c70 0x5570971466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 --2- 172.21.15.26:0/523572598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097135e20 0x55709713f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f44ddffb700 0 client.15366 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 >> 172.21.15.26:0/523572598 conn(0x55709712ada0 msgr2=0x55709712f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 shutdown_connections 2023-07-13T22:14:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 1 -- 172.21.15.26:0/523572598 wait complete. 2023-07-13T22:14:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.152+0000 7f450fde5b80 20 client.15366 trim_cache size 0 max 16384 2023-07-13T22:14:09.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 Processor -- start 2023-07-13T22:14:09.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 -- start start 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x55709713d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x557097146690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570971314f0 con 0x557097135e20 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709713e630 con 0x557097136c70 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.170+0000 7f450fde5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709713db80 con 0x5570971354d0 2023-07-13T22:14:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45466/0 (socket says 172.21.15.26:45466) 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 -- 172.21.15.26:0/2299200538 learned_addr learned my addr 172.21.15.26:0/2299200538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44fa19b700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x55709713d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44fa99c700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x557097146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 -- 172.21.15.26:0/2299200538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 msgr2=0x55709713d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x55709713d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 -- 172.21.15.26:0/2299200538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 msgr2=0x557097146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x557097146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 -- 172.21.15.26:0/2299200538 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557096e9dc60 con 0x557097135e20 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44fa99c700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x557097146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.171+0000 7f44f999a700 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f44e400bcd0 tx=0x7f44e400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8691901be2d41006 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f44e93ff700 1 -- 172.21.15.26:0/2299200538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400feb0 con 0x557097135e20 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f44e93ff700 1 -- 172.21.15.26:0/2299200538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e4011e90 con 0x557097135e20 2023-07-13T22:14:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f44e93ff700 1 -- 172.21.15.26:0/2299200538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004540 con 0x557097135e20 2023-07-13T22:14:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 -- 172.21.15.26:0/2299200538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 msgr2=0x55709713d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f44e400bcd0 tx=0x7f44e400b860 comp rx=0 tx=0).stop 2023-07-13T22:14:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 -- 172.21.15.26:0/2299200538 shutdown_connections 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x557097146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x55709713d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 --2- 172.21.15.26:0/2299200538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x55709713d540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 -- 172.21.15.26:0/2299200538 >> 172.21.15.26:0/2299200538 conn(0x55709712ada0 msgr2=0x55709713de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 -- 172.21.15.26:0/2299200538 shutdown_connections 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.172+0000 7f450fde5b80 1 -- 172.21.15.26:0/2299200538 wait complete. 2023-07-13T22:14:09.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 -- start start 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 Processor -- start 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x55709719e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557097146910 con 0x557097135e20 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.173+0000 7f450fde5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557097146a80 con 0x557097136c70 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f450fde5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709719eee0 con 0x5570971354d0 2023-07-13T22:14:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f450fde5b80 10 client.0 _ll_register_callbacks cb 0x7f44f009ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44f999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:58028/0 (socket says 172.21.15.26:58028) 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44f999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45468/0 (socket says 172.21.15.26:45468) 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 -- 172.21.15.26:0/1563387918 learned_addr learned my addr 172.21.15.26:0/1563387918 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa99c700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x55709719e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 -- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 msgr2=0x55709719e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x55709719e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 -- 172.21.15.26:0/1563387918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 msgr2=0x5570971438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 -- 172.21.15.26:0/1563387918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557096eb4de0 con 0x5570971354d0 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44f999a700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44fa19b700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005a00 tx=0x7f44f00997d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35dc72f76092756 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.174+0000 7f44ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f00a33e0 con 0x5570971354d0 2023-07-13T22:14:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44f009f250 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44e400fd40 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44f009b220 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 10 client.25384 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557096edb250 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e400fd40 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f44f009a710 con 0x5570971354d0 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.175+0000 7f450fde5b80 20 client.25384 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5570971a0420 0x5570971a2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f450fde5b80 4 client.25384 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557097042050 con 0x5570971a0420 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f44f999a700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5570971a0420 0x5570971a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f44f999a700 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5570971a0420 0x5570971a2830 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9393ad7fba7b09ec server_cookie=cf32a857d0cd1676 in_seq=0 out_seq=0 2023-07-13T22:14:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.176+0000 7f44ddffb700 10 client.25384 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f44ddffb700 1 -- 172.21.15.26:0/1563387918 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557097042050 con 0x5570971a0420 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f44ddffb700 10 client.25384 handle_command_reply: tid=0 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 client.25384 shutdown 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 msgr2=0x5570971433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005a00 tx=0x7f44f00997d0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 shutdown_connections 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5570971a0420 0x5570971a2830 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557097136c70 0x55709719e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570971354d0 0x5570971433b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 --2- 172.21.15.26:0/1563387918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557097135e20 0x5570971438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f44ddffb700 0 client.25384 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 >> 172.21.15.26:0/1563387918 conn(0x55709712ada0 msgr2=0x55709713d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 shutdown_connections 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 1 -- 172.21.15.26:0/1563387918 wait complete. 2023-07-13T22:14:09.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.177+0000 7f450fde5b80 20 client.25384 trim_cache size 0 max 16384 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "86d5de01-5b81-44a0-940e-5048c9363cc0": { 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "86d5de01-5b81-44a0-940e-5048c9363cc0", 2023-07-13T22:14:09.176 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:14:09.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:14:09.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:14:09.177 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:14:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 msgr2=0x7f44e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 0x7f44e00a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009c10 tx=0x7f44ec004040 comp rx=0 tx=0).stop 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 msgr2=0x7f44f4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44f00059d0 tx=0x7f44f0005740 comp rx=0 tx=0).stop 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 shutdown_connections 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44e00a1150 0x7f44e00a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44f40ffaf0 0x7f44f4197a40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44f4101670 0x7f44f419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 --2- 172.21.15.26:0/1841416836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f4100ce0 0x7f44f4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 >> 172.21.15.26:0/1841416836 conn(0x7f44f40fb130 msgr2=0x7f44f4106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 shutdown_connections 2023-07-13T22:14:09.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:09.178+0000 7f44fc3ff700 1 -- 172.21.15.26:0/1841416836 wait complete. 2023-07-13T22:14:09.186 INFO:tasks.cephfs.filesystem:scrub status for tag:86d5de01-5b81-44a0-940e-5048c9363cc0 - {'path': '/', 'tag': '86d5de01-5b81-44a0-940e-5048c9363cc0', 'options': 'recursive,force'} 2023-07-13T22:14:09.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:08 smithi117 ceph-mon[129074]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2023-07-13T22:14:09.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:08 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2149071769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:09.194 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:08 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/228541586' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:14:10.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:10 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:14:10.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:10 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:10 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:14:10.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:10 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:10 smithi121 ceph-mon[129187]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:10 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:14:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:10 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:10 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:14:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:10 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:10 smithi117 ceph-mon[129074]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:10.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:10 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:14:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:10 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:10 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:14:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:10 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:10.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:10 smithi026 ceph-mon[95647]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2023-07-13T22:14:12.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:12 smithi121 ceph-mon[129187]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-13T22:14:12.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:12 smithi117 ceph-mon[129074]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-13T22:14:12.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:12 smithi026 ceph-mon[95647]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2023-07-13T22:14:14.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:14 smithi121 ceph-mon[129187]: pgmap v326: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-13T22:14:14.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:14 smithi117 ceph-mon[129074]: pgmap v326: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-13T22:14:14.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:14 smithi026 ceph-mon[95647]: pgmap v326: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-07-13T22:14:16.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:16 smithi121 ceph-mon[129187]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-13T22:14:16.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:16 smithi117 ceph-mon[129074]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-13T22:14:16.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:16 smithi026 ceph-mon[95647]: pgmap v327: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-07-13T22:14:18.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:18 smithi121 ceph-mon[129187]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2023-07-13T22:14:18.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:18 smithi117 ceph-mon[129074]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2023-07-13T22:14:18.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:18 smithi026 ceph-mon[95647]: pgmap v328: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 11 op/s 2023-07-13T22:14:20.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:20 smithi121 ceph-mon[129187]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-13T22:14:20.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:20 smithi117 ceph-mon[129074]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-13T22:14:20.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:20 smithi026 ceph-mon[95647]: pgmap v329: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 10 op/s 2023-07-13T22:14:22.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:22 smithi121 ceph-mon[129187]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2023-07-13T22:14:22.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:22 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:14:22.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:22 smithi117 ceph-mon[129074]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2023-07-13T22:14:22.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:22 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:14:22.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:22 smithi026 ceph-mon[95647]: pgmap v330: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 9 op/s 2023-07-13T22:14:22.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:22 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:14:24.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:24 smithi121 ceph-mon[129187]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-13T22:14:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:24 smithi117 ceph-mon[129074]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-13T22:14:24.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:24 smithi026 ceph-mon[95647]: pgmap v331: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 12 op/s 2023-07-13T22:14:25.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:25 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:14:25.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:25 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:14:25.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:25 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:14:26.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:26 smithi121 ceph-mon[129187]: pgmap v332: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-13T22:14:26.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:26 smithi117 ceph-mon[129074]: pgmap v332: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-13T22:14:26.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:26 smithi026 ceph-mon[95647]: pgmap v332: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 9 op/s 2023-07-13T22:14:28.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: pgmap v333: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:28 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: pgmap v333: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:28.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:28 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.495 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: pgmap v333: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:28.495 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:28.495 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:28 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:29.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:14:29.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:14:29.444 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:14:29.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:14:29.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:29.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:14:29.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:14:29.494 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:14:30.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:30 smithi121 ceph-mon[129187]: pgmap v334: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:30.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:30 smithi117 ceph-mon[129074]: pgmap v334: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:30.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:30 smithi026 ceph-mon[95647]: pgmap v334: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:32.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:32 smithi121 ceph-mon[129187]: pgmap v335: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:32.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:32 smithi117 ceph-mon[129074]: pgmap v335: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:32.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:32 smithi026 ceph-mon[95647]: pgmap v335: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:34.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:34 smithi121 ceph-mon[129187]: pgmap v336: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:34.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:34 smithi117 ceph-mon[129074]: pgmap v336: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:34.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:34 smithi026 ceph-mon[95647]: pgmap v336: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:36.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:36 smithi121 ceph-mon[129187]: pgmap v337: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:14:36.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:36 smithi117 ceph-mon[129074]: pgmap v337: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:14:36.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:36 smithi026 ceph-mon[95647]: pgmap v337: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:14:38.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:38 smithi121 ceph-mon[129187]: pgmap v338: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:38.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:38 smithi117 ceph-mon[129074]: pgmap v338: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:38.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:38 smithi026 ceph-mon[95647]: pgmap v338: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-07-13T22:14:39.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 -- 172.21.15.26:0/4000744708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b0101690 msgr2=0x7f67b0107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 --2- 172.21.15.26:0/4000744708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b0101690 0x7f67b0107f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67a800b0a0 tx=0x7f67a800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 -- 172.21.15.26:0/4000744708 shutdown_connections 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 --2- 172.21.15.26:0/4000744708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b0101690 0x7f67b0107f50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 --2- 172.21.15.26:0/4000744708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b00ffb10 0x7f67b00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.436+0000 7f67b57ff700 1 --2- 172.21.15.26:0/4000744708 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0100d00 0x7f67b0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.437+0000 7f67b57ff700 1 -- 172.21.15.26:0/4000744708 >> 172.21.15.26:0/4000744708 conn(0x7f67b00fb130 msgr2=0x7f67b00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.437+0000 7f67b57ff700 1 -- 172.21.15.26:0/4000744708 shutdown_connections 2023-07-13T22:14:39.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.437+0000 7f67b57ff700 1 -- 172.21.15.26:0/4000744708 wait complete. 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 Processor -- start 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 -- start start 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b0100d00 0x7f67b0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0101690 0x7f67b0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00735e0 con 0x7f67b00ffb10 2023-07-13T22:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0073750 con 0x7f67b0101690 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.438+0000 7f67b57ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b019f9c0 con 0x7f67b0100d00 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49788/0 (socket says 172.21.15.26:49788) 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 -- 172.21.15.26:0/88369522 learned_addr learned my addr 172.21.15.26:0/88369522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b0100d00 0x7f67b0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67af7fe700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0101690 0x7f67b0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 -- 172.21.15.26:0/88369522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b0100d00 msgr2=0x7f67b0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b0100d00 0x7f67b0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 -- 172.21.15.26:0/88369522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0101690 msgr2=0x7f67b0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0101690 0x7f67b0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.439+0000 7f67aeffd700 1 -- 172.21.15.26:0/88369522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009d00 con 0x7f67b00ffb10 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f67aeffd700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e970 tx=0x7f67a000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f6797fff700 1 -- 172.21.15.26:0/88369522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0009790 con 0x7f67b00ffb10 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f6797fff700 1 -- 172.21.15.26:0/88369522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a000c7e0 con 0x7f67b00ffb10 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b019fca0 con 0x7f67b00ffb10 2023-07-13T22:14:39.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f6797fff700 1 -- 172.21.15.26:0/88369522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0003c70 con 0x7f67b00ffb10 2023-07-13T22:14:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.440+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b01a0180 con 0x7f67b00ffb10 2023-07-13T22:14:39.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.443+0000 7f6797fff700 1 -- 172.21.15.26:0/88369522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67a000cc50 con 0x7f67b00ffb10 2023-07-13T22:14:39.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.445+0000 7f6797fff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 0x7f67980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.445+0000 7f6797fff700 1 -- 172.21.15.26:0/88369522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67a0014070 con 0x7f67b00ffb10 2023-07-13T22:14:39.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.445+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 0x7f67980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 Processor -- start 2023-07-13T22:14:39.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 0x7f67980a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009c60 tx=0x7f67a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 -- start start 2023-07-13T22:14:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130d9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9cc70 0x561130d9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130ca0cb0 con 0x561130d9b4d0 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130cf3a40 con 0x561130d9be20 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130ceced0 con 0x561130d9cc70 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49796/0 (socket says 172.21.15.26:49796) 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.446+0000 7f67aeffd700 1 -- 172.21.15.26:0/2149992578 learned_addr learned my addr 172.21.15.26:0/2149992578 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130d9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67af7fe700 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9cc70 0x561130d9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 -- 172.21.15.26:0/2149992578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9cc70 msgr2=0x561130d9d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9cc70 0x561130d9d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 -- 172.21.15.26:0/2149992578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 msgr2=0x561130d9c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130d9c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 -- 172.21.15.26:0/2149992578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561130b1ade0 con 0x561130d9b4d0 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67aeffd700 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a0096940 tx=0x7f67a0003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b316642c39d4ceb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67957fa700 1 -- 172.21.15.26:0/2149992578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a009df00 con 0x561130d9b4d0 2023-07-13T22:14:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67957fa700 1 -- 172.21.15.26:0/2149992578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a00a2320 con 0x561130d9b4d0 2023-07-13T22:14:39.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.447+0000 7f67957fa700 1 -- 172.21.15.26:0/2149992578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00993c0 con 0x561130d9b4d0 2023-07-13T22:14:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2149992578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 msgr2=0x561130d9b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67a0096940 tx=0x7f67a0003710 comp rx=0 tx=0).stop 2023-07-13T22:14:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2149992578 shutdown_connections 2023-07-13T22:14:39.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9b4d0 0x561130d9b8e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9cc70 0x561130d9d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2149992578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130d9c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2149992578 >> 172.21.15.26:0/2149992578 conn(0x561130d90da0 msgr2=0x561130d961d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.448+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2149992578 shutdown_connections 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.449+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2149992578 wait complete. 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.449+0000 7f67c91d9b80 1 -- start start 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.449+0000 7f67c91d9b80 1 Processor -- start 2023-07-13T22:14:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130da56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130dac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130c8cac0 con 0x561130d9be20 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130c8c7c0 con 0x561130d9cc70 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130b70920 con 0x561130d9b4d0 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67c91d9b80 10 client.0 _ll_register_callbacks cb 0x561130c8a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52728/0 (socket says 172.21.15.26:52728) 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67aeffd700 1 -- 172.21.15.26:0/528751685 learned_addr learned my addr 172.21.15.26:0/528751685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130da56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.450+0000 7f67af7fe700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130dac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 -- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 msgr2=0x561130dac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130dac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 -- 172.21.15.26:0/528751685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 msgr2=0x561130da56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130da56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 -- 172.21.15.26:0/528751685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b01a0180 con 0x561130d9b4d0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67aeffd700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67a00963d0 tx=0x7f67a009df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feda1eb348c691f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f6792ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a009a280 con 0x561130d9b4d0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67a00ca270 con 0x561130d9b4d0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561130c464f0 con 0x561130d9b4d0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a009a3e0 con 0x561130d9b4d0 2023-07-13T22:14:39.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67c91d9b80 10 client.25396 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.451+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561130b03c60 con 0x561130d9b4d0 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a007c020 con 0x561130d9b4d0 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f67a00a1f20 con 0x561130d9b4d0 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 10 client.25396 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 20 client.25396 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130db0c00 0x561130db3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 4 client.25396 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:39.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561130ce5eb0 con 0x561130db0c00 2023-07-13T22:14:39.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.452+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130db0c00 0x561130db3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.453+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130db0c00 0x561130db3010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b0f67c5d4af93 server_cookie=c219834da993f47d in_seq=0 out_seq=0 2023-07-13T22:14:39.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.453+0000 7f6792ff5700 10 client.25396 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:39.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f6792ff5700 1 -- 172.21.15.26:0/528751685 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561130ce5eb0 con 0x561130db0c00 2023-07-13T22:14:39.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f6792ff5700 10 client.25396 handle_command_reply: tid=0 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 client.25396 shutdown 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 msgr2=0x561130da51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f67a00963d0 tx=0x7f67a009df00 comp rx=0 tx=0).stop 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 shutdown_connections 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130db0c00 0x561130db3010 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130dac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130da51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/528751685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130da56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 >> 172.21.15.26:0/528751685 conn(0x561130d90da0 msgr2=0x561130d958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f6792ff5700 0 client.25396 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 shutdown_connections 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.454+0000 7f67c91d9b80 1 -- 172.21.15.26:0/528751685 wait complete. 2023-07-13T22:14:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.455+0000 7f67c91d9b80 20 client.25396 trim_cache size 0 max 16384 2023-07-13T22:14:39.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 Processor -- start 2023-07-13T22:14:39.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 -- start start 2023-07-13T22:14:39.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130dac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da4d80 con 0x561130d9cc70 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da6330 con 0x561130d9be20 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da2e20 con 0x561130d9b4d0 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67af7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49818/0 (socket says 172.21.15.26:49818) 2023-07-13T22:14:39.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67ae7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130dac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67af7fe700 1 -- 172.21.15.26:0/2247434518 learned_addr learned my addr 172.21.15.26:0/2247434518 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.476+0000 7f67aeffd700 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 -- 172.21.15.26:0/2247434518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 msgr2=0x561130dad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 -- 172.21.15.26:0/2247434518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 msgr2=0x561130db2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 -- 172.21.15.26:0/2247434518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561130b41250 con 0x561130d9be20 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67af7fe700 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130dac6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561130db38c0 tx=0x7f67a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f23378345ae70a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f6795ffb700 1 -- 172.21.15.26:0/2247434518 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4022030 con 0x561130d9be20 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f6795ffb700 1 -- 172.21.15.26:0/2247434518 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a4023040 con 0x561130d9be20 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.477+0000 7f6795ffb700 1 -- 172.21.15.26:0/2247434518 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a4003d80 con 0x561130d9be20 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2247434518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 msgr2=0x561130dac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130dac6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561130db38c0 tx=0x7f67a4014000 comp rx=0 tx=0).stop 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2247434518 shutdown_connections 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9cc70 0x561130db2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/2247434518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9be20 0x561130dac6f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2247434518 >> 172.21.15.26:0/2247434518 conn(0x561130d90da0 msgr2=0x561130da3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2247434518 shutdown_connections 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.478+0000 7f67c91d9b80 1 -- 172.21.15.26:0/2247434518 wait complete. 2023-07-13T22:14:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.479+0000 7f67c91d9b80 1 -- start start 2023-07-13T22:14:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.479+0000 7f67c91d9b80 1 Processor -- start 2023-07-13T22:14:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130dac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130da89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da8f20 con 0x561130d9be20 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da9060 con 0x561130d9cc70 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561130da9220 con 0x561130d9b4d0 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67c91d9b80 10 client.0 _ll_register_callbacks cb 0x7f67a40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52740/0 (socket says 172.21.15.26:52740) 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 -- 172.21.15.26:0/3066824703 learned_addr learned my addr 172.21.15.26:0/3066824703 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130dac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67af7fe700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130da89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 -- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 msgr2=0x561130da89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130da89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 -- 172.21.15.26:0/3066824703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 msgr2=0x561130dac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.480+0000 7f67aeffd700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130dac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.481+0000 7f67aeffd700 1 -- 172.21.15.26:0/3066824703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0108880 con 0x561130d9b4d0 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.481+0000 7f67af7fe700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130da89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.481+0000 7f67aeffd700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67a0083d80 tx=0x7f67a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26292e421ac79c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.481+0000 7f6792ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.482+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0099ca0 con 0x561130d9b4d0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.482+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67a007d080 con 0x561130d9b4d0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.482+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67a00a2320 con 0x561130d9b4d0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67a00ca480 con 0x561130d9b4d0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 10 client.25402 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f67a4017070 con 0x561130d9b4d0 2023-07-13T22:14:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67a00a2320 con 0x561130d9b4d0 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f67a0095cc0 con 0x561130d9b4d0 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 10 client.25402 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 20 client.25402 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130e05ad0 0x561130e07ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 4 client.25402 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561130ca8050 con 0x561130e05ad0 2023-07-13T22:14:39.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.483+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130e05ad0 0x561130e07ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.484+0000 7f6792ff5700 10 client.25402 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.484+0000 7f67ae7fc700 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130e05ad0 0x561130e07ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee38d0f5ef82381 server_cookie=e7ee80221686719 in_seq=0 out_seq=0 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.484+0000 7f6792ff5700 1 -- 172.21.15.26:0/3066824703 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561130ca8050 con 0x561130e05ad0 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.484+0000 7f6792ff5700 10 client.25402 handle_command_reply: tid=0 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.484+0000 7f67c91d9b80 1 client.25402 shutdown 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 msgr2=0x561130dabf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67a0083d80 tx=0x7f67a0000bc0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 shutdown_connections 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561130e05ad0 0x561130e07ee0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561130d9cc70 0x561130da89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561130d9b4d0 0x561130dabf40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 --2- 172.21.15.26:0/3066824703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561130d9be20 0x561130dac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f6792ff5700 0 client.25402 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 >> 172.21.15.26:0/3066824703 conn(0x561130d90da0 msgr2=0x561130db2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 shutdown_connections 2023-07-13T22:14:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 1 -- 172.21.15.26:0/3066824703 wait complete. 2023-07-13T22:14:39.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.485+0000 7f67c91d9b80 20 client.25402 trim_cache size 0 max 16384 2023-07-13T22:14:39.481 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:14:39.481 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:14:39.481 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:14:39.481 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:14:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 msgr2=0x7f67980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 0x7f67980a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009c60 tx=0x7f67a4009260 comp rx=0 tx=0).stop 2023-07-13T22:14:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 msgr2=0x7f67b00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67a000e970 tx=0x7f67a000ec80 comp rx=0 tx=0).stop 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 shutdown_connections 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f67980a1150 0x7f67980a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67b00ffb10 0x7f67b00742c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67b0101690 0x7f67b0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 --2- 172.21.15.26:0/88369522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67b0100d00 0x7f67b0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 >> 172.21.15.26:0/88369522 conn(0x7f67b00fb130 msgr2=0x7f67b00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 shutdown_connections 2023-07-13T22:14:39.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.486+0000 7f67b57ff700 1 -- 172.21.15.26:0/88369522 wait complete. 2023-07-13T22:14:39.495 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:14:39.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:14:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 -- 172.21.15.26:0/3790449946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 msgr2=0x7f6c94107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 --2- 172.21.15.26:0/3790449946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c94107f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00b0a0 tx=0x7f6c8c00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 -- 172.21.15.26:0/3790449946 shutdown_connections 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 --2- 172.21.15.26:0/3790449946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c94107f40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 --2- 172.21.15.26:0/3790449946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 0x7f6c940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 --2- 172.21.15.26:0/3790449946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 -- 172.21.15.26:0/3790449946 >> 172.21.15.26:0/3790449946 conn(0x7f6c940fb0e0 msgr2=0x7f6c940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 -- 172.21.15.26:0/3790449946 shutdown_connections 2023-07-13T22:14:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.720+0000 7f6c9adff700 1 -- 172.21.15.26:0/3790449946 wait complete. 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.721+0000 7f6c9adff700 1 Processor -- start 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.721+0000 7f6c9adff700 1 -- start start 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 0x7f6c94197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c9419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419f660 con 0x7f6c94101680 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419f7a0 con 0x7f6c940ffb00 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9adff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9419f930 con 0x7f6c94100cf0 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c9939c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c9419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c98b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 0x7f6c94197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c93fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52766/0 (socket says 172.21.15.26:52766) 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c93fff700 1 -- 172.21.15.26:0/954900794 learned_addr learned my addr 172.21.15.26:0/954900794 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.722+0000 7f6c93fff700 1 -- 172.21.15.26:0/954900794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 msgr2=0x7f6c94197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c93fff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 0x7f6c94197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c93fff700 1 -- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 msgr2=0x7f6c9419f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c93fff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c9419f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c93fff700 1 -- 172.21.15.26:0/954900794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c009d00 con 0x7f6c94100cf0 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c93fff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000e9a0 tx=0x7f6c8000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80009790 con 0x7f6c94100cf0 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c80010430 con 0x7f6c94100cf0 2023-07-13T22:14:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80018680 con 0x7f6c94100cf0 2023-07-13T22:14:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9419fc10 con 0x7f6c94100cf0 2023-07-13T22:14:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.723+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941a00d0 con 0x7f6c94100cf0 2023-07-13T22:14:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.724+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9410b330 con 0x7f6c94100cf0 2023-07-13T22:14:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.727+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6c800105a0 con 0x7f6c94100cf0 2023-07-13T22:14:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.729+0000 7f6c91ffb700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 0x7f6c840a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.729+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6c80014070 con 0x7f6c94100cf0 2023-07-13T22:14:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.729+0000 7f6c98b9b700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 0x7f6c840a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.729+0000 7f6c98b9b700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 0x7f6c840a36d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c94100b50 tx=0x7f6c88009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:39.730+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c800963c0 con 0x7f6c94100cf0 2023-07-13T22:14:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.015+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c9410b550 con 0x7f6c94100cf0 2023-07-13T22:14:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.016+0000 7f6c91ffb700 1 -- 172.21.15.26:0/954900794 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f6c80026020 con 0x7f6c94100cf0 2023-07-13T22:14:40.006 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:14:40.006 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:14:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 msgr2=0x7f6c840a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 0x7f6c840a36d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6c94100b50 tx=0x7f6c88009260 comp rx=0 tx=0).stop 2023-07-13T22:14:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 msgr2=0x7f6c94198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000e9a0 tx=0x7f6c8000ecb0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 shutdown_connections 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6c840a1220 0x7f6c840a36d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c94101680 0x7f6c9419f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c940ffb00 0x7f6c94197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 --2- 172.21.15.26:0/954900794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c94100cf0 0x7f6c94198020 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 >> 172.21.15.26:0/954900794 conn(0x7f6c940fb0e0 msgr2=0x7f6c940fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 shutdown_connections 2023-07-13T22:14:40.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.020+0000 7f6c9adff700 1 -- 172.21.15.26:0/954900794 wait complete. 2023-07-13T22:14:40.012 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:14:40.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:14:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2764570483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c101670 msgr2=0x7f8b9c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2764570483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c101670 0x7f8b9c107f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009f90 tx=0x7f8b9400b330 comp rx=0 tx=0).stop 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2764570483 shutdown_connections 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2764570483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2764570483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c101670 0x7f8b9c107f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2764570483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 0x7f8b9c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2764570483 >> 172.21.15.26:0/2764570483 conn(0x7f8b9c0fb130 msgr2=0x7f8b9c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.313+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2764570483 shutdown_connections 2023-07-13T22:14:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.314+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2764570483 wait complete. 2023-07-13T22:14:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.315+0000 7f8ba23ff700 1 Processor -- start 2023-07-13T22:14:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.315+0000 7f8ba23ff700 1 -- start start 2023-07-13T22:14:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 0x7f8b9c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c101670 0x7f8b9c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c19f430 con 0x7f8b9c101670 2023-07-13T22:14:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c19f570 con 0x7f8b9c100ce0 2023-07-13T22:14:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba23ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c19f700 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8b9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8b9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52792/0 (socket says 172.21.15.26:52792) 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8b9bfff700 1 -- 172.21.15.26:0/2707621735 learned_addr learned my addr 172.21.15.26:0/2707621735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 0x7f8b9c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.316+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c101670 0x7f8b9c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 -- 172.21.15.26:0/2707621735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 msgr2=0x7f8b9c197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 0x7f8b9c197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 -- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c101670 msgr2=0x7f8b9c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c101670 0x7f8b9c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 -- 172.21.15.26:0/2707621735 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c009520 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ea50 tx=0x7f8b8c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b997fa700 1 -- 172.21.15.26:0/2707621735 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c00cbd0 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b900095f0 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b997fa700 1 -- 172.21.15.26:0/2707621735 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8c00cd30 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.317+0000 7f8b997fa700 1 -- 172.21.15.26:0/2707621735 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c018680 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.318+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b94009c40 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.322+0000 7f8b997fa700 1 -- 172.21.15.26:0/2707621735 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8b8c0188a0 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.323+0000 7f8b997fa700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 0x7f8b840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.323+0000 7f8b997fa700 1 -- 172.21.15.26:0/2707621735 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b8c014070 con 0x7f8b9c0ffaf0 2023-07-13T22:14:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.324+0000 7f8bb5c7fb80 1 Processor -- start 2023-07-13T22:14:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.324+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 0x7f8b840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.324+0000 7f8bb5c7fb80 1 -- start start 2023-07-13T22:14:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc780368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc780376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77f3bcb0 con 0x55dc78036e20 2023-07-13T22:14:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77f8ea40 con 0x55dc780364d0 2023-07-13T22:14:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77f87ed0 con 0x55dc78037c70 2023-07-13T22:14:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 0x7f8b840a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005fd0 tx=0x7f8b90009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8ba099c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8ba099c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52798/0 (socket says 172.21.15.26:52798) 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8ba099c700 1 -- 172.21.15.26:0/2069015930 learned_addr learned my addr 172.21.15.26:0/2069015930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc780368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc780376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8ba099c700 1 -- 172.21.15.26:0/2069015930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 msgr2=0x55dc780368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.325+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc780368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8ba099c700 1 -- 172.21.15.26:0/2069015930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 msgr2=0x55dc780376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc780376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8ba099c700 1 -- 172.21.15.26:0/2069015930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc77db5de0 con 0x55dc78037c70 2023-07-13T22:14:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b94015000 tx=0x7f8b94009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b777cc3dd6278d4e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8b827fc700 1 -- 172.21.15.26:0/2069015930 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400e040 con 0x55dc78037c70 2023-07-13T22:14:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8b827fc700 1 -- 172.21.15.26:0/2069015930 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b94004a50 con 0x55dc78037c70 2023-07-13T22:14:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.326+0000 7f8b827fc700 1 -- 172.21.15.26:0/2069015930 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940052a0 con 0x55dc78037c70 2023-07-13T22:14:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2069015930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 msgr2=0x55dc780380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8b94015000 tx=0x7f8b94009510 comp rx=0 tx=0).stop 2023-07-13T22:14:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2069015930 shutdown_connections 2023-07-13T22:14:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc780380e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc780368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2069015930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc780376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.327+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2069015930 >> 172.21.15.26:0/2069015930 conn(0x55dc7802bda0 msgr2=0x55dc780311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.328+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2069015930 shutdown_connections 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.328+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2069015930 wait complete. 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.328+0000 7f8bb5c7fb80 1 -- start start 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 Processor -- start 2023-07-13T22:14:40.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78036e20 0x55dc78040630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78037c70 0x55dc78047670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77f27ac0 con 0x55dc78037c70 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77f277c0 con 0x55dc78036e20 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.329+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc77e0b920 con 0x55dc780364d0 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8bb5c7fb80 10 client.0 _ll_register_callbacks cb 0x55dc77f25760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52804/0 (socket says 172.21.15.26:52804) 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 -- 172.21.15.26:0/3990568129 learned_addr learned my addr 172.21.15.26:0/3990568129 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78036e20 0x55dc78040630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8ba099c700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78037c70 0x55dc78047670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 -- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78036e20 msgr2=0x55dc78040630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78036e20 0x55dc78040630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 -- 172.21.15.26:0/3990568129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78037c70 msgr2=0x55dc78047670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78037c70 0x55dc78047670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.330+0000 7f8b9bfff700 1 -- 172.21.15.26:0/3990568129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c108860 con 0x55dc780364d0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0ca900 tx=0x7f8b8c09a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5948833c6a1f472b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8b7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b1070 con 0x55dc780364d0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b8c0a2660 con 0x55dc780364d0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc77ee14f0 con 0x55dc780364d0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.331+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b8c0a21d0 con 0x55dc780364d0 2023-07-13T22:14:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.332+0000 7f8bb5c7fb80 10 client.25420 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.332+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc77d9ec60 con 0x55dc780364d0 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.332+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc77ee14f0 con 0x55dc780364d0 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b8c099290 con 0x55dc780364d0 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 10 client.25420 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 10 client.25420 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 20 client.25420 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc7804bf00 0x55dc7804e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 4 client.25420 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.333+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc77f80eb0 con 0x55dc7804bf00 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.334+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc7804bf00 0x55dc7804e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.334+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc7804bf00 0x55dc7804e310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c2b7561b9a124eab server_cookie=8ccf36bfbcec2dd5 in_seq=0 out_seq=0 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.334+0000 7f8b7b7fe700 10 client.25420 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.335+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/3990568129 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc77f80eb0 con 0x55dc7804bf00 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.335+0000 7f8b7b7fe700 10 client.25420 handle_command_reply: tid=0 2023-07-13T22:14:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.335+0000 7f8bb5c7fb80 1 client.25420 shutdown 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.335+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 msgr2=0x55dc780400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.335+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0ca900 tx=0x7f8b8c09a750 comp rx=0 tx=0).stop 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 shutdown_connections 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc7804bf00 0x55dc7804e310 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78037c70 0x55dc78047670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc780364d0 0x55dc780400f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/3990568129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78036e20 0x55dc78040630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 >> 172.21.15.26:0/3990568129 conn(0x55dc7802bda0 msgr2=0x55dc780308b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8b7b7fe700 0 client.25420 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:14:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 shutdown_connections 2023-07-13T22:14:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/3990568129 wait complete. 2023-07-13T22:14:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.336+0000 7f8bb5c7fb80 20 client.25420 trim_cache size 0 max 16384 2023-07-13T22:14:40.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.363+0000 7f8bb5c7fb80 1 Processor -- start 2023-07-13T22:14:40.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.363+0000 7f8bb5c7fb80 1 -- start start 2023-07-13T22:14:40.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc780364d0 0x55dc78047ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78036e20 0x55dc78047630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7803fcc0 con 0x55dc780364d0 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc78032cc0 con 0x55dc78037c70 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc7802b9f0 con 0x55dc78036e20 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37872/0 (socket says 172.21.15.26:37872) 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 -- 172.21.15.26:0/2884210725 learned_addr learned my addr 172.21.15.26:0/2884210725 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78036e20 0x55dc78047630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 -- 172.21.15.26:0/2884210725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78036e20 msgr2=0x55dc78047630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78036e20 0x55dc78047630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 -- 172.21.15.26:0/2884210725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc780364d0 msgr2=0x55dc78047ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc780364d0 0x55dc78047ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.364+0000 7f8ba099c700 1 -- 172.21.15.26:0/2884210725 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc77ddc250 con 0x55dc78037c70 2023-07-13T22:14:40.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.365+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400d010 tx=0x7f8b9400b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433fcf709799d399 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.365+0000 7f8b82ffd700 1 -- 172.21.15.26:0/2884210725 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400e040 con 0x55dc78037c70 2023-07-13T22:14:40.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.365+0000 7f8b82ffd700 1 -- 172.21.15.26:0/2884210725 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b9400bc80 con 0x55dc78037c70 2023-07-13T22:14:40.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.365+0000 7f8b82ffd700 1 -- 172.21.15.26:0/2884210725 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9401b7b0 con 0x55dc78037c70 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2884210725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 msgr2=0x55dc7803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400d010 tx=0x7f8b9400b9f0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2884210725 shutdown_connections 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc780364d0 0x55dc78047ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc78037c70 0x55dc7803dfc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2884210725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78036e20 0x55dc78047630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2884210725 >> 172.21.15.26:0/2884210725 conn(0x55dc7802bda0 msgr2=0x55dc78052c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2884210725 shutdown_connections 2023-07-13T22:14:40.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.366+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2884210725 wait complete. 2023-07-13T22:14:40.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.367+0000 7f8bb5c7fb80 1 -- start start 2023-07-13T22:14:40.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.367+0000 7f8bb5c7fb80 1 Processor -- start 2023-07-13T22:14:40.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc78043330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc78046890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc78046dd0 con 0x55dc78036e20 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc78046f40 con 0x55dc780364d0 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc78047100 con 0x55dc78037c70 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8bb5c7fb80 10 client.0 _ll_register_callbacks cb 0x7f8b8c099bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8b9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8b9b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49898/0 (socket says 172.21.15.26:49898) 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8b9b7fe700 1 -- 172.21.15.26:0/2307968352 learned_addr learned my addr 172.21.15.26:0/2307968352 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc78043330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.368+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc78046890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 -- 172.21.15.26:0/2307968352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 msgr2=0x55dc78046890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc78046890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 -- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 msgr2=0x55dc78043330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc78043330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 -- 172.21.15.26:0/2307968352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc77db5de0 con 0x55dc78036e20 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc78043330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8ba099c700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc78046890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b9b7fe700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b900095c0 tx=0x7f8b900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ed4b81323d2614 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.369+0000 7f8b7b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:14:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90023040 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b94004a50 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8b90030070 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b9003a030 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8bb5c7fb80 10 client.15411 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc77db42b0 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.370+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b94004a50 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8b9002a040 con 0x55dc78036e20 2023-07-13T22:14:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 10 client.15411 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 10 client.15411 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:14:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 20 client.15411 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc780b1db0 0x55dc780b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 4 client.15411 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:14:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc77f43050 con 0x55dc780b1db0 2023-07-13T22:14:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.371+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc780b1db0 0x55dc780b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8b9bfff700 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc780b1db0 0x55dc780b41c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d6d66873d18c292 server_cookie=43ae6c35d1cc35b in_seq=0 out_seq=0 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8b7b7fe700 10 client.15411 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8b7b7fe700 1 -- 172.21.15.26:0/2307968352 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc77f43050 con 0x55dc780b1db0 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8b7b7fe700 10 client.15411 handle_command_reply: tid=0 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 client.15411 shutdown 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 msgr2=0x55dc78043870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8b900095c0 tx=0x7f8b900047c0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 shutdown_connections 2023-07-13T22:14:40.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55dc780b1db0 0x55dc780b41c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc78037c70 0x55dc78046890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc780364d0 0x55dc78043330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 --2- 172.21.15.26:0/2307968352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc78036e20 0x55dc78043870 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 >> 172.21.15.26:0/2307968352 conn(0x55dc7802bda0 msgr2=0x55dc78047ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8b7b7fe700 0 client.15411 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.372+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 shutdown_connections 2023-07-13T22:14:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8bb5c7fb80 1 -- 172.21.15.26:0/2307968352 wait complete. 2023-07-13T22:14:40.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8bb5c7fb80 20 client.15411 trim_cache size 0 max 16384 2023-07-13T22:14:40.369 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:14:40.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 msgr2=0x7f8b840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 0x7f8b840a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b90005fd0 tx=0x7f8b90009310 comp rx=0 tx=0).stop 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 msgr2=0x7f8b9c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.373+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ea50 tx=0x7f8b8c00ed60 comp rx=0 tx=0).stop 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 shutdown_connections 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8b840a1150 0x7f8b840a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b9c101670 0x7f8b9c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b9c0ffaf0 0x7f8b9c1978b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 --2- 172.21.15.26:0/2707621735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9c100ce0 0x7f8b9c197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 >> 172.21.15.26:0/2707621735 conn(0x7f8b9c0fb130 msgr2=0x7f8b9c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 shutdown_connections 2023-07-13T22:14:40.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.374+0000 7f8ba23ff700 1 -- 172.21.15.26:0/2707621735 wait complete. 2023-07-13T22:14:40.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:14:40.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 -- 172.21.15.26:0/2648292046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c100560 msgr2=0x7f981c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 --2- 172.21.15.26:0/2648292046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c100560 0x7f981c102940 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9804009990 tx=0x7f9804009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 -- 172.21.15.26:0/2648292046 shutdown_connections 2023-07-13T22:14:40.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 --2- 172.21.15.26:0/2648292046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c100560 0x7f981c102940 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 --2- 172.21.15.26:0/2648292046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c1057a0 0x7f981c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 --2- 172.21.15.26:0/2648292046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 0x7f981c105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.637+0000 7f9821bff700 1 -- 172.21.15.26:0/2648292046 >> 172.21.15.26:0/2648292046 conn(0x7f981c0fa000 msgr2=0x7f981c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.638+0000 7f9821bff700 1 -- 172.21.15.26:0/2648292046 shutdown_connections 2023-07-13T22:14:40.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.638+0000 7f9821bff700 1 -- 172.21.15.26:0/2648292046 wait complete. 2023-07-13T22:14:40.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.640+0000 7f9821bff700 1 Processor -- start 2023-07-13T22:14:40.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.640+0000 7f9821bff700 1 -- start start 2023-07-13T22:14:40.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.640+0000 7f9821bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.640+0000 7f9821bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 0x7f981c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f9821bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c1057a0 0x7f981c0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f9821bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0ff1f0 con 0x7f981c1057a0 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f9821bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0ff360 con 0x7f981c102e80 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f9821bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c19f8b0 con 0x7f981c100560 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f981bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c1057a0 0x7f981c0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52850/0 (socket says 172.21.15.26:52850) 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f981b7fe700 1 -- 172.21.15.26:0/2612484317 learned_addr learned my addr 172.21.15.26:0/2612484317 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.641+0000 7f981affd700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 0x7f981c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 -- 172.21.15.26:0/2612484317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 msgr2=0x7f981c0fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 0x7f981c0fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 -- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c1057a0 msgr2=0x7f981c0fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c1057a0 0x7f981c0fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 -- 172.21.15.26:0/2612484317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040095f0 con 0x7f981c100560 2023-07-13T22:14:40.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f981b7fe700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f980400b680 tx=0x7f9804005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f9818ff9700 1 -- 172.21.15.26:0/2612484317 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980401d070 con 0x7f981c100560 2023-07-13T22:14:40.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f9818ff9700 1 -- 172.21.15.26:0/2612484317 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9804022470 con 0x7f981c100560 2023-07-13T22:14:40.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f9818ff9700 1 -- 172.21.15.26:0/2612484317 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804017650 con 0x7f981c100560 2023-07-13T22:14:40.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c19fb30 con 0x7f981c100560 2023-07-13T22:14:40.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.642+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c19ff00 con 0x7f981c100560 2023-07-13T22:14:40.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.648+0000 7f9818ff9700 1 -- 172.21.15.26:0/2612484317 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f98040225e0 con 0x7f981c100560 2023-07-13T22:14:40.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.649+0000 7f9835503b80 1 Processor -- start 2023-07-13T22:14:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 -- start start 2023-07-13T22:14:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31ba7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba8480 0x55cb31ba88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31aa3cb0 con 0x55cb31ba8480 2023-07-13T22:14:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31af6a40 con 0x55cb31ba6ce0 2023-07-13T22:14:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.650+0000 7f9835503b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31aefed0 con 0x55cb31ba7630 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37902/0 (socket says 172.21.15.26:37902) 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 -- 172.21.15.26:0/1525627341 learned_addr learned my addr 172.21.15.26:0/1525627341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981b7fe700 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31ba7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 -- 172.21.15.26:0/1525627341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 msgr2=0x55cb31ba7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31ba7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f9818ff9700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 0x7f98080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 -- 172.21.15.26:0/1525627341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba8480 msgr2=0x55cb31ba88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba8480 0x55cb31ba88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f981bfff700 1 -- 172.21.15.26:0/1525627341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb3191dde0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.651+0000 7f9818ff9700 1 -- 172.21.15.26:0/2612484317 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98040cdc50 con 0x7f981c100560 2023-07-13T22:14:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.652+0000 7f981bfff700 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9814011600 tx=0x7f9814009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8047f22ae5ba0ee9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.652+0000 7f9811ffb700 1 -- 172.21.15.26:0/1525627341 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981400e040 con 0x55cb31ba6ce0 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.652+0000 7f9811ffb700 1 -- 172.21.15.26:0/1525627341 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f981400f040 con 0x55cb31ba6ce0 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.652+0000 7f9811ffb700 1 -- 172.21.15.26:0/1525627341 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814007760 con 0x55cb31ba6ce0 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.652+0000 7f981bfff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 0x7f98080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 -- 172.21.15.26:0/1525627341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 msgr2=0x55cb31ba70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9814011600 tx=0x7f9814009920 comp rx=0 tx=0).stop 2023-07-13T22:14:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f981bfff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 0x7f98080a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9814014fd0 tx=0x7f98140078c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 -- 172.21.15.26:0/1525627341 shutdown_connections 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba8480 0x55cb31ba88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba6ce0 0x55cb31ba70f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 --2- 172.21.15.26:0/1525627341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31ba7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.653+0000 7f9835503b80 1 -- 172.21.15.26:0/1525627341 >> 172.21.15.26:0/1525627341 conn(0x55cb31b93da0 msgr2=0x55cb31b991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.654+0000 7f9835503b80 1 -- 172.21.15.26:0/1525627341 shutdown_connections 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.654+0000 7f9835503b80 1 -- 172.21.15.26:0/1525627341 wait complete. 2023-07-13T22:14:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.654+0000 7f9835503b80 1 -- start start 2023-07-13T22:14:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 Processor -- start 2023-07-13T22:14:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31a8fac0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31a8f7c0 con 0x55cb31ba7630 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31973920 con 0x55cb31ba8480 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.655+0000 7f9835503b80 10 client.0 _ll_register_callbacks cb 0x55cb31a8d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49924/0 (socket says 172.21.15.26:49924) 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 -- 172.21.15.26:0/4273070715 learned_addr learned my addr 172.21.15.26:0/4273070715 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52874/0 (socket says 172.21.15.26:52874) 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981affd700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 -- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 msgr2=0x55cb31bb7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 -- 172.21.15.26:0/4273070715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 msgr2=0x55cb31bb0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981b7fe700 1 -- 172.21.15.26:0/4273070715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98140112b0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981bfff700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.656+0000 7f981affd700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f981b7fe700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9804097b10 tx=0x7f9804022ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4027bbc4e0d066b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f97faffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980409a790 con 0x55cb31ba6ce0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f980409b0a0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb31a494f0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980409aa00 con 0x55cb31ba6ce0 2023-07-13T22:14:40.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f9835503b80 10 client.15420 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:40.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.657+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb31906c60 con 0x55cb31ba6ce0 2023-07-13T22:14:40.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb31a494f0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9804096e90 con 0x55cb31ba6ce0 2023-07-13T22:14:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 10 client.15420 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:14:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 20 client.15420 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31bbc250 0x55cb31bbe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 4 client.15420 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.658+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb31ae8eb0 con 0x55cb31bbc250 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.659+0000 7f981affd700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31bbc250 0x55cb31bbe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.659+0000 7f981affd700 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31bbc250 0x55cb31bbe630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb894956d63a7be3 server_cookie=7ac7f41a4fb7e559 in_seq=0 out_seq=0 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.659+0000 7f97faffd700 10 client.15420 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f97faffd700 1 -- 172.21.15.26:0/4273070715 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb31ae8eb0 con 0x55cb31bbc250 2023-07-13T22:14:40.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f97faffd700 10 client.15420 handle_command_reply: tid=0 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 client.15420 shutdown 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 msgr2=0x55cb31bb0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9804097b10 tx=0x7f9804022ba0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 shutdown_connections 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31bbc250 0x55cb31bbe630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb0970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f97faffd700 0 client.15420 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 --2- 172.21.15.26:0/4273070715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.660+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 >> 172.21.15.26:0/4273070715 conn(0x55cb31b93da0 msgr2=0x55cb31b988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.661+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 shutdown_connections 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.661+0000 7f9835503b80 1 -- 172.21.15.26:0/4273070715 wait complete. 2023-07-13T22:14:40.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.661+0000 7f9835503b80 20 client.15420 trim_cache size 0 max 16384 2023-07-13T22:14:40.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.675+0000 7f9835503b80 1 Processor -- start 2023-07-13T22:14:40.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.675+0000 7f9835503b80 1 -- start start 2023-07-13T22:14:40.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.675+0000 7f9835503b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31baf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f9835503b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f9835503b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f9835503b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31bae620 con 0x55cb31ba6ce0 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f9835503b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31bafbd0 con 0x55cb31ba7630 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f9835503b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31bb0020 con 0x55cb31ba8480 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31baf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31baf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49934/0 (socket says 172.21.15.26:49934) 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981b7fe700 1 -- 172.21.15.26:0/227384425 learned_addr learned my addr 172.21.15.26:0/227384425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37928/0 (socket says 172.21.15.26:37928) 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981bfff700 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 -- 172.21.15.26:0/227384425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 msgr2=0x55cb31bb87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 -- 172.21.15.26:0/227384425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 msgr2=0x55cb31baf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31baf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.676+0000 7f981affd700 1 -- 172.21.15.26:0/227384425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb31944250 con 0x55cb31ba7630 2023-07-13T22:14:40.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.677+0000 7f981affd700 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55cb31a8f830 tx=0x7f980c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7577ba5238f5d416 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.677+0000 7f98127fc700 1 -- 172.21.15.26:0/227384425 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c017040 con 0x55cb31ba7630 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.677+0000 7f98127fc700 1 -- 172.21.15.26:0/227384425 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980c01d070 con 0x55cb31ba7630 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.677+0000 7f98127fc700 1 -- 172.21.15.26:0/227384425 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c003ed0 con 0x55cb31ba7630 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 -- 172.21.15.26:0/227384425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 msgr2=0x55cb31bb7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55cb31a8f830 tx=0x7f980c004040 comp rx=0 tx=0).stop 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 -- 172.21.15.26:0/227384425 shutdown_connections 2023-07-13T22:14:40.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31baf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba8480 0x55cb31bb87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 --2- 172.21.15.26:0/227384425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba7630 0x55cb31bb7eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 -- 172.21.15.26:0/227384425 >> 172.21.15.26:0/227384425 conn(0x55cb31b93da0 msgr2=0x55cb31bc2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 -- 172.21.15.26:0/227384425 shutdown_connections 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.678+0000 7f9835503b80 1 -- 172.21.15.26:0/227384425 wait complete. 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.679+0000 7f9835503b80 1 -- start start 2023-07-13T22:14:40.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.679+0000 7f9835503b80 1 Processor -- start 2023-07-13T22:14:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba8480 0x55cb31bb7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31c100d0 con 0x55cb31ba6ce0 2023-07-13T22:14:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31c10210 con 0x55cb31ba8480 2023-07-13T22:14:40.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb31c10350 con 0x55cb31ba7630 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f9835503b80 10 client.0 _ll_register_callbacks cb 0x55cb31a8d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f981affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f981affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52884/0 (socket says 172.21.15.26:52884) 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f981affd700 1 -- 172.21.15.26:0/2454357893 learned_addr learned my addr 172.21.15.26:0/2454357893 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.681+0000 7f981bfff700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba8480 0x55cb31bb7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 -- 172.21.15.26:0/2454357893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba8480 msgr2=0x55cb31bb7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba8480 0x55cb31bb7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 -- 172.21.15.26:0/2454357893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 msgr2=0x55cb31bb6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 -- 172.21.15.26:0/2454357893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c004470 con 0x55cb31ba7630 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f981affd700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f980c004320 tx=0x7f980c003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1f7fe0338dfaab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f97faffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.682+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c019630 con 0x55cb31ba7630 2023-07-13T22:14:40.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.683+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f980c005050 con 0x55cb31ba7630 2023-07-13T22:14:40.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.683+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f981400f040 con 0x55cb31ba7630 2023-07-13T22:14:40.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.683+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980c005210 con 0x55cb31ba7630 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.683+0000 7f9835503b80 10 client.25444 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.683+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb3191c2b0 con 0x55cb31ba7630 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f981400f040 con 0x55cb31ba7630 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f980c0096e0 con 0x55cb31ba7630 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 10 client.25444 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 20 client.25444 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31c11c10 0x55cb31c14020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 4 client.25444 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb31aab050 con 0x55cb31c11c10 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.684+0000 7f981b7fe700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31c11c10 0x55cb31c14020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f981b7fe700 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31c11c10 0x55cb31c14020 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3700e71fdc88eaf8 server_cookie=69028f542a171e3d in_seq=0 out_seq=0 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f97faffd700 10 client.25444 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f97faffd700 1 -- 172.21.15.26:0/2454357893 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb31aab050 con 0x55cb31c11c10 2023-07-13T22:14:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f97faffd700 10 client.25444 handle_command_reply: tid=0 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 client.25444 shutdown 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 msgr2=0x55cb31bb6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f980c004320 tx=0x7f980c003db0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 shutdown_connections 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb31c11c10 0x55cb31c14020 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb31ba6ce0 0x55cb31bb6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f97faffd700 0 client.25444 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:14:40.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb31ba8480 0x55cb31bb7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.685+0000 7f9835503b80 1 --2- 172.21.15.26:0/2454357893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb31ba7630 0x55cb31bb6bc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.686+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 >> 172.21.15.26:0/2454357893 conn(0x55cb31b93da0 msgr2=0x55cb31bae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.686+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 shutdown_connections 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.686+0000 7f9835503b80 1 -- 172.21.15.26:0/2454357893 wait complete. 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.686+0000 7f9835503b80 20 client.25444 trim_cache size 0 max 16384 2023-07-13T22:14:40.681 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:14:40.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 msgr2=0x7f98080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 0x7f98080a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9814014fd0 tx=0x7f98140078c0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 msgr2=0x7f981c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f980400b680 tx=0x7f9804005740 comp rx=0 tx=0).stop 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 shutdown_connections 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98080a1150 0x7f98080a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f981c1057a0 0x7f981c0fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f981c100560 0x7f981c0ffed0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 --2- 172.21.15.26:0/2612484317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f981c102e80 0x7f981c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 >> 172.21.15.26:0/2612484317 conn(0x7f981c0fa000 msgr2=0x7f981c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 shutdown_connections 2023-07-13T22:14:40.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.687+0000 7f9821bff700 1 -- 172.21.15.26:0/2612484317 wait complete. 2023-07-13T22:14:40.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:40 smithi026 ceph-mon[95647]: pgmap v339: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:40.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:40 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/954900794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:40.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:14:40.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:40 smithi121 ceph-mon[129187]: pgmap v339: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:40.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:40 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/954900794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:40.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:40 smithi117 ceph-mon[129074]: pgmap v339: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-07-13T22:14:40.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:40 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/954900794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 -- 172.21.15.26:0/229056350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 msgr2=0x7feda8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 --2- 172.21.15.26:0/229056350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 0x7feda8105150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed94009990 tx=0x7fed94009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 -- 172.21.15.26:0/229056350 shutdown_connections 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 --2- 172.21.15.26:0/229056350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda8107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 --2- 172.21.15.26:0/229056350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 0x7feda8102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 --2- 172.21.15.26:0/229056350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 0x7feda8105150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 -- 172.21.15.26:0/229056350 >> 172.21.15.26:0/229056350 conn(0x7feda80f9ed0 msgr2=0x7feda80fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 -- 172.21.15.26:0/229056350 shutdown_connections 2023-07-13T22:14:40.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.986+0000 7fedad99d700 1 -- 172.21.15.26:0/229056350 wait complete. 2023-07-13T22:14:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.987+0000 7fedad99d700 1 Processor -- start 2023-07-13T22:14:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.987+0000 7fedad99d700 1 -- start start 2023-07-13T22:14:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 0x7feda80ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 0x7feda80fe400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80ff0e0 con 0x7feda8102d70 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80ff250 con 0x7feda8100450 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad99d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda819f5f0 con 0x7feda8105690 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad19c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad19c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52900/0 (socket says 172.21.15.26:52900) 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedad19c700 1 -- 172.21.15.26:0/3455164549 learned_addr learned my addr 172.21.15.26:0/3455164549 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.988+0000 7fedac99b700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 0x7feda80ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 -- 172.21.15.26:0/3455164549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 msgr2=0x7feda80ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 0x7feda80ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 -- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 msgr2=0x7feda80fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 0x7feda80fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 -- 172.21.15.26:0/3455164549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c009520 con 0x7feda8105690 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad19c700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feda000ef30 tx=0x7feda000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7feda5ffb700 1 -- 172.21.15.26:0/3455164549 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0018070 con 0x7feda8105690 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7feda5ffb700 1 -- 172.21.15.26:0/3455164549 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda000f040 con 0x7feda8105690 2023-07-13T22:14:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed940095f0 con 0x7feda8105690 2023-07-13T22:14:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7feda5ffb700 1 -- 172.21.15.26:0/3455164549 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda000cdb0 con 0x7feda8105690 2023-07-13T22:14:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.989+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda819fc40 con 0x7feda8105690 2023-07-13T22:14:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.994+0000 7feda5ffb700 1 -- 172.21.15.26:0/3455164549 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feda0003a00 con 0x7feda8105690 2023-07-13T22:14:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.995+0000 7fedc2597b80 1 Processor -- start 2023-07-13T22:14:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.995+0000 7fedc2597b80 1 -- start start 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e028b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e02a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556df2dcb0 con 0x55556e028720 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556df80a40 con 0x55556e029070 2023-07-13T22:14:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedc2597b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556df79ed0 con 0x55556e029e30 2023-07-13T22:14:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedac99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedac99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37970/0 (socket says 172.21.15.26:37970) 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7fedac99b700 1 -- 172.21.15.26:0/1838570257 learned_addr learned my addr 172.21.15.26:0/1838570257 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.996+0000 7feda7fff700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e028b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7feda7fff700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e02a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 -- 172.21.15.26:0/1838570257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 msgr2=0x55556e02a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e02a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 -- 172.21.15.26:0/1838570257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 msgr2=0x55556e028b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e028b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 -- 172.21.15.26:0/1838570257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556dda7de0 con 0x55556e029070 2023-07-13T22:14:40.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7feda7fff700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e02a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:14:40.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7feda5ffb700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 0x7fed980a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fedac99b700 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009f10 tx=0x7fed9c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bfc4d25929b0e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7feda5ffb700 1 -- 172.21.15.26:0/3455164549 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feda00cc430 con 0x7feda8105690 2023-07-13T22:14:40.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7feda7fff700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 0x7fed980a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fed92ffd700 1 -- 172.21.15.26:0/1838570257 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c019630 con 0x55556e029070 2023-07-13T22:14:40.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fed92ffd700 1 -- 172.21.15.26:0/1838570257 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c019c50 con 0x55556e029070 2023-07-13T22:14:40.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.997+0000 7fed92ffd700 1 -- 172.21.15.26:0/1838570257 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00ec10 con 0x55556e029070 2023-07-13T22:14:40.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 -- 172.21.15.26:0/1838570257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 msgr2=0x55556e0298a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:40.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7feda7fff700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 0x7fed980a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed9400b680 tx=0x7fed94000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009f10 tx=0x7fed9c011720 comp rx=0 tx=0).stop 2023-07-13T22:14:40.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 -- 172.21.15.26:0/1838570257 shutdown_connections 2023-07-13T22:14:40.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e028b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e02a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1838570257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e0298a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.998+0000 7fedc2597b80 1 -- 172.21.15.26:0/1838570257 >> 172.21.15.26:0/1838570257 conn(0x55556e01dff0 msgr2=0x55556e023460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.999+0000 7fedc2597b80 1 -- 172.21.15.26:0/1838570257 shutdown_connections 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.999+0000 7fedc2597b80 1 -- 172.21.15.26:0/1838570257 wait complete. 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.999+0000 7fedc2597b80 1 -- start start 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:40.999+0000 7fedc2597b80 1 Processor -- start 2023-07-13T22:14:40.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e032320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e0398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556df19ac0 con 0x55556e028720 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556df197c0 con 0x55556e029070 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556ddfd920 con 0x55556e029e30 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7fedc2597b80 10 client.0 _ll_register_callbacks cb 0x55556df17760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7feda7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7feda7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37978/0 (socket says 172.21.15.26:37978) 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.000+0000 7feda7fff700 1 -- 172.21.15.26:0/1948797178 learned_addr learned my addr 172.21.15.26:0/1948797178 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e0398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 -- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 msgr2=0x55556e032860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 -- 172.21.15.26:0/1948797178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 msgr2=0x55556e032320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e032320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 -- 172.21.15.26:0/1948797178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c009b70 con 0x55556e029e30 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7feda7fff700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.001+0000 7fedad19c700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e0398a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feda00c9f90 tx=0x7feda00994f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752a6a049284b33a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fed87fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0084050 con 0x55556e029e30 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feda009d910 con 0x55556e029e30 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55556ded34f0 con 0x55556e029e30 2023-07-13T22:14:40.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feda00cc2b0 con 0x55556e029e30 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fedc2597b80 10 client.25453 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55556dd90c60 con 0x55556e029e30 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.002+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55556ded34f0 con 0x55556e029e30 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feda0098dd0 con 0x55556e029e30 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 10 client.25453 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 10 client.25453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:14:40.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 20 client.25453 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:40.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e04f070 0x55556e051480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:40.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 4 client.25453 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:40.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55556df72eb0 con 0x55556e04f070 2023-07-13T22:14:41.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.003+0000 7fedac99b700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e04f070 0x55556e051480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.004+0000 7fed87fff700 10 client.25453 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:14:41.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.004+0000 7fedac99b700 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e04f070 0x55556e051480 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66ef647d41a0b4f6 server_cookie=8da328536ecf42d in_seq=0 out_seq=0 2023-07-13T22:14:41.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fed87fff700 1 -- 172.21.15.26:0/1948797178 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55556df72eb0 con 0x55556e04f070 2023-07-13T22:14:41.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fed87fff700 10 client.25453 handle_command_reply: tid=0 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 client.25453 shutdown 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 msgr2=0x55556e0398a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e0398a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feda00c9f90 tx=0x7feda00994f0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 shutdown_connections 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e028720 0x55556e032320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e04f070 0x55556e051480 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029e30 0x55556e0398a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 --2- 172.21.15.26:0/1948797178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029070 0x55556e032860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 >> 172.21.15.26:0/1948797178 conn(0x55556e01dff0 msgr2=0x55556e022b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.005+0000 7fed87fff700 0 client.25453 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:14:41.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.006+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 shutdown_connections 2023-07-13T22:14:41.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.006+0000 7fedc2597b80 1 -- 172.21.15.26:0/1948797178 wait complete. 2023-07-13T22:14:41.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.006+0000 7fedc2597b80 20 client.25453 trim_cache size 0 max 16384 2023-07-13T22:14:41.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.019+0000 7fedc2597b80 1 Processor -- start 2023-07-13T22:14:41.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 -- start start 2023-07-13T22:14:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e028720 0x55556e055e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 0x55556e039860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 0x55556e031010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e031ef0 con 0x55556e029e30 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e02ffd0 con 0x55556e028720 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedc2597b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e029690 con 0x55556e029070 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedad19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 0x55556e031010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedad19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 0x55556e031010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49970/0 (socket says 172.21.15.26:49970) 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedad19c700 1 -- 172.21.15.26:0/2237312014 learned_addr learned my addr 172.21.15.26:0/2237312014 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7feda7fff700 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 0x55556e039860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.020+0000 7fedac99b700 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e028720 0x55556e055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 -- 172.21.15.26:0/2237312014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e028720 msgr2=0x55556e055e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e028720 0x55556e055e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 -- 172.21.15.26:0/2237312014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 msgr2=0x55556e031010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 0x55556e031010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 -- 172.21.15.26:0/2237312014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556ddce250 con 0x55556e029070 2023-07-13T22:14:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7feda7fff700 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 0x55556e039860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed94005760 tx=0x7fed94028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61c07ebfc2ad635 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.021+0000 7fed937fe700 1 -- 172.21.15.26:0/2237312014 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9402d040 con 0x55556e029070 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fed937fe700 1 -- 172.21.15.26:0/2237312014 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9401e040 con 0x55556e029070 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fed937fe700 1 -- 172.21.15.26:0/2237312014 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94024c60 con 0x55556e029070 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 -- 172.21.15.26:0/2237312014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 msgr2=0x55556e039860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 0x55556e039860 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed94005760 tx=0x7fed94028040 comp rx=0 tx=0).stop 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 -- 172.21.15.26:0/2237312014 shutdown_connections 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029e30 0x55556e031010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e028720 0x55556e055e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2237312014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e029070 0x55556e039860 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.022+0000 7fedc2597b80 1 -- 172.21.15.26:0/2237312014 >> 172.21.15.26:0/2237312014 conn(0x55556e01dff0 msgr2=0x55556e023430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.023+0000 7fedc2597b80 1 -- 172.21.15.26:0/2237312014 shutdown_connections 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.023+0000 7fedc2597b80 1 -- 172.21.15.26:0/2237312014 wait complete. 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.023+0000 7fedc2597b80 1 -- start start 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.023+0000 7fedc2597b80 1 Processor -- start 2023-07-13T22:14:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 0x55556e035330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 0x55556e035870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e038dd0 con 0x55556e029070 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e038f40 con 0x55556e029e30 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556e039100 con 0x55556e028720 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedc2597b80 10 client.0 _ll_register_callbacks cb 0x7feda0095d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedad19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedac99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 0x55556e035330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedad19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:37990/0 (socket says 172.21.15.26:37990) 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.024+0000 7fedad19c700 1 -- 172.21.15.26:0/2361380613 learned_addr learned my addr 172.21.15.26:0/2361380613 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7feda7fff700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 0x55556e035870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 -- 172.21.15.26:0/2361380613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 msgr2=0x55556e035330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 0x55556e035330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 -- 172.21.15.26:0/2361380613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 msgr2=0x55556e035870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 0x55556e035870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 -- 172.21.15.26:0/2361380613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda81086e0 con 0x55556e029e30 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7feda7fff700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 0x55556e035870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedac99b700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 0x55556e035330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedad19c700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feda0099870 tx=0x7feda00a1860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ac07b5f9acbfc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fed87fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0084040 con 0x55556e029e30 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feda009d910 con 0x55556e029e30 2023-07-13T22:14:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.025+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed9c019c50 con 0x55556e029e30 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feda00cc2c0 con 0x55556e029e30 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 10 client.25484 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55556dda62b0 con 0x55556e029e30 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed9c019c50 con 0x55556e029e30 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feda00c9030 con 0x55556e029e30 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 10 client.25484 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 10 client.25484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:14:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 20 client.25484 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e0ac060 0x55556e0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 4 client.25484 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.026+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55556df35050 con 0x55556e0ac060 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.027+0000 7fedac99b700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e0ac060 0x55556e0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.027+0000 7fedac99b700 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e0ac060 0x55556e0ae470 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d684cd17f78ed2 server_cookie=8f055ef19725648e in_seq=0 out_seq=0 2023-07-13T22:14:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.027+0000 7fed87fff700 10 client.25484 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fed87fff700 1 -- 172.21.15.26:0/2361380613 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55556df35050 con 0x55556e0ac060 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fed87fff700 10 client.25484 handle_command_reply: tid=0 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 client.25484 shutdown 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 msgr2=0x55556e038890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feda0099870 tx=0x7feda00a1860 comp rx=0 tx=0).stop 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 shutdown_connections 2023-07-13T22:14:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55556e0ac060 0x55556e0ae470 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556e029e30 0x55556e038890 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55556e028720 0x55556e035330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fed87fff700 0 client.25484 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 --2- 172.21.15.26:0/2361380613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55556e029070 0x55556e035870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 >> 172.21.15.26:0/2361380613 conn(0x55556e01dff0 msgr2=0x55556e030220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 shutdown_connections 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 1 -- 172.21.15.26:0/2361380613 wait complete. 2023-07-13T22:14:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.028+0000 7fedc2597b80 20 client.25484 trim_cache size 0 max 16384 2023-07-13T22:14:41.026 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:14:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 msgr2=0x7fed980a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 0x7fed980a3720 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed9400b680 tx=0x7fed94000bc0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 msgr2=0x7feda80fe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feda000ef30 tx=0x7feda000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 shutdown_connections 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feda8105690 0x7feda80fe940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feda8100450 0x7feda80ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fed980a1270 0x7fed980a3720 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 --2- 172.21.15.26:0/3455164549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda8102d70 0x7feda80fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 >> 172.21.15.26:0/3455164549 conn(0x7feda80f9ed0 msgr2=0x7feda81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 shutdown_connections 2023-07-13T22:14:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.029+0000 7fedad99d700 1 -- 172.21.15.26:0/3455164549 wait complete. 2023-07-13T22:14:41.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:14:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 -- 172.21.15.26:0/4160329655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d0100560 msgr2=0x7f41d0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 --2- 172.21.15.26:0/4160329655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d0100560 0x7f41d0102940 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009990 tx=0x7f41c0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 -- 172.21.15.26:0/4160329655 shutdown_connections 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 --2- 172.21.15.26:0/4160329655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d01057a0 0x7f41d0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 --2- 172.21.15.26:0/4160329655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d0100560 0x7f41d0102940 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 --2- 172.21.15.26:0/4160329655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 0x7f41d0105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 -- 172.21.15.26:0/4160329655 >> 172.21.15.26:0/4160329655 conn(0x7f41d00fa000 msgr2=0x7f41d00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 -- 172.21.15.26:0/4160329655 shutdown_connections 2023-07-13T22:14:41.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.335+0000 7f41d49ff700 1 -- 172.21.15.26:0/4160329655 wait complete. 2023-07-13T22:14:41.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.336+0000 7f41d49ff700 1 Processor -- start 2023-07-13T22:14:41.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 -- start start 2023-07-13T22:14:41.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 0x7f41d0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d01057a0 0x7f41d019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d460 con 0x7f41d0102e80 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d5a0 con 0x7f41d01057a0 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.337+0000 7f41d49ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d019d730 con 0x7f41d0100560 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52952/0 (socket says 172.21.15.26:52952) 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 -- 172.21.15.26:0/3968062753 learned_addr learned my addr 172.21.15.26:0/3968062753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 0x7f41d0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ced9d700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d01057a0 0x7f41d019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 -- 172.21.15.26:0/3968062753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d01057a0 msgr2=0x7f41d019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d01057a0 0x7f41d019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 -- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 msgr2=0x7f41d0195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 0x7f41d0195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 -- 172.21.15.26:0/3968062753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c00095f0 con 0x7f41d0100560 2023-07-13T22:14:41.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.338+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41c0004730 tx=0x7f41c0004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.339+0000 7f41bf7fe700 1 -- 172.21.15.26:0/3968062753 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c001d070 con 0x7f41d0100560 2023-07-13T22:14:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.339+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d019d9b0 con 0x7f41d0100560 2023-07-13T22:14:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.339+0000 7f41bf7fe700 1 -- 172.21.15.26:0/3968062753 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c0004c90 con 0x7f41d0100560 2023-07-13T22:14:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.339+0000 7f41bf7fe700 1 -- 172.21.15.26:0/3968062753 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00054b0 con 0x7f41d0100560 2023-07-13T22:14:41.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.339+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d019de50 con 0x7f41d0100560 2023-07-13T22:14:41.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.344+0000 7f41bf7fe700 1 -- 172.21.15.26:0/3968062753 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f41c000fbc0 con 0x7f41d0100560 2023-07-13T22:14:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.345+0000 7f41e8243b80 1 Processor -- start 2023-07-13T22:14:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.345+0000 7f41e8243b80 1 -- start start 2023-07-13T22:14:41.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca0d7cb0 con 0x5612ca1d24d0 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca12aa40 con 0x5612ca1d2e20 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41e8243b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca123ed0 con 0x5612ca1dc480 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38008/0 (socket says 172.21.15.26:38008) 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 -- 172.21.15.26:0/1592409456 learned_addr learned my addr 172.21.15.26:0/1592409456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 -- 172.21.15.26:0/1592409456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 msgr2=0x5612ca1dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1dc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 -- 172.21.15.26:0/1592409456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 msgr2=0x5612ca1d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.346+0000 7f41ce59c700 1 -- 172.21.15.26:0/1592409456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c9f51de0 con 0x5612ca1d2e20 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:41.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41bf7fe700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 0x7f41b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41c00d1b10 tx=0x7f41c00d1e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0aa57a03c17fd3f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41bf7fe700 1 -- 172.21.15.26:0/3968062753 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41c00cdbb0 con 0x7f41d0100560 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41bcff9700 1 -- 172.21.15.26:0/1592409456 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0097340 con 0x5612ca1d2e20 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41ced9d700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 0x7f41b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41bcff9700 1 -- 172.21.15.26:0/1592409456 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c00d1750 con 0x5612ca1d2e20 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.347+0000 7f41bcff9700 1 -- 172.21.15.26:0/1592409456 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0004e00 con 0x5612ca1d2e20 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 -- 172.21.15.26:0/1592409456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 msgr2=0x5612ca1dbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41ced9d700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 0x7f41b80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f41c800f520 tx=0x7f41c8009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f41c00d1b10 tx=0x7f41c00d1e20 comp rx=0 tx=0).stop 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 -- 172.21.15.26:0/1592409456 shutdown_connections 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1d28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1dc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1592409456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1dbef0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.348+0000 7f41e8243b80 1 -- 172.21.15.26:0/1592409456 >> 172.21.15.26:0/1592409456 conn(0x5612ca1c7da0 msgr2=0x5612ca1cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.349+0000 7f41e8243b80 1 -- 172.21.15.26:0/1592409456 shutdown_connections 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.349+0000 7f41e8243b80 1 -- 172.21.15.26:0/1592409456 wait complete. 2023-07-13T22:14:41.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.349+0000 7f41e8243b80 1 -- start start 2023-07-13T22:14:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.349+0000 7f41e8243b80 1 Processor -- start 2023-07-13T22:14:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca0c3ac0 con 0x5612ca1d24d0 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca0c37c0 con 0x5612ca1d2e20 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612c9fa7920 con 0x5612ca1dc480 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41e8243b80 10 client.0 _ll_register_callbacks cb 0x5612ca0c1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41cdd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41cdd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38024/0 (socket says 172.21.15.26:38024) 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41cdd9b700 1 -- 172.21.15.26:0/3245372572 learned_addr learned my addr 172.21.15.26:0/3245372572 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.350+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ced9d700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 -- 172.21.15.26:0/3245372572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 msgr2=0x5612ca1ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 -- 172.21.15.26:0/3245372572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 msgr2=0x5612ca1e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 -- 172.21.15.26:0/3245372572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d019de50 con 0x5612ca1d24d0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41ce59c700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1e4600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f41c009acb0 tx=0x7f41c0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9845e1cbc6f8b67 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.351+0000 7f41b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00d10e0 con 0x5612ca1d24d0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41c0004e00 con 0x5612ca1d24d0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612ca11ceb0 con 0x5612ca1d24d0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c00d1750 con 0x5612ca1d24d0 2023-07-13T22:14:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41e8243b80 10 client.15435 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612c9f3ac60 con 0x5612ca1d24d0 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612ca11ceb0 con 0x5612ca1d24d0 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.352+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41c00b9240 con 0x5612ca1d24d0 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 10 client.15435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 20 client.15435 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca1efe40 0x5612ca1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 4 client.15435 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612ca0df050 con 0x5612ca1efe40 2023-07-13T22:14:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.353+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca1efe40 0x5612ca1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.354+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca1efe40 0x5612ca1f2220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=741443beec881186 server_cookie=541301e88779e38b in_seq=0 out_seq=0 2023-07-13T22:14:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.354+0000 7f41b5ffb700 10 client.15435 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:14:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.354+0000 7f41b5ffb700 1 -- 172.21.15.26:0/3245372572 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612ca0df050 con 0x5612ca1efe40 2023-07-13T22:14:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.354+0000 7f41b5ffb700 10 client.15435 handle_command_reply: tid=0 2023-07-13T22:14:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 client.15435 shutdown 2023-07-13T22:14:41.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 msgr2=0x5612ca1e4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1e4600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f41c009acb0 tx=0x7f41c0005ef0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 shutdown_connections 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca1efe40 0x5612ca1f2220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1e4600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41b5ffb700 0 client.15435 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1dc480 0x5612ca1ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 --2- 172.21.15.26:0/3245372572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 >> 172.21.15.26:0/3245372572 conn(0x5612ca1c7da0 msgr2=0x5612ca1cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 shutdown_connections 2023-07-13T22:14:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.355+0000 7f41e8243b80 1 -- 172.21.15.26:0/3245372572 wait complete. 2023-07-13T22:14:41.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.356+0000 7f41e8243b80 20 client.15435 trim_cache size 0 max 16384 2023-07-13T22:14:41.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.371+0000 7f41e8243b80 1 Processor -- start 2023-07-13T22:14:41.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.371+0000 7f41e8243b80 1 -- start start 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d2e20 0x5612ca1ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1dc480 0x5612ca1df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1e57f0 con 0x5612ca1d24d0 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1c79c0 con 0x5612ca1dc480 2023-07-13T22:14:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41e8243b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1e3b00 con 0x5612ca1d2e20 2023-07-13T22:14:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50022/0 (socket says 172.21.15.26:50022) 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41ce59c700 1 -- 172.21.15.26:0/1282745947 learned_addr learned my addr 172.21.15.26:0/1282745947 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.372+0000 7f41ced9d700 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1dc480 0x5612ca1df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 -- 172.21.15.26:0/1282745947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d2e20 msgr2=0x5612ca1ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d2e20 0x5612ca1ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 -- 172.21.15.26:0/1282745947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1dc480 msgr2=0x5612ca1df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1dc480 0x5612ca1df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 -- 172.21.15.26:0/1282745947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612c9f78250 con 0x5612ca1d24d0 2023-07-13T22:14:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41ce59c700 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f41c00095c0 tx=0x7f41c00995b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9795600241516f41 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.373+0000 7f41bd7fa700 1 -- 172.21.15.26:0/1282745947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c009ab40 con 0x5612ca1d24d0 2023-07-13T22:14:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.374+0000 7f41bd7fa700 1 -- 172.21.15.26:0/1282745947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41c0005780 con 0x5612ca1d24d0 2023-07-13T22:14:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.374+0000 7f41bd7fa700 1 -- 172.21.15.26:0/1282745947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c009f030 con 0x5612ca1d24d0 2023-07-13T22:14:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- 172.21.15.26:0/1282745947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 msgr2=0x5612ca1df320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f41c00095c0 tx=0x7f41c00995b0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- 172.21.15.26:0/1282745947 shutdown_connections 2023-07-13T22:14:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1d24d0 0x5612ca1df320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1dc480 0x5612ca1df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 --2- 172.21.15.26:0/1282745947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d2e20 0x5612ca1ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- 172.21.15.26:0/1282745947 >> 172.21.15.26:0/1282745947 conn(0x5612ca1c7da0 msgr2=0x5612ca1e27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- 172.21.15.26:0/1282745947 shutdown_connections 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- 172.21.15.26:0/1282745947 wait complete. 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.375+0000 7f41e8243b80 1 -- start start 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 Processor -- start 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1eb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 0x5612ca1e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1e84a0 con 0x5612ca1dc480 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1e8610 con 0x5612ca1d2e20 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ca1e87d0 con 0x5612ca1d24d0 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41e8243b80 10 client.0 _ll_register_callbacks cb 0x5612ca0c1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.376+0000 7f41ce59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52984/0 (socket says 172.21.15.26:52984) 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 -- 172.21.15.26:0/2766110900 learned_addr learned my addr 172.21.15.26:0/2766110900 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1eb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ced9d700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 0x5612ca1e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 -- 172.21.15.26:0/2766110900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 msgr2=0x5612ca1eb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1eb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 -- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 msgr2=0x5612ca1e7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 0x5612ca1e7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 -- 172.21.15.26:0/2766110900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612c9f3ac60 con 0x5612ca1d24d0 2023-07-13T22:14:41.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ced9d700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 0x5612ca1e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:14:41.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41ce59c700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c0005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29ccebc7f1e42dfd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:41.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.377+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0005780 con 0x5612ca1d24d0 2023-07-13T22:14:41.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f41c009f030 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612c9f72f70 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41c009ab40 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 10 client.25474 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612c9f502b0 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612c9f72f70 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f41c009a4f0 con 0x5612ca1d24d0 2023-07-13T22:14:41.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 10 client.25474 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 20 client.25474 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca244c20 0x5612ca247030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 4 client.25474 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.378+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612c9fa7f60 con 0x5612ca244c20 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.379+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca244c20 0x5612ca247030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.379+0000 7f41cdd9b700 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca244c20 0x5612ca247030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9bbb5897e2b4613 server_cookie=178e93eac4e85cbf in_seq=0 out_seq=0 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.379+0000 7f41b5ffb700 10 client.25474 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41b5ffb700 1 -- 172.21.15.26:0/2766110900 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612c9fa7f60 con 0x5612ca244c20 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41b5ffb700 10 client.25474 handle_command_reply: tid=0 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 client.25474 shutdown 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 msgr2=0x5612ca1eb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c0005a90 comp rx=0 tx=0).stop 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 shutdown_connections 2023-07-13T22:14:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5612ca244c20 0x5612ca247030 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612ca1dc480 0x5612ca1e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612ca1d24d0 0x5612ca1eb3e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 --2- 172.21.15.26:0/2766110900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612ca1d2e20 0x5612ca1eb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 >> 172.21.15.26:0/2766110900 conn(0x5612ca1c7da0 msgr2=0x5612ca1e27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41b5ffb700 0 client.25474 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 shutdown_connections 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.380+0000 7f41e8243b80 1 -- 172.21.15.26:0/2766110900 wait complete. 2023-07-13T22:14:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41e8243b80 20 client.25474 trim_cache size 0 max 16384 2023-07-13T22:14:41.377 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:14:41.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 msgr2=0x7f41b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 0x7f41b80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f41c800f520 tx=0x7f41c8009920 comp rx=0 tx=0).stop 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 msgr2=0x7f41d01958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f41c0004730 tx=0x7f41c0004760 comp rx=0 tx=0).stop 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 shutdown_connections 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d01057a0 0x7f41d019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41d0100560 0x7f41d01958e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f41b80a1150 0x7f41b80a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 --2- 172.21.15.26:0/3968062753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41d0102e80 0x7f41d0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.381+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 >> 172.21.15.26:0/3968062753 conn(0x7f41d00fa000 msgr2=0x7f41d01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.382+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 shutdown_connections 2023-07-13T22:14:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.382+0000 7f41d49ff700 1 -- 172.21.15.26:0/3968062753 wait complete. 2023-07-13T22:14:41.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:14:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3811457155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80fe2a0 msgr2=0x7fe2c80fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3811457155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c80fe6b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009990 tx=0x7fe2b8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3811457155 shutdown_connections 2023-07-13T22:14:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3811457155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c80fe6b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3811457155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80ff5a0 0x7fe2c80ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3811457155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c80ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3811457155 >> 172.21.15.26:0/3811457155 conn(0x7fe2c80f9ed0 msgr2=0x7fe2c80fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3811457155 shutdown_connections 2023-07-13T22:14:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.657+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3811457155 wait complete. 2023-07-13T22:14:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.658+0000 7fe2cd39d700 1 Processor -- start 2023-07-13T22:14:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 -- start start 2023-07-13T22:14:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c81977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80ff5a0 0x7fe2c819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819f330 con 0x7fe2c80ff5a0 2023-07-13T22:14:41.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819f470 con 0x7fe2c80fe2a0 2023-07-13T22:14:41.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2cd39d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c819f600 con 0x7fe2c80febf0 2023-07-13T22:14:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.659+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52992/0 (socket says 172.21.15.26:52992) 2023-07-13T22:14:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 -- 172.21.15.26:0/3465863402 learned_addr learned my addr 172.21.15.26:0/3465863402 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80ff5a0 0x7fe2c819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c81977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 -- 172.21.15.26:0/3465863402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80fe2a0 msgr2=0x7fe2c81977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c81977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 -- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80ff5a0 msgr2=0x7fe2c819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80ff5a0 0x7fe2c819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 -- 172.21.15.26:0/3465863402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80095f0 con 0x7fe2c80febf0 2023-07-13T22:14:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00e9a0 tx=0x7fe2bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.660+0000 7fe2c57fa700 1 -- 172.21.15.26:0/3465863402 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc009790 con 0x7fe2c80febf0 2023-07-13T22:14:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.661+0000 7fe2c57fa700 1 -- 172.21.15.26:0/3465863402 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2bc010430 con 0x7fe2c80febf0 2023-07-13T22:14:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.661+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c819f8e0 con 0x7fe2c80febf0 2023-07-13T22:14:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.661+0000 7fe2c57fa700 1 -- 172.21.15.26:0/3465863402 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc018680 con 0x7fe2c80febf0 2023-07-13T22:14:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.661+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c819fdf0 con 0x7fe2c80febf0 2023-07-13T22:14:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.666+0000 7fe2c57fa700 1 -- 172.21.15.26:0/3465863402 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe2bc0105a0 con 0x7fe2c80febf0 2023-07-13T22:14:41.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 Processor -- start 2023-07-13T22:14:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 -- start start 2023-07-13T22:14:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce674b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce579cb0 con 0x5607ce675e30 2023-07-13T22:14:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce5cca40 con 0x5607ce674720 2023-07-13T22:14:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.667+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce5c5ed0 con 0x5607ce675070 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53008/0 (socket says 172.21.15.26:53008) 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2265248894 learned_addr learned my addr 172.21.15.26:0/2265248894 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c57fa700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 0x7fe2b00a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce674b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2265248894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 msgr2=0x5607ce674b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce674b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2265248894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 msgr2=0x5607ce6762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2265248894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ce3f3de0 con 0x5607ce675070 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.668+0000 7fe2c57fa700 1 -- 172.21.15.26:0/3465863402 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2bc014070 con 0x7fe2c80febf0 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.669+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 0x7fe2b00a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.669+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8015000 tx=0x7fe2b8005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2007f32353f24d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.669+0000 7fe2ae7fc700 1 -- 172.21.15.26:0/2265248894 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b801d070 con 0x5607ce675070 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.669+0000 7fe2ae7fc700 1 -- 172.21.15.26:0/2265248894 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b801e040 con 0x5607ce675070 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.669+0000 7fe2ae7fc700 1 -- 172.21.15.26:0/2265248894 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004960 con 0x5607ce675070 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2265248894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 msgr2=0x5607ce6758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8015000 tx=0x7fe2b8005ea0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 0x7fe2b00a36a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009ec0 tx=0x7fe2c0004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2265248894 shutdown_connections 2023-07-13T22:14:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce674b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2265248894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce6758a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2265248894 >> 172.21.15.26:0/2265248894 conn(0x5607ce669ff0 msgr2=0x5607ce66f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2265248894 shutdown_connections 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.670+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2265248894 wait complete. 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.671+0000 7fe2e2010b80 1 -- start start 2023-07-13T22:14:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.671+0000 7fe2e2010b80 1 Processor -- start 2023-07-13T22:14:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce67e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce67e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 0x5607ce6858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce565ac0 con 0x5607ce674720 2023-07-13T22:14:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce5657c0 con 0x5607ce675e30 2023-07-13T22:14:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce449920 con 0x5607ce675070 2023-07-13T22:14:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2e2010b80 10 client.0 _ll_register_callbacks cb 0x5607ce563760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce67e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce67e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce67e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50064/0 (socket says 172.21.15.26:50064) 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2391385003 learned_addr learned my addr 172.21.15.26:0/2391385003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.672+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 0x5607ce6858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 -- 172.21.15.26:0/2391385003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 msgr2=0x5607ce67e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce67e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 -- 172.21.15.26:0/2391385003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 msgr2=0x5607ce67e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce67e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 -- 172.21.15.26:0/2391385003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c810a940 con 0x5607ce675e30 2023-07-13T22:14:41.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 0x5607ce6858a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010d00 tx=0x7fe2c000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4219186cf2c1df42 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.673+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000e040 con 0x5607ce675e30 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2c001b030 con 0x5607ce675e30 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607ce51f4f0 con 0x5607ce675e30 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2c0015ab0 con 0x5607ce675e30 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2e2010b80 10 client.25511 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5607ce3dcc60 con 0x5607ce675e30 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.674+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607ce51f4f0 con 0x5607ce675e30 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2c0022c80 con 0x5607ce675e30 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 20 client.25511 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce68a190 0x5607ce68c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 4 client.25511 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607ce5beeb0 con 0x5607ce68a190 2023-07-13T22:14:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.675+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce68a190 0x5607ce68c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.676+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce68a190 0x5607ce68c5a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f02b7d59add7063 server_cookie=2231d32261721433 in_seq=0 out_seq=0 2023-07-13T22:14:41.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.676+0000 7fe2a77fe700 10 client.25511 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2391385003 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5607ce5beeb0 con 0x5607ce68a190 2023-07-13T22:14:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2a77fe700 10 client.25511 handle_command_reply: tid=0 2023-07-13T22:14:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 client.25511 shutdown 2023-07-13T22:14:41.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 msgr2=0x5607ce6858a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 0x5607ce6858a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0010d00 tx=0x7fe2c000b040 comp rx=0 tx=0).stop 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 shutdown_connections 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce67e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce68a190 0x5607ce68c5a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675e30 0x5607ce6858a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2391385003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce67e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2a77fe700 0 client.25511 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:41.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.677+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 >> 172.21.15.26:0/2391385003 conn(0x5607ce669ff0 msgr2=0x5607ce66eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.678+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 shutdown_connections 2023-07-13T22:14:41.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.678+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2391385003 wait complete. 2023-07-13T22:14:41.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.678+0000 7fe2e2010b80 20 client.25511 trim_cache size 0 max 16384 2023-07-13T22:14:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.695+0000 7fe2e2010b80 1 Processor -- start 2023-07-13T22:14:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.695+0000 7fe2e2010b80 1 -- start start 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce67d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce5beee0 con 0x5607ce675e30 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce51f520 con 0x5607ce674720 2023-07-13T22:14:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce669bc0 con 0x5607ce675070 2023-07-13T22:14:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53026/0 (socket says 172.21.15.26:53026) 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2c77fe700 1 -- 172.21.15.26:0/275161983 learned_addr learned my addr 172.21.15.26:0/275161983 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.696+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce67d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 -- 172.21.15.26:0/275161983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 msgr2=0x5607ce67d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce67d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 -- 172.21.15.26:0/275161983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 msgr2=0x5607ce6860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 -- 172.21.15.26:0/275161983 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607ce41a250 con 0x5607ce675070 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce67d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0ca5d0 tx=0x7fe2bc005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8140a1036f1563fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2aeffd700 1 -- 172.21.15.26:0/275161983 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc09df10 con 0x5607ce675070 2023-07-13T22:14:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2aeffd700 1 -- 172.21.15.26:0/275161983 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2bc0a2330 con 0x5607ce675070 2023-07-13T22:14:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.697+0000 7fe2aeffd700 1 -- 172.21.15.26:0/275161983 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0a1ea0 con 0x5607ce675070 2023-07-13T22:14:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 -- 172.21.15.26:0/275161983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 msgr2=0x5607ce685860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0ca5d0 tx=0x7fe2bc005760 comp rx=0 tx=0).stop 2023-07-13T22:14:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 -- 172.21.15.26:0/275161983 shutdown_connections 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce675e30 0x5607ce6860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce674720 0x5607ce67d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/275161983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675070 0x5607ce685860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 -- 172.21.15.26:0/275161983 >> 172.21.15.26:0/275161983 conn(0x5607ce669ff0 msgr2=0x5607ce690f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.698+0000 7fe2e2010b80 1 -- 172.21.15.26:0/275161983 shutdown_connections 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.699+0000 7fe2e2010b80 1 -- 172.21.15.26:0/275161983 wait complete. 2023-07-13T22:14:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.699+0000 7fe2e2010b80 1 -- start start 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.699+0000 7fe2e2010b80 1 Processor -- start 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce681a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 0x5607ce681fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675e30 0x5607ce684e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce685340 con 0x5607ce674720 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce6854b0 con 0x5607ce675070 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607ce685670 con 0x5607ce675e30 2023-07-13T22:14:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2e2010b80 10 client.0 _ll_register_callbacks cb 0x7fe2c0009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce681a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce681a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50090/0 (socket says 172.21.15.26:50090) 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2c7fff700 1 -- 172.21.15.26:0/2715463090 learned_addr learned my addr 172.21.15.26:0/2715463090 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.700+0000 7fe2ccb9c700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675e30 0x5607ce684e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 0x5607ce681fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 -- 172.21.15.26:0/2715463090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675e30 msgr2=0x5607ce684e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675e30 0x5607ce684e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 -- 172.21.15.26:0/2715463090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 msgr2=0x5607ce681a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce681a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 -- 172.21.15.26:0/2715463090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c819fdf0 con 0x5607ce675070 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2c77fe700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 0x5607ce681fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc099e70 tx=0x7fe2bc0adf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc11653a83d7a4a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.701+0000 7fe2a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:14:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc09df10 con 0x5607ce675070 2023-07-13T22:14:41.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2bc0a1ea0 con 0x5607ce675070 2023-07-13T22:14:41.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b801e040 con 0x5607ce675070 2023-07-13T22:14:41.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2bc0a2060 con 0x5607ce675070 2023-07-13T22:14:41.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 10 client.25517 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:41.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5607ce3f22b0 con 0x5607ce675070 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2b801e040 con 0x5607ce675070 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2bc095ed0 con 0x5607ce675070 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 10 client.25517 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 20 client.25517 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce6def10 0x5607ce6e1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 4 client.25517 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:14:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.702+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607ce581050 con 0x5607ce6def10 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.703+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce6def10 0x5607ce6e1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.703+0000 7fe2a77fe700 10 client.25517 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.703+0000 7fe2c7fff700 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce6def10 0x5607ce6e1320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f99f4c7eac48279 server_cookie=ddf2ec677734c092 in_seq=0 out_seq=0 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.703+0000 7fe2a77fe700 1 -- 172.21.15.26:0/2715463090 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607ce581050 con 0x5607ce6def10 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.703+0000 7fe2a77fe700 10 client.25517 handle_command_reply: tid=0 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 client.25517 shutdown 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 msgr2=0x5607ce681fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 0x5607ce681fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc099e70 tx=0x7fe2bc0adf90 comp rx=0 tx=0).stop 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 shutdown_connections 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607ce674720 0x5607ce681a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5607ce6def10 0x5607ce6e1320 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5607ce675e30 0x5607ce684e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 --2- 172.21.15.26:0/2715463090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607ce675070 0x5607ce681fd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2a77fe700 0 client.25517 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 >> 172.21.15.26:0/2715463090 conn(0x5607ce669ff0 msgr2=0x5607ce67c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 shutdown_connections 2023-07-13T22:14:41.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 1 -- 172.21.15.26:0/2715463090 wait complete. 2023-07-13T22:14:41.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.704+0000 7fe2e2010b80 20 client.25517 trim_cache size 0 max 16384 2023-07-13T22:14:41.701 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:14:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 msgr2=0x7fe2b00a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 0x7fe2b00a36a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009ec0 tx=0x7fe2c0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 msgr2=0x7fe2c8197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00e9a0 tx=0x7fe2bc00ecb0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 shutdown_connections 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2b00a11f0 0x7fe2b00a36a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2c80ff5a0 0x7fe2c819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2c80fe2a0 0x7fe2c81977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 --2- 172.21.15.26:0/3465863402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe2c80febf0 0x7fe2c8197cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 >> 172.21.15.26:0/3465863402 conn(0x7fe2c80f9ed0 msgr2=0x7fe2c80fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.705+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 shutdown_connections 2023-07-13T22:14:41.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:41.706+0000 7fe2cd39d700 1 -- 172.21.15.26:0/3465863402 wait complete. 2023-07-13T22:14:42.721 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:14:42.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:14:42.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:42 smithi121 ceph-mon[129187]: pgmap v340: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:42.792 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:42 smithi026 ceph-mon[95647]: pgmap v340: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:42.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:42 smithi117 ceph-mon[129074]: pgmap v340: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:14:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.974+0000 7f25a17ff700 1 -- 172.21.15.26:0/1865315663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c0ffae0 msgr2=0x7f259c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:42.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.974+0000 7f25a17ff700 1 --2- 172.21.15.26:0/1865315663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c0ffae0 0x7f259c0ffef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f258c0098f0 tx=0x7f258c009c00 comp rx=0 tx=0).stop 2023-07-13T22:14:42.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 -- 172.21.15.26:0/1865315663 shutdown_connections 2023-07-13T22:14:42.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 --2- 172.21.15.26:0/1865315663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c101660 0x7f259c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 --2- 172.21.15.26:0/1865315663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c0ffae0 0x7f259c0ffef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 --2- 172.21.15.26:0/1865315663 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c100cd0 0x7f259c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 -- 172.21.15.26:0/1865315663 >> 172.21.15.26:0/1865315663 conn(0x7f259c0fb0e0 msgr2=0x7f259c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:42.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 -- 172.21.15.26:0/1865315663 shutdown_connections 2023-07-13T22:14:42.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.975+0000 7f25a17ff700 1 -- 172.21.15.26:0/1865315663 wait complete. 2023-07-13T22:14:42.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.977+0000 7f25a17ff700 1 Processor -- start 2023-07-13T22:14:42.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.977+0000 7f25a17ff700 1 -- start start 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 0x7f259c19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c100cd0 0x7f259c19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 0x7f259c1a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a39c0 con 0x7f259c0ffae0 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a3b00 con 0x7f259c100cd0 2023-07-13T22:14:42.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f25a17ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1a3c90 con 0x7f259c101660 2023-07-13T22:14:42.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c100cd0 0x7f259c19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f259affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 0x7f259c19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f259affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 0x7f259c19be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44838/0 (socket says 172.21.15.26:44838) 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f259affd700 1 -- 172.21.15.26:0/2726748149 learned_addr learned my addr 172.21.15.26:0/2726748149 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.980+0000 7f259b7fe700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 0x7f259c1a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 -- 172.21.15.26:0/2726748149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c100cd0 msgr2=0x7f259c19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:42.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c100cd0 0x7f259c19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 -- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 msgr2=0x7f259c19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 0x7f259c19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 -- 172.21.15.26:0/2726748149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900095f0 con 0x7f259c101660 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f259b7fe700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 0x7f259c1a33f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f259400efc0 tx=0x7f259400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f2583fff700 1 -- 172.21.15.26:0/2726748149 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594018070 con 0x7f259c101660 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.981+0000 7f2583fff700 1 -- 172.21.15.26:0/2726748149 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f259400f040 con 0x7f259c101660 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.982+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c009520 con 0x7f259c101660 2023-07-13T22:14:42.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.982+0000 7f2583fff700 1 -- 172.21.15.26:0/2726748149 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400cd40 con 0x7f259c101660 2023-07-13T22:14:42.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.982+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c1a41e0 con 0x7f259c101660 2023-07-13T22:14:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.988+0000 7f25b5213b80 1 Processor -- start 2023-07-13T22:14:42.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f2583fff700 1 -- 172.21.15.26:0/2726748149 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2594003a00 con 0x7f259c101660 2023-07-13T22:14:42.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 -- start start 2023-07-13T22:14:42.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba51f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba520160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba423d30 con 0x5587ba51eea0 2023-07-13T22:14:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba476ac0 con 0x5587ba51fcf0 2023-07-13T22:14:42.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.989+0000 7f25b5213b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba46ff50 con 0x5587ba51e550 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:43172/0 (socket says 172.21.15.26:43172) 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 -- 172.21.15.26:0/3716950542 learned_addr learned my addr 172.21.15.26:0/3716950542 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259affd700 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba51f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba520160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 -- 172.21.15.26:0/3716950542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 msgr2=0x5587ba520160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba520160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 -- 172.21.15.26:0/3716950542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 msgr2=0x5587ba51f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba51f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 -- 172.21.15.26:0/3716950542 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ba29de60 con 0x5587ba51e550 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.990+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2590009890 tx=0x7f2590009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d8ee839ec595e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f25817fa700 1 -- 172.21.15.26:0/3716950542 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590011110 con 0x5587ba51e550 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f25817fa700 1 -- 172.21.15.26:0/3716950542 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25900044f0 con 0x5587ba51e550 2023-07-13T22:14:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f25817fa700 1 -- 172.21.15.26:0/3716950542 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000fde0 con 0x5587ba51e550 2023-07-13T22:14:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f2583fff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 0x7f25840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f2583fff700 1 -- 172.21.15.26:0/2726748149 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25940cc4e0 con 0x7f259c101660 2023-07-13T22:14:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.991+0000 7f259affd700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 0x7f25840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 -- 172.21.15.26:0/3716950542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 msgr2=0x5587ba51e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2590009890 tx=0x7f2590009820 comp rx=0 tx=0).stop 2023-07-13T22:14:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 -- 172.21.15.26:0/3716950542 shutdown_connections 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba520160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f259affd700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 0x7f25840a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c010 tx=0x7f258c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba51e960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3716950542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba51f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 -- 172.21.15.26:0/3716950542 >> 172.21.15.26:0/3716950542 conn(0x5587ba513e20 msgr2=0x5587ba519250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 -- 172.21.15.26:0/3716950542 shutdown_connections 2023-07-13T22:14:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.992+0000 7f25b5213b80 1 -- 172.21.15.26:0/3716950542 wait complete. 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.993+0000 7f25b5213b80 1 -- start start 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.993+0000 7f25b5213b80 1 Processor -- start 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba5286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba52f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba40fb40 con 0x5587ba51eea0 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba40f840 con 0x5587ba51fcf0 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba2f39a0 con 0x5587ba51e550 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f25b5213b80 10 client.0 _ll_register_callbacks cb 0x5587ba40d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f259affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f259affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:43174/0 (socket says 172.21.15.26:43174) 2023-07-13T22:14:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.994+0000 7f259affd700 1 -- 172.21.15.26:0/3921457206 learned_addr learned my addr 172.21.15.26:0/3921457206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba5286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 -- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 msgr2=0x5587ba52f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba52f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 -- 172.21.15.26:0/3921457206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 msgr2=0x5587ba5286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba5286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 -- 172.21.15.26:0/3921457206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2590019040 con 0x5587ba51e550 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba5286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f259affd700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f258c009230 tx=0x7f258c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941fa9d38510b51e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f257eff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02b070 con 0x5587ba51e550 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587ba3c9570 con 0x5587ba51e550 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.995+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f258c0053a0 con 0x5587ba51e550 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.996+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c005560 con 0x5587ba51e550 2023-07-13T22:14:42.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.996+0000 7f25b5213b80 10 client.25504 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.996+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5587ba286ce0 con 0x5587ba51e550 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.996+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587ba3c9570 con 0x5587ba51e550 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f258c024520 con 0x5587ba51e550 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 10 client.25504 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 10 client.25504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 20 client.25504 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:42.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba533f50 0x5587ba536360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 4 client.25504 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587ba468f30 con 0x5587ba533f50 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.997+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba533f50 0x5587ba536360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.998+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba533f50 0x5587ba536360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab52336496222d6 server_cookie=33077c4e94c6d1d5 in_seq=0 out_seq=0 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.998+0000 7f257eff5700 10 client.25504 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.998+0000 7f257eff5700 1 -- 172.21.15.26:0/3921457206 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587ba468f30 con 0x5587ba533f50 2023-07-13T22:14:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.998+0000 7f257eff5700 10 client.25504 handle_command_reply: tid=0 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 client.25504 shutdown 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 msgr2=0x5587ba528180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f258c009230 tx=0x7f258c009c00 comp rx=0 tx=0).stop 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 shutdown_connections 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba533f50 0x5587ba536360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51fcf0 0x5587ba52f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba528180 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f257eff5700 0 client.25504 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3921457206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba5286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 >> 172.21.15.26:0/3921457206 conn(0x5587ba513e20 msgr2=0x5587ba518930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 shutdown_connections 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:42.999+0000 7f25b5213b80 1 -- 172.21.15.26:0/3921457206 wait complete. 2023-07-13T22:14:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.000+0000 7f25b5213b80 20 client.25504 trim_cache size 0 max 16384 2023-07-13T22:14:43.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 Processor -- start 2023-07-13T22:14:43.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 -- start start 2023-07-13T22:14:43.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba523130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51fcf0 0x5587ba52f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba527d50 con 0x5587ba51eea0 2023-07-13T22:14:43.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba529370 con 0x5587ba51e550 2023-07-13T22:14:43.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.018+0000 7f25b5213b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba513a40 con 0x5587ba51fcf0 2023-07-13T22:14:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43718/0 (socket says 172.21.15.26:43718) 2023-07-13T22:14:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 -- 172.21.15.26:0/3320870693 learned_addr learned my addr 172.21.15.26:0/3320870693 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259a7fc700 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba523130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259b7fe700 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51fcf0 0x5587ba52f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 -- 172.21.15.26:0/3320870693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51fcf0 msgr2=0x5587ba52f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51fcf0 0x5587ba52f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 -- 172.21.15.26:0/3320870693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 msgr2=0x5587ba523130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba523130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 -- 172.21.15.26:0/3320870693 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ba5271d0 con 0x5587ba51e550 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.019+0000 7f259affd700 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f258c005e90 tx=0x7f258c009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6a1c81ed46caf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.020+0000 7f2581ffb700 1 -- 172.21.15.26:0/3320870693 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02b070 con 0x5587ba51e550 2023-07-13T22:14:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.020+0000 7f2581ffb700 1 -- 172.21.15.26:0/3320870693 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f258c024c30 con 0x5587ba51e550 2023-07-13T22:14:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.020+0000 7f2581ffb700 1 -- 172.21.15.26:0/3320870693 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02f780 con 0x5587ba51e550 2023-07-13T22:14:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.020+0000 7f25b5213b80 1 -- 172.21.15.26:0/3320870693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 msgr2=0x5587ba522bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.020+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f258c005e90 tx=0x7f258c009420 comp rx=0 tx=0).stop 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 -- 172.21.15.26:0/3320870693 shutdown_connections 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51fcf0 0x5587ba52f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51e550 0x5587ba522bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 --2- 172.21.15.26:0/3320870693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51eea0 0x5587ba523130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 -- 172.21.15.26:0/3320870693 >> 172.21.15.26:0/3320870693 conn(0x5587ba513e20 msgr2=0x5587ba52fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 -- 172.21.15.26:0/3320870693 shutdown_connections 2023-07-13T22:14:43.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 -- 172.21.15.26:0/3320870693 wait complete. 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 -- start start 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.021+0000 7f25b5213b80 1 Processor -- start 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51eea0 0x5587ba52c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51fcf0 0x5587ba587e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba52cf00 con 0x5587ba51fcf0 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba52d070 con 0x5587ba51eea0 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ba588340 con 0x5587ba51e550 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f25b5213b80 10 client.0 _ll_register_callbacks cb 0x7f258c00b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f259affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f259affd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:43192/0 (socket says 172.21.15.26:43192) 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f259affd700 1 -- 172.21.15.26:0/1537689577 learned_addr learned my addr 172.21.15.26:0/1537689577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.022+0000 7f259a7fc700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51eea0 0x5587ba52c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259b7fe700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51fcf0 0x5587ba587e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 -- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51eea0 msgr2=0x5587ba52c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51eea0 0x5587ba52c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 -- 172.21.15.26:0/1537689577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51fcf0 msgr2=0x5587ba587e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51fcf0 0x5587ba587e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 -- 172.21.15.26:0/1537689577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587ba286ce0 con 0x5587ba51e550 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f259affd700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f258c0246d0 tx=0x7f258c0039b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e211d3407844cc2c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f257eff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:14:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.023+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02b040 con 0x5587ba51e550 2023-07-13T22:14:43.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587ba42b0d0 con 0x5587ba51e550 2023-07-13T22:14:43.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f258c024c30 con 0x5587ba51e550 2023-07-13T22:14:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c024420 con 0x5587ba51e550 2023-07-13T22:14:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 10 client.25510 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:43.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5587ba520390 con 0x5587ba51e550 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587ba42b0d0 con 0x5587ba51e550 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f258c017400 con 0x5587ba51e550 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 10 client.25510 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 20 client.25510 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.024+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba589800 0x5587ba58bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.025+0000 7f25b5213b80 4 client.25510 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.025+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5587ba2beff0 con 0x5587ba589800 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.025+0000 7f259a7fc700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba589800 0x5587ba58bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.025+0000 7f259a7fc700 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba589800 0x5587ba58bc10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12cd39c0741b834 server_cookie=9c412036fe51e0b8 in_seq=0 out_seq=0 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.025+0000 7f257eff5700 10 client.25510 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f257eff5700 1 -- 172.21.15.26:0/1537689577 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5587ba2beff0 con 0x5587ba589800 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f257eff5700 10 client.25510 handle_command_reply: tid=0 2023-07-13T22:14:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 client.25510 shutdown 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 msgr2=0x5587ba52c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f258c0246d0 tx=0x7f258c0039b0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 shutdown_connections 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5587ba51fcf0 0x5587ba587e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5587ba589800 0x5587ba58bc10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587ba51e550 0x5587ba52c480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 --2- 172.21.15.26:0/1537689577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5587ba51eea0 0x5587ba52c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 >> 172.21.15.26:0/1537689577 conn(0x5587ba513e20 msgr2=0x5587ba522bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f257eff5700 0 client.25510 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 shutdown_connections 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.027+0000 7f25b5213b80 1 -- 172.21.15.26:0/1537689577 wait complete. 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25b5213b80 20 client.25510 trim_cache size 0 max 16384 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:14:43.023 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:14:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "39a37ce7-0e98-4912-afd4-520e65e55696", 2023-07-13T22:14:43.024 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:14:43.024 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:14:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 msgr2=0x7f25840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 0x7f25840a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c010 tx=0x7f258c019040 comp rx=0 tx=0).stop 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 msgr2=0x7f259c1a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 0x7f259c1a33f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f259400efc0 tx=0x7f259400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 shutdown_connections 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25840a1150 0x7f25840a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0ffae0 0x7f259c19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101660 0x7f259c1a33f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 --2- 172.21.15.26:0/2726748149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f259c100cd0 0x7f259c19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.028+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 >> 172.21.15.26:0/2726748149 conn(0x7f259c0fb0e0 msgr2=0x7f259c10bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.029+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 shutdown_connections 2023-07-13T22:14:43.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.029+0000 7f25a17ff700 1 -- 172.21.15.26:0/2726748149 wait complete. 2023-07-13T22:14:43.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:14:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.316+0000 7f21047ff700 1 -- 172.21.15.26:0/572137658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc100560 msgr2=0x7f20fc102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.316+0000 7f21047ff700 1 --2- 172.21.15.26:0/572137658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc100560 0x7f20fc102940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009990 tx=0x7f20f8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 -- 172.21.15.26:0/572137658 shutdown_connections 2023-07-13T22:14:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 --2- 172.21.15.26:0/572137658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc1057a0 0x7f20fc107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 --2- 172.21.15.26:0/572137658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc100560 0x7f20fc102940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 --2- 172.21.15.26:0/572137658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc102e80 0x7f20fc105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 -- 172.21.15.26:0/572137658 >> 172.21.15.26:0/572137658 conn(0x7f20fc0fa000 msgr2=0x7f20fc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 -- 172.21.15.26:0/572137658 shutdown_connections 2023-07-13T22:14:43.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.317+0000 7f21047ff700 1 -- 172.21.15.26:0/572137658 wait complete. 2023-07-13T22:14:43.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.318+0000 7f21047ff700 1 Processor -- start 2023-07-13T22:14:43.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.318+0000 7f21047ff700 1 -- start start 2023-07-13T22:14:43.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc1057a0 0x7f20fc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19f5f0 con 0x7f20fc100560 2023-07-13T22:14:43.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19f730 con 0x7f20fc102e80 2023-07-13T22:14:43.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f21047ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19f8c0 con 0x7f20fc1057a0 2023-07-13T22:14:43.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f210259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f2101d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f2101d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43744/0 (socket says 172.21.15.26:43744) 2023-07-13T22:14:43.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f2101d9a700 1 -- 172.21.15.26:0/3414783632 learned_addr learned my addr 172.21.15.26:0/3414783632 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.319+0000 7f210259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44882/0 (socket says 172.21.15.26:44882) 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f2102d9c700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc1057a0 0x7f20fc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 -- 172.21.15.26:0/3414783632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc1057a0 msgr2=0x7f20fc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc1057a0 0x7f20fc19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 -- 172.21.15.26:0/3414783632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 msgr2=0x7f20fc197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 -- 172.21.15.26:0/3414783632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80095f0 con 0x7f20fc100560 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f2101d9a700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.320+0000 7f210259b700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20f80098e0 tx=0x7f20f8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.321+0000 7f20f37fe700 1 -- 172.21.15.26:0/3414783632 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f801d070 con 0x7f20fc100560 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.321+0000 7f20f37fe700 1 -- 172.21.15.26:0/3414783632 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f8021410 con 0x7f20fc100560 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.321+0000 7f20f37fe700 1 -- 172.21.15.26:0/3414783632 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8005720 con 0x7f20fc100560 2023-07-13T22:14:43.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.321+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc19fb40 con 0x7f20fc100560 2023-07-13T22:14:43.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.321+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc19ff10 con 0x7f20fc100560 2023-07-13T22:14:43.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.326+0000 7f20f37fe700 1 -- 172.21.15.26:0/3414783632 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f20f8021580 con 0x7f20fc100560 2023-07-13T22:14:43.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.327+0000 7f21180ccb80 1 Processor -- start 2023-07-13T22:14:43.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.327+0000 7f21180ccb80 1 -- start start 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e4d8cb0 con 0x564b0e5d3e20 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e52ba40 con 0x564b0e5d4c70 2023-07-13T22:14:43.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f21180ccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e524ed0 con 0x564b0e5d34d0 2023-07-13T22:14:43.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f2101d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f2101d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43752/0 (socket says 172.21.15.26:43752) 2023-07-13T22:14:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f2101d9a700 1 -- 172.21.15.26:0/1894635603 learned_addr learned my addr 172.21.15.26:0/1894635603 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.328+0000 7f210259b700 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f20f37fe700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 0x7f20e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 -- 172.21.15.26:0/1894635603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 msgr2=0x564b0e5d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 -- 172.21.15.26:0/1894635603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 msgr2=0x564b0e5d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 -- 172.21.15.26:0/1894635603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b0e352de0 con 0x564b0e5d4c70 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f20f37fe700 1 -- 172.21.15.26:0/3414783632 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f80cdc60 con 0x7f20fc100560 2023-07-13T22:14:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2102d9c700 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:14:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2102d9c700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 0x7f20e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.329+0000 7f2101d9a700 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0117e0 tx=0x7f20ec011af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf88b5ec529f6d85 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.330+0000 7f20f0ff9700 1 -- 172.21.15.26:0/1894635603 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00f480 con 0x564b0e5d4c70 2023-07-13T22:14:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.330+0000 7f20f0ff9700 1 -- 172.21.15.26:0/1894635603 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ec00faa0 con 0x564b0e5d4c70 2023-07-13T22:14:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.330+0000 7f20f0ff9700 1 -- 172.21.15.26:0/1894635603 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec019b60 con 0x564b0e5d4c70 2023-07-13T22:14:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 -- 172.21.15.26:0/1894635603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 msgr2=0x564b0e5d50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0117e0 tx=0x7f20ec011af0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f2102d9c700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 0x7f20e80a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009ec0 tx=0x7f20f4009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 -- 172.21.15.26:0/1894635603 shutdown_connections 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d50e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 --2- 172.21.15.26:0/1894635603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 -- 172.21.15.26:0/1894635603 >> 172.21.15.26:0/1894635603 conn(0x564b0e5c8da0 msgr2=0x564b0e5ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 -- 172.21.15.26:0/1894635603 shutdown_connections 2023-07-13T22:14:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.331+0000 7f21180ccb80 1 -- 172.21.15.26:0/1894635603 wait complete. 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.332+0000 7f21180ccb80 1 -- start start 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.332+0000 7f21180ccb80 1 Processor -- start 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.332+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 0x564b0e5e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.332+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.332+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 0x564b0e5ecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f21180ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e4c4ac0 con 0x564b0e5d4c70 2023-07-13T22:14:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f21180ccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e4c47c0 con 0x564b0e5d34d0 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f21180ccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e3a8920 con 0x564b0e5d3e20 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f21180ccb80 10 client.0 _ll_register_callbacks cb 0x564b0e4c2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 0x564b0e5ecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 0x564b0e5e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 0x564b0e5ecee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44900/0 (socket says 172.21.15.26:44900) 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f2102d9c700 1 -- 172.21.15.26:0/261006949 learned_addr learned my addr 172.21.15.26:0/261006949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 -- 172.21.15.26:0/261006949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 msgr2=0x564b0e5e5ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5e5ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 -- 172.21.15.26:0/261006949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 msgr2=0x564b0e5ecee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 0x564b0e5ecee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.333+0000 7f210259b700 1 -- 172.21.15.26:0/261006949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec011440 con 0x564b0e5d34d0 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f210259b700 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 0x564b0e5e5960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20f80cb710 tx=0x7f20f8021b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3a85328716c86f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f20e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f809a7a0 con 0x564b0e5d34d0 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f80d2040 con 0x564b0e5d34d0 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b0e47e4f0 con 0x564b0e5d34d0 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f809b0b0 con 0x564b0e5d34d0 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f21180ccb80 10 client.25544 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.334+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b0e33bc60 con 0x564b0e5d34d0 2023-07-13T22:14:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b0e47e4f0 con 0x564b0e5d34d0 2023-07-13T22:14:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20f80aed20 con 0x564b0e5d34d0 2023-07-13T22:14:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 10 client.25544 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:43.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 20 client.25544 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e5f12b0 0x564b0e5f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 4 client.25544 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:14:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b0e51deb0 con 0x564b0e5f12b0 2023-07-13T22:14:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.335+0000 7f2101d9a700 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e5f12b0 0x564b0e5f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.336+0000 7f2101d9a700 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e5f12b0 0x564b0e5f3690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c4af6a069e392f server_cookie=24be2fde71795ace in_seq=0 out_seq=0 2023-07-13T22:14:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.336+0000 7f20e5ffb700 10 client.25544 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f20e5ffb700 1 -- 172.21.15.26:0/261006949 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564b0e51deb0 con 0x564b0e5f12b0 2023-07-13T22:14:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f20e5ffb700 10 client.25544 handle_command_reply: tid=0 2023-07-13T22:14:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 client.25544 shutdown 2023-07-13T22:14:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 msgr2=0x564b0e5e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 0x564b0e5e5960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f20f80cb710 tx=0x7f20f8021b80 comp rx=0 tx=0).stop 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 shutdown_connections 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d4c70 0x564b0e5ecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e5f12b0 0x564b0e5f3690 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d34d0 0x564b0e5e5960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 --2- 172.21.15.26:0/261006949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5e5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 >> 172.21.15.26:0/261006949 conn(0x564b0e5c8da0 msgr2=0x564b0e5cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f20e5ffb700 0 client.25544 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.337+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 shutdown_connections 2023-07-13T22:14:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.338+0000 7f21180ccb80 1 -- 172.21.15.26:0/261006949 wait complete. 2023-07-13T22:14:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.338+0000 7f21180ccb80 20 client.25544 trim_cache size 0 max 16384 2023-07-13T22:14:43.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 Processor -- start 2023-07-13T22:14:43.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 -- start start 2023-07-13T22:14:43.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5c8970 con 0x564b0e5d34d0 2023-07-13T22:14:43.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5ede20 con 0x564b0e5d4c70 2023-07-13T22:14:43.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.370+0000 7f21180ccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5e6860 con 0x564b0e5d3e20 2023-07-13T22:14:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44906/0 (socket says 172.21.15.26:44906) 2023-07-13T22:14:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 -- 172.21.15.26:0/2182403869 learned_addr learned my addr 172.21.15.26:0/2182403869 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f2101d9a700 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 -- 172.21.15.26:0/2182403869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 msgr2=0x564b0e5ecea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5ecea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 -- 172.21.15.26:0/2182403869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 msgr2=0x564b0e5d7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.371+0000 7f210259b700 1 -- 172.21.15.26:0/2182403869 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b0e5d5560 con 0x564b0e5d34d0 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.372+0000 7f210259b700 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f20f80975b0 tx=0x7f20f800b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743ed33bc04282d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.372+0000 7f20f17fa700 1 -- 172.21.15.26:0/2182403869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80a3270 con 0x564b0e5d34d0 2023-07-13T22:14:43.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.372+0000 7f20f17fa700 1 -- 172.21.15.26:0/2182403869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f809b0b0 con 0x564b0e5d34d0 2023-07-13T22:14:43.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.372+0000 7f20f17fa700 1 -- 172.21.15.26:0/2182403869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f809a7e0 con 0x564b0e5d34d0 2023-07-13T22:14:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 -- 172.21.15.26:0/2182403869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 msgr2=0x564b0e5db020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f20f80975b0 tx=0x7f20f800b320 comp rx=0 tx=0).stop 2023-07-13T22:14:43.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 -- 172.21.15.26:0/2182403869 shutdown_connections 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d34d0 0x564b0e5db020 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5d7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 --2- 172.21.15.26:0/2182403869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d3e20 0x564b0e5ecea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 -- 172.21.15.26:0/2182403869 >> 172.21.15.26:0/2182403869 conn(0x564b0e5c8da0 msgr2=0x564b0e5ed710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 -- 172.21.15.26:0/2182403869 shutdown_connections 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.373+0000 7f21180ccb80 1 -- 172.21.15.26:0/2182403869 wait complete. 2023-07-13T22:14:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 -- start start 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 Processor -- start 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5ec8b0 con 0x564b0e5d3e20 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5eca20 con 0x564b0e5d4c70 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b0e5ecbe0 con 0x564b0e5d34d0 2023-07-13T22:14:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.374+0000 7f21180ccb80 10 client.0 _ll_register_callbacks cb 0x564b0e4c2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43762/0 (socket says 172.21.15.26:43762) 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 -- 172.21.15.26:0/180864177 learned_addr learned my addr 172.21.15.26:0/180864177 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f210259b700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 -- 172.21.15.26:0/180864177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 msgr2=0x564b0e5e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5e9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 -- 172.21.15.26:0/180864177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 msgr2=0x564b0e5e9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:14:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 -- 172.21.15.26:0/180864177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b0e33bc60 con 0x564b0e5d4c70 2023-07-13T22:14:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f2102d9c700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20f4025410 tx=0x7f20f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2b33460fb9e1f3e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:14:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.375+0000 7f20e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4007ea0 con 0x564b0e5d4c70 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ec00faa0 con 0x564b0e5d4c70 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f401c030 con 0x564b0e5d4c70 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f4033030 con 0x564b0e5d4c70 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f21180ccb80 10 client.25550 fetch_fsmap learned FSMap version 27 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564b0e379250 con 0x564b0e5d4c70 2023-07-13T22:14:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.376+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ec00faa0 con 0x564b0e5d4c70 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20f40051d0 con 0x564b0e5d4c70 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 10 client.25550 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 20 client.25550 populate_metadata read hostname 'smithi026' 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e6462c0 0x564b0e6486d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 4 client.25550 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:14:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564b0e4e0050 con 0x564b0e6462c0 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.377+0000 7f210259b700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e6462c0 0x564b0e6486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f20e5ffb700 10 client.25550 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f210259b700 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e6462c0 0x564b0e6486d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be06024f00ede7f1 server_cookie=d1e89bdb3501861b in_seq=0 out_seq=0 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f20e5ffb700 1 -- 172.21.15.26:0/180864177 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564b0e4e0050 con 0x564b0e6462c0 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f20e5ffb700 10 client.25550 handle_command_reply: tid=0 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f21180ccb80 1 client.25550 shutdown 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 msgr2=0x564b0e5ec370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.378+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f20f4025410 tx=0x7f20f4004ad0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 shutdown_connections 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564b0e6462c0 0x564b0e6486d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564b0e5d4c70 0x564b0e5ec370 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564b0e5d34d0 0x564b0e5e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 --2- 172.21.15.26:0/180864177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564b0e5d3e20 0x564b0e5e9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f20e5ffb700 0 client.25550 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:14:43.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 >> 172.21.15.26:0/180864177 conn(0x564b0e5c8da0 msgr2=0x564b0e5db020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 shutdown_connections 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 1 -- 172.21.15.26:0/180864177 wait complete. 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.379+0000 7f21180ccb80 20 client.25550 trim_cache size 0 max 16384 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:14:43.375 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: "39a37ce7-0e98-4912-afd4-520e65e55696": { 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "39a37ce7-0e98-4912-afd4-520e65e55696", 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:14:43.376 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:14:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 msgr2=0x7f20e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 0x7f20e80a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009ec0 tx=0x7f20f4009040 comp rx=0 tx=0).stop 2023-07-13T22:14:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 msgr2=0x7f20fc197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:14:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20f80098e0 tx=0x7f20f8009580 comp rx=0 tx=0).stop 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 shutdown_connections 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1150 0x7f20e80a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20fc100560 0x7f20fc197a70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20fc1057a0 0x7f20fc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 --2- 172.21.15.26:0/3414783632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20fc102e80 0x7f20fc197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 >> 172.21.15.26:0/3414783632 conn(0x7f20fc0fa000 msgr2=0x7f20fc1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 shutdown_connections 2023-07-13T22:14:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:14:43.380+0000 7f21047ff700 1 -- 172.21.15.26:0/3414783632 wait complete. 2023-07-13T22:14:43.388 INFO:tasks.cephfs.filesystem:scrub status for tag:39a37ce7-0e98-4912-afd4-520e65e55696 - {'path': '/', 'tag': '39a37ce7-0e98-4912-afd4-520e65e55696', 'options': 'recursive,force'} 2023-07-13T22:14:44.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:44 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:14:44.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:44 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:44 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:14:44.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:44 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.745 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:44 smithi121 ceph-mon[129187]: pgmap v341: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:44.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:44 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:14:44.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:44 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:44 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:14:44.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:44 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.944 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:44 smithi117 ceph-mon[129074]: pgmap v341: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:44.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:44 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:14:44.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:44 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:44 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:14:44.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:44 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:14:44.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:44 smithi026 ceph-mon[95647]: pgmap v341: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:14:46.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:46 smithi121 ceph-mon[129187]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-13T22:14:46.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:46 smithi117 ceph-mon[129074]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-13T22:14:46.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:46 smithi026 ceph-mon[95647]: pgmap v342: 97 pgs: 97 active+clean; 651 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2023-07-13T22:14:47.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:47 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:14:47.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:47 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:14:47.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:47 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:14:48.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:48 smithi121 ceph-mon[129187]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-13T22:14:48.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:48 smithi117 ceph-mon[129074]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-13T22:14:48.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:48 smithi026 ceph-mon[95647]: pgmap v343: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 11 op/s 2023-07-13T22:14:50.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:50 smithi121 ceph-mon[129187]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-13T22:14:50.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:50 smithi117 ceph-mon[129074]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-13T22:14:50.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:50 smithi026 ceph-mon[95647]: pgmap v344: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2023-07-13T22:14:52.744 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:52 smithi121 ceph-mon[129187]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:52.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:52 smithi117 ceph-mon[129074]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:52.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:52 smithi026 ceph-mon[95647]: pgmap v345: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:54.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:54 smithi117 ceph-mon[129074]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-13T22:14:54.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:54 smithi026 ceph-mon[95647]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-13T22:14:54.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:54 smithi121 ceph-mon[129187]: pgmap v346: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 12 op/s 2023-07-13T22:14:56.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:56 smithi117 ceph-mon[129074]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:56.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:56 smithi026 ceph-mon[95647]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:56.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:56 smithi121 ceph-mon[129187]: pgmap v347: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2023-07-13T22:14:58.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:14:58 smithi117 ceph-mon[129074]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2023-07-13T22:14:58.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:14:58 smithi026 ceph-mon[95647]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2023-07-13T22:14:58.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:14:58 smithi121 ceph-mon[129187]: pgmap v348: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2023-07-13T22:15:00.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:00 smithi117 ceph-mon[129074]: pgmap v349: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-13T22:15:00.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:00 smithi026 ceph-mon[95647]: pgmap v349: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-13T22:15:00.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:00 smithi121 ceph-mon[129187]: pgmap v349: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-07-13T22:15:02.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:02 smithi117 ceph-mon[129074]: pgmap v350: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2023-07-13T22:15:02.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:02 smithi026 ceph-mon[95647]: pgmap v350: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2023-07-13T22:15:02.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:02 smithi121 ceph-mon[129187]: pgmap v350: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 8 op/s 2023-07-13T22:15:04.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:04 smithi117 ceph-mon[129074]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:15:04.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:04 smithi026 ceph-mon[95647]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:15:04.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:04 smithi121 ceph-mon[129187]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2023-07-13T22:15:06.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:06 smithi117 ceph-mon[129074]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:06.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:06 smithi026 ceph-mon[95647]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:06.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:06 smithi121 ceph-mon[129187]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:07.870 INFO:tasks.workunit.client.0.smithi026.stderr:Updating files: 62% (7330/11710) Updating files: 63% (7378/11710) Updating files: 64% (7495/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 99% (11647/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-13T22:15:08.872 INFO:tasks.workunit.client.0.smithi026.stderr:Note: switching to '3fc0089cc1082313a98853d48b892acdc1385b8b'. 2023-07-13T22:15:08.872 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.872 INFO:tasks.workunit.client.0.smithi026.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-13T22:15:08.872 INFO:tasks.workunit.client.0.smithi026.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-13T22:15:08.872 INFO:tasks.workunit.client.0.smithi026.stderr:state without impacting any branches by switching back to a branch. 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr: git switch -c 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr:Or undo this operation with: 2023-07-13T22:15:08.873 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.874 INFO:tasks.workunit.client.0.smithi026.stderr: git switch - 2023-07-13T22:15:08.874 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.874 INFO:tasks.workunit.client.0.smithi026.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-13T22:15:08.874 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:08.874 INFO:tasks.workunit.client.0.smithi026.stderr:HEAD is now at 3fc0089cc10 Merge remote-tracking branch 'rishabh/fs-auth-subcmd' into wip-rishabh-2023Jul13 2023-07-13T22:15:08.888 DEBUG:teuthology.orchestra.run.smithi026:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-13T22:15:08.941 INFO:tasks.workunit.client.0.smithi026.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-13T22:15:08.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:08 smithi117 ceph-mon[129074]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:15:08.946 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-13T22:15:08.946 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-13T22:15:08.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:08 smithi026 ceph-mon[95647]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:15:08.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:08 smithi121 ceph-mon[129187]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2023-07-13T22:15:09.503 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-13T22:15:09.636 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-13T22:15:09.700 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-13T22:15:09.704 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-13T22:15:09.704 INFO:tasks.workunit.client.0.smithi026.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-13T22:15:09.768 INFO:tasks.workunit.client.0.smithi026.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-13T22:15:09.775 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:15:09.775 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-13T22:15:09.818 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2023-07-13T22:15:09.820 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2023-07-13T22:15:09.820 DEBUG:teuthology.orchestra.run.smithi026:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3fc0089cc1082313a98853d48b892acdc1385b8b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2023-07-13T22:15:09.917 INFO:tasks.workunit.client.0.smithi026.stdout:getting iogen 2023-07-13T22:15:09.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'getting iogen' 2023-07-13T22:15:09.917 INFO:tasks.workunit.client.0.smithi026.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2023-07-13T22:15:09.948 INFO:tasks.workunit.client.0.smithi026.stderr:--2023-07-13 22:15:09-- http://download.ceph.com/qa/iogen_3.1p0.tar 2023-07-13T22:15:10.025 INFO:tasks.workunit.client.0.smithi026.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-07-13T22:15:10.132 INFO:tasks.workunit.client.0.smithi026.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-07-13T22:15:10.232 INFO:tasks.workunit.client.0.smithi026.stderr:HTTP request sent, awaiting response... 200 OK 2023-07-13T22:15:10.232 INFO:tasks.workunit.client.0.smithi026.stderr:Length: 8071 (7.9K) [application/octet-stream] 2023-07-13T22:15:10.234 INFO:tasks.workunit.client.0.smithi026.stderr:Saving to: ‘iogen_3.1p0.tar’ 2023-07-13T22:15:10.239 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:10.239 INFO:tasks.workunit.client.0.smithi026.stderr: 0K ....... 100% 404M=0s 2023-07-13T22:15:10.239 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:10.239 INFO:tasks.workunit.client.0.smithi026.stderr:2023-07-13 22:15:10 (404 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2023-07-13T22:15:10.239 INFO:tasks.workunit.client.0.smithi026.stderr: 2023-07-13T22:15:10.240 INFO:tasks.workunit.client.0.smithi026.stderr:+ tar -xvzf iogen_3.1p0.tar 2023-07-13T22:15:10.247 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0 2023-07-13T22:15:10.248 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/src 2023-07-13T22:15:10.249 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/src/iogen.8 2023-07-13T22:15:10.251 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/src/iogen.c 2023-07-13T22:15:10.252 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/README 2023-07-13T22:15:10.253 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/Makefile.bsd 2023-07-13T22:15:10.254 INFO:tasks.workunit.client.0.smithi026.stdout:iogen_3.1p0/Makefile 2023-07-13T22:15:10.256 INFO:tasks.workunit.client.0.smithi026.stderr:+ cd iogen_3.1p0 2023-07-13T22:15:10.256 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'making iogen' 2023-07-13T22:15:10.256 INFO:tasks.workunit.client.0.smithi026.stderr:+ make 2023-07-13T22:15:10.256 INFO:tasks.workunit.client.0.smithi026.stdout:making iogen 2023-07-13T22:15:10.267 INFO:tasks.workunit.client.0.smithi026.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2023-07-13T22:15:10.487 INFO:tasks.workunit.client.0.smithi026.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2023-07-13T22:15:10.597 INFO:tasks.workunit.client.0.smithi026.stdout:running iogen 2023-07-13T22:15:10.598 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'running iogen' 2023-07-13T22:15:10.598 INFO:tasks.workunit.client.0.smithi026.stderr:+ ./iogen -n 5 -s 2g 2023-07-13T22:15:10.599 INFO:tasks.workunit.client.0.smithi026.stderr:child spawned 146323 2023-07-13T22:15:10.599 INFO:tasks.workunit.client.0.smithi026.stderr:child spawned 146324 2023-07-13T22:15:10.600 INFO:tasks.workunit.client.0.smithi026.stderr:child spawned 146325 2023-07-13T22:15:10.600 INFO:tasks.workunit.client.0.smithi026.stderr:child spawned 146326 2023-07-13T22:15:10.600 INFO:tasks.workunit.client.0.smithi026.stderr:child spawned 146327 2023-07-13T22:15:10.601 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'sleep for 10 min' 2023-07-13T22:15:10.601 INFO:tasks.workunit.client.0.smithi026.stdout:sleep for 10 min 2023-07-13T22:15:10.601 INFO:tasks.workunit.client.0.smithi026.stderr:+ sleep 600 2023-07-13T22:15:10.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:10 smithi117 ceph-mon[129074]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:15:10.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:10 smithi026 ceph-mon[95647]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:15:10.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:10 smithi121 ceph-mon[129187]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-07-13T22:15:12.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:12 smithi117 ceph-mon[129074]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:12.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:12 smithi026 ceph-mon[95647]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:12.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:12 smithi121 ceph-mon[129187]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2023-07-13T22:15:13.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:15:14.688 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:14 smithi026 ceph-mon[95647]: pgmap v356: 97 pgs: 97 active+clean; 97 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 19 op/s 2023-07-13T22:15:14.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.790+0000 7f7a901ff700 1 -- 172.21.15.26:0/142640605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a8810bd60 msgr2=0x7f7a8810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.790+0000 7f7a901ff700 1 --2- 172.21.15.26:0/142640605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a8810bd60 0x7f7a8810e1f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a780098f0 tx=0x7f7a78009c00 comp rx=0 tx=0).stop 2023-07-13T22:15:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.793+0000 7f7a901ff700 1 -- 172.21.15.26:0/142640605 shutdown_connections 2023-07-13T22:15:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.793+0000 7f7a901ff700 1 --2- 172.21.15.26:0/142640605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a8810bd60 0x7f7a8810e1f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.793+0000 7f7a901ff700 1 --2- 172.21.15.26:0/142640605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a880712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.793+0000 7f7a901ff700 1 --2- 172.21.15.26:0/142640605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a88071830 0x7f7a8810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.793+0000 7f7a901ff700 1 -- 172.21.15.26:0/142640605 >> 172.21.15.26:0/142640605 conn(0x7f7a8806ca90 msgr2=0x7f7a8806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:14.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.814+0000 7f7a901ff700 1 -- 172.21.15.26:0/142640605 shutdown_connections 2023-07-13T22:15:14.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.814+0000 7f7a901ff700 1 -- 172.21.15.26:0/142640605 wait complete. 2023-07-13T22:15:14.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.815+0000 7f7a901ff700 1 Processor -- start 2023-07-13T22:15:14.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 -- start start 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a88071830 0x7f7a88113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a8810bd60 0x7f7a881b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88117090 con 0x7f7a88071830 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88117200 con 0x7f7a88070ee0 2023-07-13T22:15:14.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.816+0000 7f7a901ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a881b4e20 con 0x7f7a8810bd60 2023-07-13T22:15:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.817+0000 7f7a8df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.817+0000 7f7a8df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48562/0 (socket says 172.21.15.26:48562) 2023-07-13T22:15:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.817+0000 7f7a8df9b700 1 -- 172.21.15.26:0/587863597 learned_addr learned my addr 172.21.15.26:0/587863597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.819+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a88071830 0x7f7a88113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.822+0000 7f7a8df9b700 1 -- 172.21.15.26:0/587863597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a8810bd60 msgr2=0x7f7a881b48e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.822+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a8810bd60 0x7f7a881b48e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.822+0000 7f7a8df9b700 1 -- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a88071830 msgr2=0x7f7a88113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.822+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a88071830 0x7f7a88113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.822+0000 7f7a8df9b700 1 -- 172.21.15.26:0/587863597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009520 con 0x7f7a88070ee0 2023-07-13T22:15:14.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.823+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000ba50 tx=0x7f7a8000be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.833+0000 7f7a7effd700 1 -- 172.21.15.26:0/587863597 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000ca50 con 0x7f7a88070ee0 2023-07-13T22:15:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.833+0000 7f7a7effd700 1 -- 172.21.15.26:0/587863597 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a8000cbb0 con 0x7f7a88070ee0 2023-07-13T22:15:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.833+0000 7f7a7effd700 1 -- 172.21.15.26:0/587863597 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800075b0 con 0x7f7a88070ee0 2023-07-13T22:15:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.833+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a881b4fc0 con 0x7f7a88070ee0 2023-07-13T22:15:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.833+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a881b5500 con 0x7f7a88070ee0 2023-07-13T22:15:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.850+0000 7f7a7effd700 1 -- 172.21.15.26:0/587863597 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a80014440 con 0x7f7a88070ee0 2023-07-13T22:15:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.866+0000 7f7a7effd700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 0x7f7a740a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.867+0000 7f7a7effd700 1 -- 172.21.15.26:0/587863597 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a800cbff0 con 0x7f7a88070ee0 2023-07-13T22:15:14.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.884+0000 7f7aa3b92b80 1 Processor -- start 2023-07-13T22:15:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.885+0000 7f7aa3b92b80 1 -- start start 2023-07-13T22:15:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f6c5cb0 con 0x55fd3f7c04d0 2023-07-13T22:15:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f718a40 con 0x55fd3f7c1c70 2023-07-13T22:15:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.886+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f711ed0 con 0x55fd3f7c0e20 2023-07-13T22:15:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.896+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.897+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7c16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42070/0 (socket says 172.21.15.26:42070) 2023-07-13T22:15:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.897+0000 7f7a8d79a700 1 -- 172.21.15.26:0/2586194078 learned_addr learned my addr 172.21.15.26:0/2586194078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.897+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.907+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.912+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2586194078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 msgr2=0x55fd3f7c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.912+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7c16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.912+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2586194078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 msgr2=0x55fd3f7c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.912+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7c20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.912+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2586194078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3f53fde0 con 0x55fd3f7c04d0 2023-07-13T22:15:14.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.916+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 0x7f7a740a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.920+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7c08e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000ba20 tx=0x7f7a80098d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48482eb1d4c5f37 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.921+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 0x7f7a740a3800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d5c0 tx=0x7f7a84009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:14.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.932+0000 7f7a73fff700 1 -- 172.21.15.26:0/2586194078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8009d3e0 con 0x55fd3f7c04d0 2023-07-13T22:15:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.932+0000 7f7a73fff700 1 -- 172.21.15.26:0/2586194078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a800a1800 con 0x55fd3f7c04d0 2023-07-13T22:15:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.932+0000 7f7a73fff700 1 -- 172.21.15.26:0/2586194078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80099870 con 0x55fd3f7c04d0 2023-07-13T22:15:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.944+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2586194078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 msgr2=0x55fd3f7c08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.944+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7c08e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000ba20 tx=0x7f7a80098d90 comp rx=0 tx=0).stop 2023-07-13T22:15:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.995+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2586194078 shutdown_connections 2023-07-13T22:15:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.995+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7c08e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.995+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.995+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2586194078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:14.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:14.995+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2586194078 >> 172.21.15.26:0/2586194078 conn(0x55fd3f7b5da0 msgr2=0x55fd3f7bb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:14.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:14 smithi121 ceph-mon[129187]: pgmap v356: 97 pgs: 97 active+clean; 97 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 19 op/s 2023-07-13T22:15:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.023+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2586194078 shutdown_connections 2023-07-13T22:15:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.024+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2586194078 wait complete. 2023-07-13T22:15:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.025+0000 7f7aa3b92b80 1 -- start start 2023-07-13T22:15:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.025+0000 7f7aa3b92b80 1 Processor -- start 2023-07-13T22:15:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f6b1ac0 con 0x55fd3f7c0e20 2023-07-13T22:15:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f6b17c0 con 0x55fd3f7c1c70 2023-07-13T22:15:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f595920 con 0x55fd3f7c04d0 2023-07-13T22:15:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.026+0000 7f7aa3b92b80 10 client.0 _ll_register_callbacks cb 0x55fd3f6af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.027+0000 7f7a8df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.027+0000 7f7a8df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42076/0 (socket says 172.21.15.26:42076) 2023-07-13T22:15:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.027+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2693842893 learned_addr learned my addr 172.21.15.26:0/2693842893 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.027+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.028+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 msgr2=0x55fd3f7d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.028+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.028+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2693842893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 msgr2=0x55fd3f7ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.028+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.028+0000 7f7a8df9b700 1 -- 172.21.15.26:0/2693842893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a881b5500 con 0x55fd3f7c04d0 2023-07-13T22:15:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.030+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a80095e20 tx=0x7f7a800c9a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3850d88fa65c9ef server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.030+0000 7f7a717fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.030+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800c3be0 con 0x55fd3f7c04d0 2023-07-13T22:15:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.030+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a800c3d40 con 0x55fd3f7c04d0 2023-07-13T22:15:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.031+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd3f66b4f0 con 0x55fd3f7c04d0 2023-07-13T22:15:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.031+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a80099610 con 0x55fd3f7c04d0 2023-07-13T22:15:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.032+0000 7f7aa3b92b80 10 client.25534 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.032+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd3f528c60 con 0x55fd3f7c04d0 2023-07-13T22:15:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd3f66b4f0 con 0x55fd3f7c04d0 2023-07-13T22:15:15.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a80098310 con 0x55fd3f7c04d0 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 10 client.25534 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 20 client.25534 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f7d5c50 0x55fd3f7d8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 4 client.25534 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.033+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd3f70aeb0 con 0x55fd3f7d5c50 2023-07-13T22:15:15.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.034+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f7d5c50 0x55fd3f7d8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.042+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f7d5c50 0x55fd3f7d8060 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ec61737a33afe7 server_cookie=e54162843a53c373 in_seq=0 out_seq=0 2023-07-13T22:15:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.042+0000 7f7a717fa700 10 client.25534 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.043+0000 7f7a717fa700 1 -- 172.21.15.26:0/2693842893 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd3f70aeb0 con 0x55fd3f7d5c50 2023-07-13T22:15:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.043+0000 7f7a717fa700 10 client.25534 handle_command_reply: tid=0 2023-07-13T22:15:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.043+0000 7f7aa3b92b80 1 client.25534 shutdown 2023-07-13T22:15:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 msgr2=0x55fd3f7ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a80095e20 tx=0x7f7a800c9a40 comp rx=0 tx=0).stop 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 shutdown_connections 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f7d5c50 0x55fd3f7d8060 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c04d0 0x55fd3f7ca1b0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7a80095e20 tx=0x7f7a800c9a40 comp rx=0 tx=0).stop 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/2693842893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 >> 172.21.15.26:0/2693842893 conn(0x55fd3f7b5da0 msgr2=0x55fd3f7ba8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.044+0000 7f7a717fa700 0 client.25534 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.045+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 shutdown_connections 2023-07-13T22:15:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.045+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/2693842893 wait complete. 2023-07-13T22:15:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.046+0000 7f7aa3b92b80 20 client.25534 trim_cache size 0 max 16384 2023-07-13T22:15:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.085+0000 7f7aa3b92b80 1 Processor -- start 2023-07-13T22:15:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.085+0000 7f7aa3b92b80 1 -- start start 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7b59c0 con 0x55fd3f7c0e20 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7b7ec0 con 0x55fd3f7b7310 2023-07-13T22:15:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.088+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7bc2e0 con 0x55fd3f7c1c70 2023-07-13T22:15:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.089+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.089+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42084/0 (socket says 172.21.15.26:42084) 2023-07-13T22:15:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.089+0000 7f7a8d79a700 1 -- 172.21.15.26:0/1281735883 learned_addr learned my addr 172.21.15.26:0/1281735883 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.089+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8e79c700 1 -- 172.21.15.26:0/1281735883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 msgr2=0x55fd3f7d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8e79c700 1 -- 172.21.15.26:0/1281735883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 msgr2=0x55fd3f7ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ba8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.095+0000 7f7a8e79c700 1 -- 172.21.15.26:0/1281735883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd3f7c2560 con 0x55fd3f7b7310 2023-07-13T22:15:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.096+0000 7f7a8e79c700 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d7c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009230 tx=0x7f7a7800f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf9a71646aae0647 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.107+0000 7f7a7cff9700 1 -- 172.21.15.26:0/1281735883 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7801d070 con 0x55fd3f7b7310 2023-07-13T22:15:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.107+0000 7f7a7cff9700 1 -- 172.21.15.26:0/1281735883 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a7800f760 con 0x55fd3f7b7310 2023-07-13T22:15:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.107+0000 7f7a7cff9700 1 -- 172.21.15.26:0/1281735883 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800bb60 con 0x55fd3f7b7310 2023-07-13T22:15:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.115+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/1281735883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 msgr2=0x55fd3f7d7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.115+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d7c60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009230 tx=0x7f7a7800f5b0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.118+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/1281735883 shutdown_connections 2023-07-13T22:15:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.118+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7ba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.118+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d7c60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.118+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/1281735883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.118+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/1281735883 >> 172.21.15.26:0/1281735883 conn(0x55fd3f7b5da0 msgr2=0x55fd3f7c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.132+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/1281735883 shutdown_connections 2023-07-13T22:15:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.132+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/1281735883 wait complete. 2023-07-13T22:15:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.133+0000 7f7aa3b92b80 1 -- start start 2023-07-13T22:15:15.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.133+0000 7f7aa3b92b80 1 Processor -- start 2023-07-13T22:15:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7ce9e0 con 0x55fd3f7c1c70 2023-07-13T22:15:15.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7d1860 con 0x55fd3f7b7310 2023-07-13T22:15:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd3f7d1a20 con 0x55fd3f7c0e20 2023-07-13T22:15:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.134+0000 7f7aa3b92b80 10 client.0 _ll_register_callbacks cb 0x7f7a78009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.137+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.137+0000 7f7a8d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42092/0 (socket says 172.21.15.26:42092) 2023-07-13T22:15:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.137+0000 7f7a8d79a700 1 -- 172.21.15.26:0/3240704397 learned_addr learned my addr 172.21.15.26:0/3240704397 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:15.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.137+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.141+0000 7f7a8df9b700 1 -- 172.21.15.26:0/3240704397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 msgr2=0x55fd3f7cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.141+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.141+0000 7f7a8df9b700 1 -- 172.21.15.26:0/3240704397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c1c70 msgr2=0x55fd3f7ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.141+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.141+0000 7f7a8df9b700 1 -- 172.21.15.26:0/3240704397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a88117640 con 0x55fd3f7b7310 2023-07-13T22:15:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.144+0000 7f7a8df9b700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d11b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e000 tx=0x7f7a800a15a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f219602b778fd98b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.145+0000 7f7a717fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.156+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80014a90 con 0x55fd3f7b7310 2023-07-13T22:15:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.156+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a80099610 con 0x55fd3f7b7310 2023-07-13T22:15:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.156+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a800a1800 con 0x55fd3f7b7310 2023-07-13T22:15:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.165+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a80098d10 con 0x55fd3f7b7310 2023-07-13T22:15:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.165+0000 7f7aa3b92b80 10 client.25565 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:15.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.165+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd3f566250 con 0x55fd3f7b7310 2023-07-13T22:15:15.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a800a1800 con 0x55fd3f7b7310 2023-07-13T22:15:15.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a800b7b50 con 0x55fd3f7b7310 2023-07-13T22:15:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 20 client.25565 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f82ad00 0x55fd3f82d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 4 client.25565 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:15:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.166+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd3f6cd050 con 0x55fd3f82ad00 2023-07-13T22:15:15.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.168+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f82ad00 0x55fd3f82d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.170+0000 7f7a8d79a700 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f82ad00 0x55fd3f82d0e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81218a41fd740d14 server_cookie=c5199c5a946cdc53 in_seq=0 out_seq=0 2023-07-13T22:15:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.170+0000 7f7a717fa700 10 client.25565 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.171+0000 7f7a717fa700 1 -- 172.21.15.26:0/3240704397 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fd3f6cd050 con 0x55fd3f82ad00 2023-07-13T22:15:15.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.171+0000 7f7a717fa700 10 client.25565 handle_command_reply: tid=0 2023-07-13T22:15:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.171+0000 7f7aa3b92b80 1 client.25565 shutdown 2023-07-13T22:15:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.171+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 msgr2=0x55fd3f7d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.171+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d11b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000e000 tx=0x7f7a800a15a0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 shutdown_connections 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fd3f7c1c70 0x55fd3f7ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55fd3f82ad00 0x55fd3f82d0e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fd3f7b7310 0x55fd3f7d11b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 --2- 172.21.15.26:0/3240704397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd3f7c0e20 0x55fd3f7cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 >> 172.21.15.26:0/3240704397 conn(0x55fd3f7b5da0 msgr2=0x55fd3f7ba8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:15.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7a717fa700 0 client.25565 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:15.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.172+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 shutdown_connections 2023-07-13T22:15:15.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.173+0000 7f7aa3b92b80 1 -- 172.21.15.26:0/3240704397 wait complete. 2023-07-13T22:15:15.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.174+0000 7f7aa3b92b80 20 client.25565 trim_cache size 0 max 16384 2023-07-13T22:15:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:15:15.164 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:15:15.164 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:15:15.164 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:15:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 msgr2=0x7f7a740a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 0x7f7a740a3800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d5c0 tx=0x7f7a84009310 comp rx=0 tx=0).stop 2023-07-13T22:15:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 msgr2=0x7f7a88113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000ba50 tx=0x7f7a8000be10 comp rx=0 tx=0).stop 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 shutdown_connections 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a8810bd60 0x7f7a881b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a88070ee0 0x7f7a88113a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a740a1350 0x7f7a740a3800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 --2- 172.21.15.26:0/587863597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a88071830 0x7f7a88113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.175+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 >> 172.21.15.26:0/587863597 conn(0x7f7a8806ca90 msgr2=0x7f7a8806e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.176+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 shutdown_connections 2023-07-13T22:15:15.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.176+0000 7f7a901ff700 1 -- 172.21.15.26:0/587863597 wait complete. 2023-07-13T22:15:15.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:14 smithi117 ceph-mon[129074]: pgmap v356: 97 pgs: 97 active+clean; 97 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.0 MiB/s wr, 19 op/s 2023-07-13T22:15:15.207 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:15:15.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:15:15.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.896+0000 7f752b9ff700 1 -- 172.21.15.26:0/1138865059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f752410bd60 msgr2=0x7f752410e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.896+0000 7f752b9ff700 1 --2- 172.21.15.26:0/1138865059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f752410bd60 0x7f752410e1e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f751c01c0d0 tx=0x7f751c01c3e0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 -- 172.21.15.26:0/1138865059 shutdown_connections 2023-07-13T22:15:15.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 --2- 172.21.15.26:0/1138865059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7524070ed0 0x7f75240712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 --2- 172.21.15.26:0/1138865059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f752410bd60 0x7f752410e1e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 --2- 172.21.15.26:0/1138865059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f752410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 -- 172.21.15.26:0/1138865059 >> 172.21.15.26:0/1138865059 conn(0x7f752406ca40 msgr2=0x7f752406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.907+0000 7f752b9ff700 1 -- 172.21.15.26:0/1138865059 shutdown_connections 2023-07-13T22:15:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.918+0000 7f752b9ff700 1 -- 172.21.15.26:0/1138865059 wait complete. 2023-07-13T22:15:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.919+0000 7f752b9ff700 1 Processor -- start 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 -- start start 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7524070ed0 0x7f752419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f75241a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241a3980 con 0x7f752410bd60 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241a3af0 con 0x7f7524071820 2023-07-13T22:15:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.920+0000 7f752b9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241a3cb0 con 0x7f7524070ed0 2023-07-13T22:15:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.921+0000 7f7529f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.921+0000 7f7529f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47478/0 (socket says 172.21.15.26:47478) 2023-07-13T22:15:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.921+0000 7f7529f9c700 1 -- 172.21.15.26:0/711138958 learned_addr learned my addr 172.21.15.26:0/711138958 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f752979b700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7524070ed0 0x7f752419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7528f9a700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f75241a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7529f9c700 1 -- 172.21.15.26:0/711138958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7524070ed0 msgr2=0x7f752419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7529f9c700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7524070ed0 0x7f752419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7529f9c700 1 -- 172.21.15.26:0/711138958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 msgr2=0x7f75241a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7529f9c700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f75241a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7529f9c700 1 -- 172.21.15.26:0/711138958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c01ad10 con 0x7f752410bd60 2023-07-13T22:15:15.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.922+0000 7f7528f9a700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f75241a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:15:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.923+0000 7f7529f9c700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f751c003c30 tx=0x7f751c003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.924+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c01f070 con 0x7f752410bd60 2023-07-13T22:15:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.924+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f751c01de60 con 0x7f752410bd60 2023-07-13T22:15:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.924+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c02ba90 con 0x7f752410bd60 2023-07-13T22:15:15.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.924+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75241a3f30 con 0x7f752410bd60 2023-07-13T22:15:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.924+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75241a4380 con 0x7f752410bd60 2023-07-13T22:15:15.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.933+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75080052f0 con 0x7f752410bd60 2023-07-13T22:15:15.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.935+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f751c0052c0 con 0x7f752410bd60 2023-07-13T22:15:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.942+0000 7f751a7fc700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 0x7f75100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:15.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.942+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f751c0de370 con 0x7f752410bd60 2023-07-13T22:15:15.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.951+0000 7f752979b700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 0x7f75100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:15.958+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f751c0a7b00 con 0x7f752410bd60 2023-07-13T22:15:16.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.014+0000 7f752979b700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 0x7f75100a3870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7514006fd0 tx=0x7f7514008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:16.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.784+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7508005f40 con 0x7f752410bd60 2023-07-13T22:15:16.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.799+0000 7f751a7fc700 1 -- 172.21.15.26:0/711138958 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f751c0ab120 con 0x7f752410bd60 2023-07-13T22:15:16.801 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:15:16.801 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:12:26.449971+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:15:16.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.842+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 msgr2=0x7f75100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:16.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.842+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 0x7f75100a3870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7514006fd0 tx=0x7f7514008040 comp rx=0 tx=0).stop 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.842+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 msgr2=0x7f75241a31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.842+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f751c003c30 tx=0x7f751c003d10 comp rx=0 tx=0).stop 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 shutdown_connections 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f75100a13c0 0x7f75100a3870 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7514006fd0 tx=0x7f7514008040 comp rx=0 tx=0).stop 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f752410bd60 0x7f75241a31b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7524070ed0 0x7f752419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 --2- 172.21.15.26:0/711138958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7524071820 0x7f75241a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.843+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 >> 172.21.15.26:0/711138958 conn(0x7f752406ca40 msgr2=0x7f752406e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.855+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 shutdown_connections 2023-07-13T22:15:16.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:16.863+0000 7f752b9ff700 1 -- 172.21.15.26:0/711138958 wait complete. 2023-07-13T22:15:16.888 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-07-13T22:15:16.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:15:16.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:16 smithi121 ceph-mon[129187]: pgmap v357: 97 pgs: 97 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-07-13T22:15:17.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:16 smithi117 ceph-mon[129074]: pgmap v357: 97 pgs: 97 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-07-13T22:15:17.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:16 smithi026 ceph-mon[95647]: pgmap v357: 97 pgs: 97 active+clean; 497 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-07-13T22:15:17.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.785+0000 7f57ad3ff700 1 -- 172.21.15.26:0/4160678158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a810bd60 msgr2=0x7f57a810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:17.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.785+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/4160678158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a810bd60 0x7f57a810e1f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b0a0 tx=0x7f57a000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.798+0000 7f57ad3ff700 1 -- 172.21.15.26:0/4160678158 shutdown_connections 2023-07-13T22:15:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.798+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/4160678158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a8070ee0 0x7f57a80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.798+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/4160678158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a810bd60 0x7f57a810e1f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.798+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/4160678158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8071830 0x7f57a810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.798+0000 7f57ad3ff700 1 -- 172.21.15.26:0/4160678158 >> 172.21.15.26:0/4160678158 conn(0x7f57a806ca90 msgr2=0x7f57a806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.809+0000 7f57ad3ff700 1 -- 172.21.15.26:0/4160678158 shutdown_connections 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.809+0000 7f57ad3ff700 1 -- 172.21.15.26:0/4160678158 wait complete. 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.811+0000 7f57ad3ff700 1 Processor -- start 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 -- start start 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8070ee0 0x7f57a81a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a810bd60 0x7f57a81abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81ac350 con 0x7f57a810bd60 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a810ec40 con 0x7f57a8071830 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.812+0000 7f57ad3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a810edd0 con 0x7f57a8070ee0 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.813+0000 7f57a67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.813+0000 7f57a6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8070ee0 0x7f57a81a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.813+0000 7f57a67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48644/0 (socket says 172.21.15.26:48644) 2023-07-13T22:15:17.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.813+0000 7f57a67fc700 1 -- 172.21.15.26:0/3943188240 learned_addr learned my addr 172.21.15.26:0/3943188240 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.823+0000 7f57a67fc700 1 -- 172.21.15.26:0/3943188240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8070ee0 msgr2=0x7f57a81a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.823+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8070ee0 0x7f57a81a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.823+0000 7f57a67fc700 1 -- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a810bd60 msgr2=0x7f57a81abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.823+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a810bd60 0x7f57a81abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.823+0000 7f57a67fc700 1 -- 172.21.15.26:0/3943188240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009d00 con 0x7f57a8071830 2023-07-13T22:15:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.829+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f579000e970 tx=0x7f579000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.837+0000 7f579ffff700 1 -- 172.21.15.26:0/3943188240 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790009790 con 0x7f57a8071830 2023-07-13T22:15:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.837+0000 7f579ffff700 1 -- 172.21.15.26:0/3943188240 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579000eec0 con 0x7f57a8071830 2023-07-13T22:15:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.837+0000 7f579ffff700 1 -- 172.21.15.26:0/3943188240 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790003c80 con 0x7f57a8071830 2023-07-13T22:15:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.838+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a810f0b0 con 0x7f57a8071830 2023-07-13T22:15:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.839+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a810f5c0 con 0x7f57a8071830 2023-07-13T22:15:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.908+0000 7f579ffff700 1 -- 172.21.15.26:0/3943188240 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f579000cc60 con 0x7f57a8071830 2023-07-13T22:15:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.910+0000 7f579ffff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 0x7f57940a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.911+0000 7f579ffff700 1 -- 172.21.15.26:0/3943188240 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5790014070 con 0x7f57a8071830 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.916+0000 7f57c0c5bb80 1 Processor -- start 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.916+0000 7f57c0c5bb80 1 -- start start 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:17.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8e4ccb0 con 0x555df8f48c70 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8e9fa40 con 0x555df8f474d0 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.917+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8e98ed0 con 0x555df8f47e20 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.923+0000 7f57a67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.923+0000 7f57a6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.923+0000 7f57a6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48654/0 (socket says 172.21.15.26:48654) 2023-07-13T22:15:17.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.923+0000 7f57a6ffd700 1 -- 172.21.15.26:0/2769662387 learned_addr learned my addr 172.21.15.26:0/2769662387 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.940+0000 7f57a6ffd700 1 -- 172.21.15.26:0/2769662387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 msgr2=0x555df8f486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.940+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.940+0000 7f57a6ffd700 1 -- 172.21.15.26:0/2769662387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 msgr2=0x555df8f490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.940+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.940+0000 7f57a6ffd700 1 -- 172.21.15.26:0/2769662387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df8cc6de0 con 0x555df8f474d0 2023-07-13T22:15:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.941+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5798009c00 tx=0x7f5798009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449ce5ed1f040a7c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.955+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 0x7f57940a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:17.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.974+0000 7f579d7fa700 1 -- 172.21.15.26:0/2769662387 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798009700 con 0x555df8f474d0 2023-07-13T22:15:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.974+0000 7f579d7fa700 1 -- 172.21.15.26:0/2769662387 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5798014470 con 0x555df8f474d0 2023-07-13T22:15:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.975+0000 7f579d7fa700 1 -- 172.21.15.26:0/2769662387 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798013630 con 0x555df8f474d0 2023-07-13T22:15:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.981+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2769662387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 msgr2=0x555df8f478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.981+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5798009c00 tx=0x7f5798009f10 comp rx=0 tx=0).stop 2023-07-13T22:15:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.986+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 0x7f57940a3800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5798019000 tx=0x7f5798013920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.993+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2769662387 shutdown_connections 2023-07-13T22:15:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.993+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.993+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f478e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.993+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2769662387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:17.993+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2769662387 >> 172.21.15.26:0/2769662387 conn(0x555df8f3cda0 msgr2=0x555df8f421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.027+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2769662387 shutdown_connections 2023-07-13T22:15:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.029+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2769662387 wait complete. 2023-07-13T22:15:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.033+0000 7f57c0c5bb80 1 -- start start 2023-07-13T22:15:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.034+0000 7f57c0c5bb80 1 Processor -- start 2023-07-13T22:15:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.041+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 0x555df8f51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f47e20 0x555df8f51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8e38ac0 con 0x555df8f47e20 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8e387c0 con 0x555df8f48c70 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8d1c920 con 0x555df8f474d0 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.042+0000 7f57c0c5bb80 10 client.0 _ll_register_callbacks cb 0x555df8e36760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.045+0000 7f57a6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 0x555df8f51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.045+0000 7f57a6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 0x555df8f51120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42162/0 (socket says 172.21.15.26:42162) 2023-07-13T22:15:18.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.045+0000 7f57a6ffd700 1 -- 172.21.15.26:0/4223069491 learned_addr learned my addr 172.21.15.26:0/4223069491 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.046+0000 7f57a77fe700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.047+0000 7f57a77fe700 1 -- 172.21.15.26:0/4223069491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 msgr2=0x555df8f51120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.047+0000 7f57a77fe700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 0x555df8f51120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.047+0000 7f57a77fe700 1 -- 172.21.15.26:0/4223069491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f47e20 msgr2=0x555df8f51660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.047+0000 7f57a77fe700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f47e20 0x555df8f51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.047+0000 7f57a77fe700 1 -- 172.21.15.26:0/4223069491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5798009860 con 0x555df8f48c70 2023-07-13T22:15:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.057+0000 7f57a77fe700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f586a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b0a0 tx=0x7f57a00074e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14bd076eb999864 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.058+0000 7f578e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.059+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000e040 con 0x555df8f48c70 2023-07-13T22:15:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.059+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f57a000bcb0 con 0x555df8f48c70 2023-07-13T22:15:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.060+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555df8df24f0 con 0x555df8f48c70 2023-07-13T22:15:18.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.066+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57a000be70 con 0x555df8f48c70 2023-07-13T22:15:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.085+0000 7f57c0c5bb80 10 client.25580 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:18.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.085+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555df8cafc60 con 0x555df8f48c70 2023-07-13T22:15:18.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.102+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555df8df24f0 con 0x555df8f48c70 2023-07-13T22:15:18.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.103+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57a0019040 con 0x555df8f48c70 2023-07-13T22:15:18.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 10 client.25580 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:18.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 10 client.25580 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:15:18.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 20 client.25580 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:18.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8f5cf10 0x555df8f5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 4 client.25580 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:18.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.105+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555df8e91eb0 con 0x555df8f5cf10 2023-07-13T22:15:18.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.123+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8f5cf10 0x555df8f5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.156+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8f5cf10 0x555df8f5f320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=452c9d00533daf8d server_cookie=dc8b5ef491e44489 in_seq=0 out_seq=0 2023-07-13T22:15:18.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.156+0000 7f578e7fc700 10 client.25580 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.157+0000 7f578e7fc700 1 -- 172.21.15.26:0/4223069491 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555df8e91eb0 con 0x555df8f5cf10 2023-07-13T22:15:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.157+0000 7f578e7fc700 10 client.25580 handle_command_reply: tid=0 2023-07-13T22:15:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.159+0000 7f57c0c5bb80 1 client.25580 shutdown 2023-07-13T22:15:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.159+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 msgr2=0x555df8f586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.159+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f586a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b0a0 tx=0x7f57a00074e0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 shutdown_connections 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8f5cf10 0x555df8f5f320 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f586a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f474d0 0x555df8f51120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/4223069491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f47e20 0x555df8f51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 >> 172.21.15.26:0/4223069491 conn(0x555df8f3cda0 msgr2=0x555df8f418b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.160+0000 7f578e7fc700 0 client.25580 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.172+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 shutdown_connections 2023-07-13T22:15:18.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.174+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/4223069491 wait complete. 2023-07-13T22:15:18.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.174+0000 7f57c0c5bb80 20 client.25580 trim_cache size 0 max 16384 2023-07-13T22:15:18.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:17 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/711138958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.207+0000 7f57c0c5bb80 1 Processor -- start 2023-07-13T22:15:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.207+0000 7f57c0c5bb80 1 -- start start 2023-07-13T22:15:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.207+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f474d0 0x555df8f4f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f58660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f50600 con 0x555df8f474d0 2023-07-13T22:15:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f48220 con 0x555df8f48c70 2023-07-13T22:15:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f4ee20 con 0x555df8f47e20 2023-07-13T22:15:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57a77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57a77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48670/0 (socket says 172.21.15.26:48670) 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.208+0000 7f57a77fe700 1 -- 172.21.15.26:0/2123038152 learned_addr learned my addr 172.21.15.26:0/2123038152 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 -- 172.21.15.26:0/2123038152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 msgr2=0x555df8f58660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f58660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 -- 172.21.15.26:0/2123038152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f474d0 msgr2=0x555df8f4f020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f474d0 0x555df8f4f020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 -- 172.21.15.26:0/2123038152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555df8ced250 con 0x555df8f48c70 2023-07-13T22:15:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f57a77fe700 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b070 tx=0x7f57a001dd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef3698841602c04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.209+0000 7f579dffb700 1 -- 172.21.15.26:0/2123038152 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a000e070 con 0x555df8f48c70 2023-07-13T22:15:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.210+0000 7f579dffb700 1 -- 172.21.15.26:0/2123038152 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57a00060e0 con 0x555df8f48c70 2023-07-13T22:15:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.210+0000 7f579dffb700 1 -- 172.21.15.26:0/2123038152 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a001d6d0 con 0x555df8f48c70 2023-07-13T22:15:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.211+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2123038152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 msgr2=0x555df8f51120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.211+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f57a000b070 tx=0x7f57a001dd80 comp rx=0 tx=0).stop 2023-07-13T22:15:18.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:17 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/711138958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.215+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2123038152 shutdown_connections 2023-07-13T22:15:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.215+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f474d0 0x555df8f4f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.215+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f48c70 0x555df8f51120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.215+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/2123038152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f58660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.216+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2123038152 >> 172.21.15.26:0/2123038152 conn(0x555df8f3cda0 msgr2=0x555df8f63c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.216+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2123038152 shutdown_connections 2023-07-13T22:15:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.216+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/2123038152 wait complete. 2023-07-13T22:15:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.218+0000 7f57c0c5bb80 1 -- start start 2023-07-13T22:15:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.218+0000 7f57c0c5bb80 1 Processor -- start 2023-07-13T22:15:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f57fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f54a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f55130 con 0x555df8f48c70 2023-07-13T22:15:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f552a0 con 0x555df8f474d0 2023-07-13T22:15:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.222+0000 7f57c0c5bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555df8f55460 con 0x555df8f47e20 2023-07-13T22:15:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.223+0000 7f57c0c5bb80 10 client.0 _ll_register_callbacks cb 0x7f5790005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.223+0000 7f57a67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.223+0000 7f57a67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42170/0 (socket says 172.21.15.26:42170) 2023-07-13T22:15:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.223+0000 7f57a67fc700 1 -- 172.21.15.26:0/3452062706 learned_addr learned my addr 172.21.15.26:0/3452062706 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.223+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f57fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a77fe700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f54a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a67fc700 1 -- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 msgr2=0x555df8f57fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f57fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a67fc700 1 -- 172.21.15.26:0/3452062706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 msgr2=0x555df8f54a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f54a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.224+0000 7f57a67fc700 1 -- 172.21.15.26:0/3452062706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555df8cafc60 con 0x555df8f47e20 2023-07-13T22:15:18.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.225+0000 7f57a67fc700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f579000c5b0 tx=0x7f5790099930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbfa243c56dc3cff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.225+0000 7f578e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.225+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790099f50 con 0x555df8f47e20 2023-07-13T22:15:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.226+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f579009df80 con 0x555df8f47e20 2023-07-13T22:15:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.226+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5798014470 con 0x555df8f47e20 2023-07-13T22:15:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.226+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57900a23a0 con 0x555df8f47e20 2023-07-13T22:15:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.226+0000 7f57c0c5bb80 10 client.25561 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.226+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555df8cc52b0 con 0x555df8f47e20 2023-07-13T22:15:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.227+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5798014470 con 0x555df8f47e20 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f57900a1f10 con 0x555df8f47e20 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 10 client.25561 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 10 client.25561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 20 client.25561 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8fb22e0 0x555df8fb46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 4 client.25561 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555df8e54050 con 0x555df8fb22e0 2023-07-13T22:15:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.228+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8fb22e0 0x555df8fb46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.229+0000 7f578e7fc700 10 client.25561 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.229+0000 7f57a6ffd700 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8fb22e0 0x555df8fb46f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4cf5f080195eaae server_cookie=614e7a36c93ae34f in_seq=0 out_seq=0 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.229+0000 7f578e7fc700 1 -- 172.21.15.26:0/3452062706 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555df8e54050 con 0x555df8fb22e0 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.229+0000 7f578e7fc700 10 client.25561 handle_command_reply: tid=0 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.229+0000 7f57c0c5bb80 1 client.25561 shutdown 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 msgr2=0x555df8f584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f579000c5b0 tx=0x7f5790099930 comp rx=0 tx=0).stop 2023-07-13T22:15:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 shutdown_connections 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x555df8fb22e0 0x555df8fb46f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555df8f48c70 0x555df8f54a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555df8f474d0 0x555df8f57fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 --2- 172.21.15.26:0/3452062706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555df8f47e20 0x555df8f584f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 >> 172.21.15.26:0/3452062706 conn(0x555df8f3cda0 msgr2=0x555df8f4f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f578e7fc700 0 client.25561 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:18.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 shutdown_connections 2023-07-13T22:15:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.230+0000 7f57c0c5bb80 1 -- 172.21.15.26:0/3452062706 wait complete. 2023-07-13T22:15:18.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.231+0000 7f57c0c5bb80 20 client.25561 trim_cache size 0 max 16384 2023-07-13T22:15:18.221 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 msgr2=0x7f57940a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 0x7f57940a3800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5798019000 tx=0x7f5798013920 comp rx=0 tx=0).stop 2023-07-13T22:15:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 msgr2=0x7f57a81a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f579000e970 tx=0x7f579000ec80 comp rx=0 tx=0).stop 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 shutdown_connections 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f57940a1350 0x7f57940a3800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a810bd60 0x7f57a81abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57a8070ee0 0x7f57a81a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 --2- 172.21.15.26:0/3943188240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57a8071830 0x7f57a81a4d10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.232+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 >> 172.21.15.26:0/3943188240 conn(0x7f57a806ca90 msgr2=0x7f57a806dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.233+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 shutdown_connections 2023-07-13T22:15:18.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.233+0000 7f57ad3ff700 1 -- 172.21.15.26:0/3943188240 wait complete. 2023-07-13T22:15:18.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:17 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/711138958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:18.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:15:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 -- 172.21.15.26:0/2572718864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 msgr2=0x7ff71810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/2572718864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff71810e550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b0a0 tx=0x7ff71000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 -- 172.21.15.26:0/2572718864 shutdown_connections 2023-07-13T22:15:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/2572718864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff71810e550 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/2572718864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718106110 0x7ff718106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/2572718864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718107300 0x7ff718107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 -- 172.21.15.26:0/2572718864 >> 172.21.15.26:0/2572718864 conn(0x7ff718075160 msgr2=0x7ff718077590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.859+0000 7ff71e5ff700 1 -- 172.21.15.26:0/2572718864 shutdown_connections 2023-07-13T22:15:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.860+0000 7ff71e5ff700 1 -- 172.21.15.26:0/2572718864 wait complete. 2023-07-13T22:15:18.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.862+0000 7ff71e5ff700 1 Processor -- start 2023-07-13T22:15:18.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.866+0000 7ff71e5ff700 1 -- start start 2023-07-13T22:15:18.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.866+0000 7ff71e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718106110 0x7ff71806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.866+0000 7ff71e5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.866+0000 7ff71e5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff7181ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.866+0000 7ff71e5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180722e0 con 0x7ff718107c90 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.867+0000 7ff71e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718072420 con 0x7ff718106110 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.867+0000 7ff71e5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180725b0 con 0x7ff718107300 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.867+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.867+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42182/0 (socket says 172.21.15.26:42182) 2023-07-13T22:15:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.867+0000 7ff7177fe700 1 -- 172.21.15.26:0/3422888466 learned_addr learned my addr 172.21.15.26:0/3422888466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff717fff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718106110 0x7ff71806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff71cb9c700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff7181ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff7177fe700 1 -- 172.21.15.26:0/3422888466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718106110 msgr2=0x7ff71806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff7177fe700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718106110 0x7ff71806ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff7177fe700 1 -- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 msgr2=0x7ff7181ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff7177fe700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff7181ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.868+0000 7ff7177fe700 1 -- 172.21.15.26:0/3422888466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009d00 con 0x7ff718107300 2023-07-13T22:15:18.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.869+0000 7ff7177fe700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00c920 tx=0x7ff70c00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.869+0000 7ff7157fa700 1 -- 172.21.15.26:0/3422888466 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c007b00 con 0x7ff718107300 2023-07-13T22:15:18.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.870+0000 7ff7157fa700 1 -- 172.21.15.26:0/3422888466 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70c00ce70 con 0x7ff718107300 2023-07-13T22:15:18.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.870+0000 7ff7157fa700 1 -- 172.21.15.26:0/3422888466 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0054e0 con 0x7ff718107300 2023-07-13T22:15:18.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.870+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff718078eb0 con 0x7ff718107300 2023-07-13T22:15:18.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.870+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff718079370 con 0x7ff718107300 2023-07-13T22:15:18.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.876+0000 7ff7157fa700 1 -- 172.21.15.26:0/3422888466 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff70c00f450 con 0x7ff718107300 2023-07-13T22:15:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.882+0000 7ff7157fa700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 0x7ff7000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.882+0000 7ff7157fa700 1 -- 172.21.15.26:0/3422888466 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff70c0cd3c0 con 0x7ff718107300 2023-07-13T22:15:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.882+0000 7ff717fff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 0x7ff7000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.883+0000 7ff731fefb80 1 Processor -- start 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.883+0000 7ff731fefb80 1 -- start start 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff717fff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 0x7ff7000a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff718107160 tx=0x7ff708009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e84e4cb0 con 0x55c8e85dfe20 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8537a40 con 0x55c8e85df4d0 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff731fefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8530ed0 con 0x55c8e85e0c70 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff71cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff71cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42184/0 (socket says 172.21.15.26:42184) 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff71cb9c700 1 -- 172.21.15.26:0/516792053 learned_addr learned my addr 172.21.15.26:0/516792053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.884+0000 7ff717fff700 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.885+0000 7ff71cb9c700 1 -- 172.21.15.26:0/516792053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 msgr2=0x55c8e85df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.885+0000 7ff71cb9c700 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.885+0000 7ff71cb9c700 1 -- 172.21.15.26:0/516792053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 msgr2=0x55c8e85e06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.885+0000 7ff71cb9c700 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.885+0000 7ff71cb9c700 1 -- 172.21.15.26:0/516792053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e835ede0 con 0x55c8e85e0c70 2023-07-13T22:15:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.886+0000 7ff71cb9c700 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b780 tx=0x7ff710009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f3b2c6a4f10b32 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.886+0000 7ff6fe7fc700 1 -- 172.21.15.26:0/516792053 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000e050 con 0x55c8e85e0c70 2023-07-13T22:15:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.886+0000 7ff6fe7fc700 1 -- 172.21.15.26:0/516792053 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff710004a50 con 0x55c8e85e0c70 2023-07-13T22:15:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.886+0000 7ff6fe7fc700 1 -- 172.21.15.26:0/516792053 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710005470 con 0x55c8e85e0c70 2023-07-13T22:15:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.887+0000 7ff731fefb80 1 -- 172.21.15.26:0/516792053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 msgr2=0x55c8e85e10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.887+0000 7ff731fefb80 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b780 tx=0x7ff710009530 comp rx=0 tx=0).stop 2023-07-13T22:15:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 -- 172.21.15.26:0/516792053 shutdown_connections 2023-07-13T22:15:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e10e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 --2- 172.21.15.26:0/516792053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 -- 172.21.15.26:0/516792053 >> 172.21.15.26:0/516792053 conn(0x55c8e85d4da0 msgr2=0x55c8e85da1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.888+0000 7ff731fefb80 1 -- 172.21.15.26:0/516792053 shutdown_connections 2023-07-13T22:15:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.889+0000 7ff731fefb80 1 -- 172.21.15.26:0/516792053 wait complete. 2023-07-13T22:15:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.889+0000 7ff731fefb80 1 -- start start 2023-07-13T22:15:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.890+0000 7ff731fefb80 1 Processor -- start 2023-07-13T22:15:18.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e84d0ac0 con 0x55c8e85dfe20 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e84d07c0 con 0x55c8e85df4d0 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e83b4920 con 0x55c8e85e0c70 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.892+0000 7ff731fefb80 10 client.0 _ll_register_callbacks cb 0x55c8e84ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48718/0 (socket says 172.21.15.26:48718) 2023-07-13T22:15:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 -- 172.21.15.26:0/1915616621 learned_addr learned my addr 172.21.15.26:0/1915616621 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 -- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 msgr2=0x55c8e85f0710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85f0710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 -- 172.21.15.26:0/1915616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 msgr2=0x55c8e85e96d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.893+0000 7ff717fff700 1 -- 172.21.15.26:0/1915616621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71810ee80 con 0x55c8e85df4d0 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.894+0000 7ff717fff700 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff70801e7c0 tx=0x7ff7080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e78dd3048217b15 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.894+0000 7ff6f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.894+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70800e6c0 con 0x55c8e85df4d0 2023-07-13T22:15:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.894+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff70800e820 con 0x55c8e85df4d0 2023-07-13T22:15:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.894+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8e848a4f0 con 0x55c8e85df4d0 2023-07-13T22:15:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.895+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70800e9e0 con 0x55c8e85df4d0 2023-07-13T22:15:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.895+0000 7ff731fefb80 10 client.25595 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.895+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c8e8347c60 con 0x55c8e85df4d0 2023-07-13T22:15:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.895+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8e848a4f0 con 0x55c8e85df4d0 2023-07-13T22:15:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff70802e070 con 0x55c8e85df4d0 2023-07-13T22:15:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 10 client.25595 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 10 client.25595 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:15:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 20 client.25595 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e85f5030 0x55c8e85f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 4 client.25595 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.896+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8e8529eb0 con 0x55c8e85f5030 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.897+0000 7ff7177fe700 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e85f5030 0x55c8e85f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.897+0000 7ff7177fe700 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e85f5030 0x55c8e85f7440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de889a583a7ca42 server_cookie=9e4c19f8b34c5ae5 in_seq=0 out_seq=0 2023-07-13T22:15:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.897+0000 7ff6f77fe700 10 client.25595 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.898+0000 7ff6f77fe700 1 -- 172.21.15.26:0/1915616621 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c8e8529eb0 con 0x55c8e85f5030 2023-07-13T22:15:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.898+0000 7ff6f77fe700 10 client.25595 handle_command_reply: tid=0 2023-07-13T22:15:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.898+0000 7ff731fefb80 1 client.25595 shutdown 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 msgr2=0x55c8e85e9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff70801e7c0 tx=0x7ff7080047c0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 shutdown_connections 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e85f5030 0x55c8e85f7440 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e9190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1915616621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff6f77fe700 0 client.25595 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 >> 172.21.15.26:0/1915616621 conn(0x55c8e85d4da0 msgr2=0x55c8e85d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.899+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 shutdown_connections 2023-07-13T22:15:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.900+0000 7ff731fefb80 1 -- 172.21.15.26:0/1915616621 wait complete. 2023-07-13T22:15:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.900+0000 7ff731fefb80 20 client.25595 trim_cache size 0 max 16384 2023-07-13T22:15:18.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.928+0000 7ff731fefb80 1 Processor -- start 2023-07-13T22:15:18.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.928+0000 7ff731fefb80 1 -- start start 2023-07-13T22:15:18.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85db0f0 con 0x55c8e85dfe20 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85dbcd0 con 0x55c8e85df4d0 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff731fefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85e0300 con 0x55c8e85e0c70 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff717fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff717fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48734/0 (socket says 172.21.15.26:48734) 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.929+0000 7ff717fff700 1 -- 172.21.15.26:0/1243895379 learned_addr learned my addr 172.21.15.26:0/1243895379 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.930+0000 7ff717fff700 1 -- 172.21.15.26:0/1243895379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 msgr2=0x55c8e85e7e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.930+0000 7ff717fff700 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e7e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.930+0000 7ff717fff700 1 -- 172.21.15.26:0/1243895379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 msgr2=0x55c8e85f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.931+0000 7ff717fff700 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.931+0000 7ff717fff700 1 -- 172.21.15.26:0/1243895379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e8385250 con 0x55c8e85df4d0 2023-07-13T22:15:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.941+0000 7ff717fff700 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff70801c020 tx=0x7ff7080117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962c98b33b0a63f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.942+0000 7ff6feffd700 1 -- 172.21.15.26:0/1243895379 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70801c4e0 con 0x55c8e85df4d0 2023-07-13T22:15:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.942+0000 7ff6feffd700 1 -- 172.21.15.26:0/1243895379 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70801cbe0 con 0x55c8e85df4d0 2023-07-13T22:15:18.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.942+0000 7ff6feffd700 1 -- 172.21.15.26:0/1243895379 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080269e0 con 0x55c8e85df4d0 2023-07-13T22:15:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 -- 172.21.15.26:0/1243895379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 msgr2=0x55c8e85e8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff70801c020 tx=0x7ff7080117d0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 -- 172.21.15.26:0/1243895379 shutdown_connections 2023-07-13T22:15:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85e0c70 0x55c8e85e7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85e8840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 --2- 172.21.15.26:0/1243895379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85dfe20 0x55c8e85f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.943+0000 7ff731fefb80 1 -- 172.21.15.26:0/1243895379 >> 172.21.15.26:0/1243895379 conn(0x55c8e85d4da0 msgr2=0x55c8e85fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.944+0000 7ff731fefb80 1 -- 172.21.15.26:0/1243895379 shutdown_connections 2023-07-13T22:15:18.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.944+0000 7ff731fefb80 1 -- 172.21.15.26:0/1243895379 wait complete. 2023-07-13T22:15:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.945+0000 7ff731fefb80 1 -- start start 2023-07-13T22:15:18.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.945+0000 7ff731fefb80 1 Processor -- start 2023-07-13T22:15:18.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85e0c70 0x55c8e85efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85f00a0 con 0x55c8e85e0c70 2023-07-13T22:15:18.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85f0210 con 0x55c8e85df4d0 2023-07-13T22:15:18.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e85f03d0 con 0x55c8e85dfe20 2023-07-13T22:15:18.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.946+0000 7ff731fefb80 10 client.0 _ll_register_callbacks cb 0x7ff708011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:18.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.947+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.947+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42206/0 (socket says 172.21.15.26:42206) 2023-07-13T22:15:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.947+0000 7ff7177fe700 1 -- 172.21.15.26:0/723639101 learned_addr learned my addr 172.21.15.26:0/723639101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.947+0000 7ff71cb9c700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85e0c70 0x55c8e85efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 -- 172.21.15.26:0/723639101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 msgr2=0x55c8e85ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 -- 172.21.15.26:0/723639101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85e0c70 msgr2=0x55c8e85efb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85e0c70 0x55c8e85efb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 -- 172.21.15.26:0/723639101 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8e8347c60 con 0x55c8e85dfe20 2023-07-13T22:15:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff7177fe700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff70c09a770 tx=0x7ff70c0a2ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687e8848745231e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:18.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.948+0000 7ff6f77fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c098c40 con 0x55c8e85dfe20 2023-07-13T22:15:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff70c0ba070 con 0x55c8e85dfe20 2023-07-13T22:15:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff710004a50 con 0x55c8e85dfe20 2023-07-13T22:15:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70c0ba230 con 0x55c8e85dfe20 2023-07-13T22:15:18.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff731fefb80 10 client.25573 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.949+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c8e835d2b0 con 0x55c8e85dfe20 2023-07-13T22:15:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.950+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70c0c9070 con 0x55c8e85dfe20 2023-07-13T22:15:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.950+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff70c09aa40 con 0x55c8e85dfe20 2023-07-13T22:15:18.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 10 client.25573 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 10 client.25573 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 20 client.25573 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e8649fb0 0x55c8e864c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 4 client.25573 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.951+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8e84ec050 con 0x55c8e8649fb0 2023-07-13T22:15:18.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.952+0000 7ff717fff700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e8649fb0 0x55c8e864c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.952+0000 7ff717fff700 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e8649fb0 0x55c8e864c3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=feb2582e62b12055 server_cookie=4b136714215c5b63 in_seq=0 out_seq=0 2023-07-13T22:15:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.952+0000 7ff6f77fe700 10 client.25573 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.953+0000 7ff6f77fe700 1 -- 172.21.15.26:0/723639101 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8e84ec050 con 0x55c8e8649fb0 2023-07-13T22:15:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.953+0000 7ff6f77fe700 10 client.25573 handle_command_reply: tid=0 2023-07-13T22:15:18.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.953+0000 7ff731fefb80 1 client.25573 shutdown 2023-07-13T22:15:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.955+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 msgr2=0x55c8e85ecb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.955+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff70c09a770 tx=0x7ff70c0a2ce0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 shutdown_connections 2023-07-13T22:15:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c8e8649fb0 0x55c8e864c3c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c8e85e0c70 0x55c8e85efb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff6f77fe700 0 client.25573 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c8e85df4d0 0x55c8e85ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 --2- 172.21.15.26:0/723639101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8e85dfe20 0x55c8e85ecb40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 >> 172.21.15.26:0/723639101 conn(0x55c8e85d4da0 msgr2=0x55c8e85e8580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 shutdown_connections 2023-07-13T22:15:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 1 -- 172.21.15.26:0/723639101 wait complete. 2023-07-13T22:15:18.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.956+0000 7ff731fefb80 20 client.25573 trim_cache size 0 max 16384 2023-07-13T22:15:18.946 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 msgr2=0x7ff7000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 0x7ff7000a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff718107160 tx=0x7ff708009260 comp rx=0 tx=0).stop 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 msgr2=0x7ff71806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00c920 tx=0x7ff70c00cc30 comp rx=0 tx=0).stop 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 shutdown_connections 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff7000a11d0 0x7ff7000a3680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff718107c90 0x7ff7181ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff718106110 0x7ff71806ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 --2- 172.21.15.26:0/3422888466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff718107300 0x7ff71806f200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:18.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.961+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 >> 172.21.15.26:0/3422888466 conn(0x7ff718075160 msgr2=0x7ff7180773b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:18.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.962+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 shutdown_connections 2023-07-13T22:15:18.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:18.962+0000 7ff71e5ff700 1 -- 172.21.15.26:0/3422888466 wait complete. 2023-07-13T22:15:18.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:15:19.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:18 smithi117 ceph-mon[129074]: pgmap v358: 97 pgs: 97 active+clean; 989 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 40 op/s 2023-07-13T22:15:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:18 smithi026 ceph-mon[95647]: pgmap v358: 97 pgs: 97 active+clean; 989 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 40 op/s 2023-07-13T22:15:19.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:18 smithi121 ceph-mon[129187]: pgmap v358: 97 pgs: 97 active+clean; 989 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 82 MiB/s wr, 40 op/s 2023-07-13T22:15:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 -- 172.21.15.26:0/1281270433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 msgr2=0x7f990410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 --2- 172.21.15.26:0/1281270433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 0x7f990410e1f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009990 tx=0x7f98f4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 -- 172.21.15.26:0/1281270433 shutdown_connections 2023-07-13T22:15:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 --2- 172.21.15.26:0/1281270433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 0x7f990410e1f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 --2- 172.21.15.26:0/1281270433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904070ee0 0x7f99040712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 --2- 172.21.15.26:0/1281270433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904071830 0x7f990410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.891+0000 7f990b7ff700 1 -- 172.21.15.26:0/1281270433 >> 172.21.15.26:0/1281270433 conn(0x7f990406ca90 msgr2=0x7f990406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:19.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.892+0000 7f990b7ff700 1 -- 172.21.15.26:0/1281270433 shutdown_connections 2023-07-13T22:15:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.892+0000 7f990b7ff700 1 -- 172.21.15.26:0/1281270433 wait complete. 2023-07-13T22:15:19.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.893+0000 7f990b7ff700 1 Processor -- start 2023-07-13T22:15:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.894+0000 7f990b7ff700 1 -- start start 2023-07-13T22:15:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.894+0000 7f990b7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904070ee0 0x7f99041a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.894+0000 7f990b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.895+0000 7f990b7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 0x7f99041abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.895+0000 7f990b7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99041ac0c0 con 0x7f990410bd60 2023-07-13T22:15:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.895+0000 7f990b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990410ec40 con 0x7f9904071830 2023-07-13T22:15:19.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.895+0000 7f990b7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990410edd0 con 0x7f9904070ee0 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48762/0 (socket says 172.21.15.26:48762) 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 -- 172.21.15.26:0/3940039981 learned_addr learned my addr 172.21.15.26:0/3940039981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.895+0000 7f990959b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904070ee0 0x7f99041a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 -- 172.21.15.26:0/3940039981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904070ee0 msgr2=0x7f99041a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:19.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904070ee0 0x7f99041a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 -- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 msgr2=0x7f99041abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 0x7f99041abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.897+0000 7f9908d9a700 1 -- 172.21.15.26:0/3940039981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40095f0 con 0x7f9904071830 2023-07-13T22:15:19.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.898+0000 7f9908d9a700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98fc007e00 tx=0x7f98fc00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.901+0000 7f98fa7fc700 1 -- 172.21.15.26:0/3940039981 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00ad40 con 0x7f9904071830 2023-07-13T22:15:19.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.901+0000 7f98fa7fc700 1 -- 172.21.15.26:0/3940039981 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98fc007c30 con 0x7f9904071830 2023-07-13T22:15:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.901+0000 7f98fa7fc700 1 -- 172.21.15.26:0/3940039981 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc011700 con 0x7f9904071830 2023-07-13T22:15:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.901+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990410f0b0 con 0x7f9904071830 2023-07-13T22:15:19.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.901+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990410f570 con 0x7f9904071830 2023-07-13T22:15:19.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.910+0000 7f991f1a8b80 1 Processor -- start 2023-07-13T22:15:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 -- start start 2023-07-13T22:15:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bd4d0 0x55a10b2bd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b1c2cb0 con 0x55a10b2bd4d0 2023-07-13T22:15:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b215a40 con 0x55a10b2bec70 2023-07-13T22:15:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.922+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b20eed0 con 0x55a10b2bde20 2023-07-13T22:15:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.923+0000 7f990959b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.923+0000 7f990959b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42224/0 (socket says 172.21.15.26:42224) 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.923+0000 7f990959b700 1 -- 172.21.15.26:0/3300548868 learned_addr learned my addr 172.21.15.26:0/3300548868 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 -- 172.21.15.26:0/3300548868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 msgr2=0x55a10b2bf0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2bf0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 -- 172.21.15.26:0/3300548868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bd4d0 msgr2=0x55a10b2bd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bd4d0 0x55a10b2bd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 -- 172.21.15.26:0/3300548868 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10b03cde0 con 0x55a10b2bde20 2023-07-13T22:15:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f98fa7fc700 1 -- 172.21.15.26:0/3940039981 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f98fc011860 con 0x7f9904071830 2023-07-13T22:15:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.924+0000 7f990959b700 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f990000acb0 tx=0x7f990000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453c932202ce3b72 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.925+0000 7f98ef7fe700 1 -- 172.21.15.26:0/3300548868 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990000a7b0 con 0x55a10b2bde20 2023-07-13T22:15:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.925+0000 7f98ef7fe700 1 -- 172.21.15.26:0/3300548868 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9900018900 con 0x55a10b2bde20 2023-07-13T22:15:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.925+0000 7f98ef7fe700 1 -- 172.21.15.26:0/3300548868 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900010990 con 0x55a10b2bde20 2023-07-13T22:15:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 -- 172.21.15.26:0/3300548868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 msgr2=0x55a10b2be6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f990000acb0 tx=0x7f990000f3c0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 -- 172.21.15.26:0/3300548868 shutdown_connections 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bd4d0 0x55a10b2bd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/3300548868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2be6e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f991f1a8b80 1 -- 172.21.15.26:0/3300548868 >> 172.21.15.26:0/3300548868 conn(0x55a10b2b2da0 msgr2=0x55a10b2b81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f98fa7fc700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 0x7f98f00a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.926+0000 7f98fa7fc700 1 -- 172.21.15.26:0/3940039981 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98fc0ccc30 con 0x7f9904071830 2023-07-13T22:15:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.933+0000 7f991f1a8b80 1 -- 172.21.15.26:0/3300548868 shutdown_connections 2023-07-13T22:15:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.933+0000 7f991f1a8b80 1 -- 172.21.15.26:0/3300548868 wait complete. 2023-07-13T22:15:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.934+0000 7f991f1a8b80 1 -- start start 2023-07-13T22:15:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.934+0000 7f991f1a8b80 1 Processor -- start 2023-07-13T22:15:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.934+0000 7f9909d9c700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 0x7f98f00a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b1aeac0 con 0x55a10b2bec70 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b1ae7c0 con 0x55a10b2bde20 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b092920 con 0x55a10b2bd4d0 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.935+0000 7f991f1a8b80 10 client.0 _ll_register_callbacks cb 0x55a10b1ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48778/0 (socket says 172.21.15.26:48778) 2023-07-13T22:15:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 -- 172.21.15.26:0/1199962824 learned_addr learned my addr 172.21.15.26:0/1199962824 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f990959b700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 -- 172.21.15.26:0/1199962824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 msgr2=0x55a10b2c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 -- 172.21.15.26:0/1199962824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 msgr2=0x55a10b2ce7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2ce7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.936+0000 7f9908d9a700 1 -- 172.21.15.26:0/1199962824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990000a910 con 0x55a10b2bde20 2023-07-13T22:15:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.937+0000 7f9908d9a700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0ca790 tx=0x7f98fc0964b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a27e34b18df331 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.955+0000 7f98ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:19.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.955+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc0a24a0 con 0x55a10b2bde20 2023-07-13T22:15:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.955+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98fc09a2b0 con 0x55a10b2bde20 2023-07-13T22:15:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.955+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a10b1684f0 con 0x55a10b2bde20 2023-07-13T22:15:19.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.955+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98fc09a470 con 0x55a10b2bde20 2023-07-13T22:15:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.970+0000 7f9909d9c700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 0x7f98f00a3780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009960 tx=0x7f98f400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.980+0000 7f991f1a8b80 10 client.25604 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.980+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a10b025c60 con 0x55a10b2bde20 2023-07-13T22:15:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.982+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98fc0c8020 con 0x55a10b2bde20 2023-07-13T22:15:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f98fc0a2010 con 0x55a10b2bde20 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 10 client.25604 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 20 client.25604 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b2d30b0 0x55a10b2d54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 4 client.25604 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.983+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a10b207eb0 con 0x55a10b2d30b0 2023-07-13T22:15:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.984+0000 7f990959b700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b2d30b0 0x55a10b2d54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.984+0000 7f990959b700 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b2d30b0 0x55a10b2d54c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ea0917b84dda7dc server_cookie=848e5bb0aa142608 in_seq=0 out_seq=0 2023-07-13T22:15:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.985+0000 7f98ecff9700 10 client.25604 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.985+0000 7f98ecff9700 1 -- 172.21.15.26:0/1199962824 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a10b207eb0 con 0x55a10b2d30b0 2023-07-13T22:15:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.985+0000 7f98ecff9700 10 client.25604 handle_command_reply: tid=0 2023-07-13T22:15:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.986+0000 7f991f1a8b80 1 client.25604 shutdown 2023-07-13T22:15:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.991+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 msgr2=0x55a10b2c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:19.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:19.991+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0ca790 tx=0x7f98fc0964b0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 shutdown_connections 2023-07-13T22:15:19.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b2d30b0 0x55a10b2d54c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1199962824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bde20 0x55a10b2c7760 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:19.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 >> 172.21.15.26:0/1199962824 conn(0x55a10b2b2da0 msgr2=0x55a10b2b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:19.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.002+0000 7f98ecff9700 0 client.25604 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:19.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.003+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 shutdown_connections 2023-07-13T22:15:19.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.003+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1199962824 wait complete. 2023-07-13T22:15:20.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.019+0000 7f991f1a8b80 20 client.25604 trim_cache size 0 max 16384 2023-07-13T22:15:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.074+0000 7f991f1a8b80 1 Processor -- start 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.085+0000 7f991f1a8b80 1 -- start start 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bde20 0x55a10b2ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2b2970 con 0x55a10b2bde20 2023-07-13T22:15:20.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2c4e50 con 0x55a10b2bec70 2023-07-13T22:15:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.086+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2d2ce0 con 0x55a10b2bd4d0 2023-07-13T22:15:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.087+0000 7f990959b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.087+0000 7f990959b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42256/0 (socket says 172.21.15.26:42256) 2023-07-13T22:15:20.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.087+0000 7f990959b700 1 -- 172.21.15.26:0/1968173320 learned_addr learned my addr 172.21.15.26:0/1968173320 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.087+0000 7f9909d9c700 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.095+0000 7f9908d9a700 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bde20 0x55a10b2ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.096+0000 7f9909d9c700 1 -- 172.21.15.26:0/1968173320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 msgr2=0x55a10b2c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.096+0000 7f9909d9c700 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.096+0000 7f9909d9c700 1 -- 172.21.15.26:0/1968173320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bde20 msgr2=0x55a10b2ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.096+0000 7f9909d9c700 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bde20 0x55a10b2ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.096+0000 7f9909d9c700 1 -- 172.21.15.26:0/1968173320 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a10b063250 con 0x55a10b2bec70 2023-07-13T22:15:20.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.103+0000 7f9909d9c700 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2d50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009350 tx=0x7f98f401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10228c6dd7d93400 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.106+0000 7f98effff700 1 -- 172.21.15.26:0/1968173320 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4028070 con 0x55a10b2bec70 2023-07-13T22:15:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.106+0000 7f98effff700 1 -- 172.21.15.26:0/1968173320 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98f4005510 con 0x55a10b2bec70 2023-07-13T22:15:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.106+0000 7f98effff700 1 -- 172.21.15.26:0/1968173320 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f401d070 con 0x55a10b2bec70 2023-07-13T22:15:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1968173320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 msgr2=0x55a10b2d50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2d50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009350 tx=0x7f98f401d000 comp rx=0 tx=0).stop 2023-07-13T22:15:20.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1968173320 shutdown_connections 2023-07-13T22:15:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bec70 0x55a10b2d50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bd4d0 0x55a10b2c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/1968173320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bde20 0x55a10b2ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.110+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1968173320 >> 172.21.15.26:0/1968173320 conn(0x55a10b2b2da0 msgr2=0x55a10b2d9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.115+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1968173320 shutdown_connections 2023-07-13T22:15:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.120+0000 7f991f1a8b80 1 -- 172.21.15.26:0/1968173320 wait complete. 2023-07-13T22:15:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.121+0000 7f991f1a8b80 1 -- start start 2023-07-13T22:15:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.121+0000 7f991f1a8b80 1 Processor -- start 2023-07-13T22:15:20.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bd4d0 0x55a10b2ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2ce1a0 con 0x55a10b2bec70 2023-07-13T22:15:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2ce310 con 0x55a10b2bd4d0 2023-07-13T22:15:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a10b2ce4d0 con 0x55a10b2bde20 2023-07-13T22:15:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.122+0000 7f991f1a8b80 10 client.0 _ll_register_callbacks cb 0x7f98fc099b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42266/0 (socket says 172.21.15.26:42266) 2023-07-13T22:15:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 -- 172.21.15.26:0/2754383346 learned_addr learned my addr 172.21.15.26:0/2754383346 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f990959b700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bd4d0 0x55a10b2ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 -- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bd4d0 msgr2=0x55a10b2ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bd4d0 0x55a10b2ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 -- 172.21.15.26:0/2754383346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 msgr2=0x55a10b2cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.123+0000 7f9908d9a700 1 -- 172.21.15.26:0/2754383346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9904117640 con 0x55a10b2bde20 2023-07-13T22:15:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.124+0000 7f9908d9a700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98fc098750 tx=0x7f98fc0a24a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c6de5e7948547c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.124+0000 7f98ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.124+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc0a2190 con 0x55a10b2bde20 2023-07-13T22:15:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.124+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9900018900 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.124+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f98fc0c9080 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98fc099750 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 10 client.25588 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a10b03b2b0 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9900018900 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f98fc0c4070 con 0x55a10b2bde20 2023-07-13T22:15:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 10 client.25588 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 10 client.25588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 20 client.25588 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b327e20 0x55a10b32a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 4 client.25588 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.125+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a10b1ca050 con 0x55a10b327e20 2023-07-13T22:15:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.126+0000 7f990959b700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b327e20 0x55a10b32a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.126+0000 7f990959b700 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b327e20 0x55a10b32a230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bea40ea70ef2192 server_cookie=341da6e13cf85a71 in_seq=0 out_seq=0 2023-07-13T22:15:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.126+0000 7f98ecff9700 10 client.25588 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.127+0000 7f98ecff9700 1 -- 172.21.15.26:0/2754383346 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a10b1ca050 con 0x55a10b327e20 2023-07-13T22:15:20.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.127+0000 7f98ecff9700 10 client.25588 handle_command_reply: tid=0 2023-07-13T22:15:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.127+0000 7f991f1a8b80 1 client.25588 shutdown 2023-07-13T22:15:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 msgr2=0x55a10b2cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98fc098750 tx=0x7f98fc0a24a0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 shutdown_connections 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a10b2bec70 0x55a10b2cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a10b327e20 0x55a10b32a230 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a10b2bd4d0 0x55a10b2ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 --2- 172.21.15.26:0/2754383346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a10b2bde20 0x55a10b2cac40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 >> 172.21.15.26:0/2754383346 conn(0x55a10b2b2da0 msgr2=0x55a10b2c50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.129+0000 7f98ecff9700 0 client.25588 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.130+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 shutdown_connections 2023-07-13T22:15:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.130+0000 7f991f1a8b80 1 -- 172.21.15.26:0/2754383346 wait complete. 2023-07-13T22:15:20.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.130+0000 7f991f1a8b80 20 client.25588 trim_cache size 0 max 16384 2023-07-13T22:15:20.120 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.131+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 msgr2=0x7f98f00a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.131+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 0x7f98f00a3780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009960 tx=0x7f98f400b540 comp rx=0 tx=0).stop 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.131+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 msgr2=0x7f99041a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.131+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f98fc007e00 tx=0x7f98fc00a340 comp rx=0 tx=0).stop 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.131+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 shutdown_connections 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f98f00a12d0 0x7f98f00a3780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f990410bd60 0x7f99041abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9904070ee0 0x7f99041a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 --2- 172.21.15.26:0/3940039981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9904071830 0x7f99041a4a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 >> 172.21.15.26:0/3940039981 conn(0x7f990406ca90 msgr2=0x7f990406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 shutdown_connections 2023-07-13T22:15:20.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.132+0000 7f990b7ff700 1 -- 172.21.15.26:0/3940039981 wait complete. 2023-07-13T22:15:20.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:15:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 -- 172.21.15.26:0/3669042619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 msgr2=0x7f1a7c079690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 --2- 172.21.15.26:0/3669042619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c079690 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400b4f0 tx=0x7f1a7400b800 comp rx=0 tx=0).stop 2023-07-13T22:15:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 -- 172.21.15.26:0/3669042619 shutdown_connections 2023-07-13T22:15:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 --2- 172.21.15.26:0/3669042619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c079690 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 --2- 172.21.15.26:0/3669042619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c071200 0x7f1a7c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 --2- 172.21.15.26:0/3669042619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c074850 0x7f1a7c076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 -- 172.21.15.26:0/3669042619 >> 172.21.15.26:0/3669042619 conn(0x7f1a7c06ca90 msgr2=0x7f1a7c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 -- 172.21.15.26:0/3669042619 shutdown_connections 2023-07-13T22:15:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.710+0000 7f1a83fff700 1 -- 172.21.15.26:0/3669042619 wait complete. 2023-07-13T22:15:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.712+0000 7f1a83fff700 1 Processor -- start 2023-07-13T22:15:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.712+0000 7f1a83fff700 1 -- start start 2023-07-13T22:15:20.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 0x7f1a7c1af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 0x7f1a7c1af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c1b6c90 con 0x7f1a7c077210 2023-07-13T22:15:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c1b6dd0 con 0x7f1a7c071200 2023-07-13T22:15:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.713+0000 7f1a83fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7c1b6f60 con 0x7f1a7c074850 2023-07-13T22:15:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.714+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.714+0000 7f1a8259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47664/0 (socket says 172.21.15.26:47664) 2023-07-13T22:15:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.714+0000 7f1a8259c700 1 -- 172.21.15.26:0/2072633107 learned_addr learned my addr 172.21.15.26:0/2072633107 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.714+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 0x7f1a7c1af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8259c700 1 -- 172.21.15.26:0/2072633107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 msgr2=0x7f1a7c1af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8159a700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 0x7f1a7c1af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8259c700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 0x7f1a7c1af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8259c700 1 -- 172.21.15.26:0/2072633107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 msgr2=0x7f1a7c1af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8259c700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 0x7f1a7c1af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8259c700 1 -- 172.21.15.26:0/2072633107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a74009d00 con 0x7f1a7c077210 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 0x7f1a7c1af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.715+0000 7f1a8159a700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 0x7f1a7c1af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:15:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.716+0000 7f1a8259c700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007e10 tx=0x7f1a74004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.716+0000 7f1a72ffd700 1 -- 172.21.15.26:0/2072633107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7400e040 con 0x7f1a7c077210 2023-07-13T22:15:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.716+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7c1b71e0 con 0x7f1a7c077210 2023-07-13T22:15:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.716+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7c1b76b0 con 0x7f1a7c077210 2023-07-13T22:15:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.717+0000 7f1a72ffd700 1 -- 172.21.15.26:0/2072633107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a74009b80 con 0x7f1a7c077210 2023-07-13T22:15:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.718+0000 7f1a72ffd700 1 -- 172.21.15.26:0/2072633107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401b450 con 0x7f1a7c077210 2023-07-13T22:15:20.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.722+0000 7f1a72ffd700 1 -- 172.21.15.26:0/2072633107 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a7401b5b0 con 0x7f1a7c077210 2023-07-13T22:15:20.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.723+0000 7f1a9799eb80 1 Processor -- start 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 -- start start 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f6d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f77480 0x557270f778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e72cb0 con 0x557270f77480 2023-07-13T22:15:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270ec5a40 con 0x557270f6d4d0 2023-07-13T22:15:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.725+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270ebeed0 con 0x557270f6de20 2023-07-13T22:15:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.726+0000 7f1a72ffd700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 0x7f1a680a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.726+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f6d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.726+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f6d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48808/0 (socket says 172.21.15.26:48808) 2023-07-13T22:15:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.726+0000 7f1a8159a700 1 -- 172.21.15.26:0/1490532 learned_addr learned my addr 172.21.15.26:0/1490532 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.726+0000 7f1a72ffd700 1 -- 172.21.15.26:0/2072633107 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a740cdbc0 con 0x7f1a7c077210 2023-07-13T22:15:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 -- 172.21.15.26:0/1490532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 msgr2=0x557270f6d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f6d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 -- 172.21.15.26:0/1490532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f77480 msgr2=0x557270f778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f77480 0x557270f778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.727+0000 7f1a81d9b700 1 -- 172.21.15.26:0/1490532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270cecde0 con 0x557270f6de20 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.728+0000 7f1a8259c700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 0x7f1a680a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.728+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f76ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0114f0 tx=0x7f1a6c011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb13948cf8cb44d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.729+0000 7f1a8259c700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 0x7f1a680a3700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1a7409acf0 tx=0x7f1a740099e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.729+0000 7f1a67fff700 1 -- 172.21.15.26:0/1490532 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c004230 con 0x557270f6de20 2023-07-13T22:15:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.729+0000 7f1a67fff700 1 -- 172.21.15.26:0/1490532 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c004390 con 0x557270f6de20 2023-07-13T22:15:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.729+0000 7f1a67fff700 1 -- 172.21.15.26:0/1490532 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00e8b0 con 0x557270f6de20 2023-07-13T22:15:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.730+0000 7f1a9799eb80 1 -- 172.21.15.26:0/1490532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 msgr2=0x557270f76ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.730+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f76ef0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0114f0 tx=0x7f1a6c011800 comp rx=0 tx=0).stop 2023-07-13T22:15:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 -- 172.21.15.26:0/1490532 shutdown_connections 2023-07-13T22:15:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f77480 0x557270f778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f6d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/1490532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f76ef0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 -- 172.21.15.26:0/1490532 >> 172.21.15.26:0/1490532 conn(0x557270f62da0 msgr2=0x557270f681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.740+0000 7f1a9799eb80 1 -- 172.21.15.26:0/1490532 shutdown_connections 2023-07-13T22:15:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.741+0000 7f1a9799eb80 1 -- 172.21.15.26:0/1490532 wait complete. 2023-07-13T22:15:20.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.741+0000 7f1a9799eb80 1 -- start start 2023-07-13T22:15:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.742+0000 7f1a9799eb80 1 Processor -- start 2023-07-13T22:15:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 0x557270f7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 0x557270f7ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f77480 0x557270f86f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e5eac0 con 0x557270f6d4d0 2023-07-13T22:15:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270e5e7c0 con 0x557270f6de20 2023-07-13T22:15:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270d42920 con 0x557270f77480 2023-07-13T22:15:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.743+0000 7f1a9799eb80 10 client.0 _ll_register_callbacks cb 0x557270e5c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.746+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 0x557270f7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.746+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 0x557270f7f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47682/0 (socket says 172.21.15.26:47682) 2023-07-13T22:15:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.746+0000 7f1a8159a700 1 -- 172.21.15.26:0/2863943250 learned_addr learned my addr 172.21.15.26:0/2863943250 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.746+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 0x557270f7ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.746+0000 7f1a8159a700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f77480 0x557270f86f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.747+0000 7f1a81d9b700 1 -- 172.21.15.26:0/2863943250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f77480 msgr2=0x557270f86f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.747+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f77480 0x557270f86f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.747+0000 7f1a81d9b700 1 -- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 msgr2=0x557270f7f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.747+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 0x557270f7f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.747+0000 7f1a81d9b700 1 -- 172.21.15.26:0/2863943250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c009c90 con 0x557270f6de20 2023-07-13T22:15:20.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.748+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 0x557270f7ff30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c000c00 tx=0x7f1a6c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ac19b5c4e78a1ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.750+0000 7f1a657fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.750+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c003680 con 0x557270f6de20 2023-07-13T22:15:20.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.750+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a6c0037e0 con 0x557270f6de20 2023-07-13T22:15:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.752+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557270e184f0 con 0x557270f6de20 2023-07-13T22:15:20.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.752+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a6c0039a0 con 0x557270f6de20 2023-07-13T22:15:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.752+0000 7f1a9799eb80 10 client.25622 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:20.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.752+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557270cd5c60 con 0x557270f6de20 2023-07-13T22:15:20.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.753+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557270e184f0 con 0x557270f6de20 2023-07-13T22:15:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.753+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a6c021040 con 0x557270f6de20 2023-07-13T22:15:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.754+0000 7f1a9799eb80 10 client.25622 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.754+0000 7f1a9799eb80 10 client.25622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:15:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.755+0000 7f1a9799eb80 20 client.25622 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.755+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f8b6c0 0x557270f8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.755+0000 7f1a9799eb80 4 client.25622 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.755+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557270eb7eb0 con 0x557270f8b6c0 2023-07-13T22:15:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.755+0000 7f1a8159a700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f8b6c0 0x557270f8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.757+0000 7f1a8159a700 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f8b6c0 0x557270f8dad0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c239ba622f135e0b server_cookie=a262d93205ba1236 in_seq=0 out_seq=0 2023-07-13T22:15:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.757+0000 7f1a657fa700 10 client.25622 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:15:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.758+0000 7f1a657fa700 1 -- 172.21.15.26:0/2863943250 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557270eb7eb0 con 0x557270f8b6c0 2023-07-13T22:15:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.758+0000 7f1a657fa700 10 client.25622 handle_command_reply: tid=0 2023-07-13T22:15:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.758+0000 7f1a9799eb80 1 client.25622 shutdown 2023-07-13T22:15:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.758+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 msgr2=0x557270f7ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.758+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 0x557270f7ff30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c000c00 tx=0x7f1a6c005270 comp rx=0 tx=0).stop 2023-07-13T22:15:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 shutdown_connections 2023-07-13T22:15:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f8b6c0 0x557270f8dad0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6d4d0 0x557270f7f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f77480 0x557270f86f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/2863943250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6de20 0x557270f7ff30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 >> 172.21.15.26:0/2863943250 conn(0x557270f62da0 msgr2=0x557270f678b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.762+0000 7f1a657fa700 0 client.25622 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:15:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.766+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 shutdown_connections 2023-07-13T22:15:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.766+0000 7f1a9799eb80 1 -- 172.21.15.26:0/2863943250 wait complete. 2023-07-13T22:15:20.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.774+0000 7f1a9799eb80 20 client.25622 trim_cache size 0 max 16384 2023-07-13T22:15:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 Processor -- start 2023-07-13T22:15:20.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 -- start start 2023-07-13T22:15:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f62da0 0x557270f7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f86e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f6e480 con 0x557270f62da0 2023-07-13T22:15:20.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f80be0 con 0x557270f6d4d0 2023-07-13T22:15:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.805+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f7ed70 con 0x557270f6de20 2023-07-13T22:15:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.806+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.806+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48830/0 (socket says 172.21.15.26:48830) 2023-07-13T22:15:20.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.806+0000 7f1a81d9b700 1 -- 172.21.15.26:0/3454200332 learned_addr learned my addr 172.21.15.26:0/3454200332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 -- 172.21.15.26:0/3454200332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 msgr2=0x557270f86e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f86e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 -- 172.21.15.26:0/3454200332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f62da0 msgr2=0x557270f7a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f62da0 0x557270f7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 -- 172.21.15.26:0/3454200332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270d13250 con 0x557270f6d4d0 2023-07-13T22:15:20.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0355f0 tx=0x7f1a6c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a6110328b5c9ac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a70ff9700 1 -- 172.21.15.26:0/3454200332 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c009260 con 0x557270f6d4d0 2023-07-13T22:15:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a70ff9700 1 -- 172.21.15.26:0/3454200332 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c00e470 con 0x557270f6d4d0 2023-07-13T22:15:20.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.807+0000 7f1a70ff9700 1 -- 172.21.15.26:0/3454200332 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00eea0 con 0x557270f6d4d0 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3454200332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 msgr2=0x557270f7a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0355f0 tx=0x7f1a6c001e50 comp rx=0 tx=0).stop 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3454200332 shutdown_connections 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f62da0 0x557270f7a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6de20 0x557270f86e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3454200332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f6d4d0 0x557270f7a900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.808+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3454200332 >> 172.21.15.26:0/3454200332 conn(0x557270f64600 msgr2=0x557270f7d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.809+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3454200332 shutdown_connections 2023-07-13T22:15:20.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.809+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3454200332 wait complete. 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.809+0000 7f1a9799eb80 1 -- start start 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 Processor -- start 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 0x557270f833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6de20 0x557270f863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f86900 con 0x557270f6de20 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f86a70 con 0x557270f62da0 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270f86c30 con 0x557270f6d4d0 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.810+0000 7f1a9799eb80 10 client.0 _ll_register_callbacks cb 0x7f1a78009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a81d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 0x557270f833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48834/0 (socket says 172.21.15.26:48834) 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 -- 172.21.15.26:0/3264679322 learned_addr learned my addr 172.21.15.26:0/3264679322 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:20.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 -- 172.21.15.26:0/3264679322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 msgr2=0x557270f833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 0x557270f833a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 -- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6de20 msgr2=0x557270f863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6de20 0x557270f863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a8159a700 1 -- 172.21.15.26:0/3264679322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557270cd5c60 con 0x557270f62da0 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.811+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 0x557270f833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.812+0000 7f1a8159a700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x557270f8e3b0 tx=0x7f1a78014bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c65dea2f67a19d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:20.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.812+0000 7f1a657fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:20.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.812+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78019070 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.812+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a78014110 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.813+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6c004390 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.813+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a780142d0 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.813+0000 7f1a9799eb80 10 client.25628 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.813+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557270ceb2b0 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.813+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c004390 con 0x557270f62da0 2023-07-13T22:15:20.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1a78012b80 con 0x557270f62da0 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 10 client.25628 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 10 client.25628 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 20 client.25628 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f77480 0x557270fe2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 4 client.25628 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.814+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557270e7a050 con 0x557270f77480 2023-07-13T22:15:20.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.815+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f77480 0x557270fe2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:20.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.816+0000 7f1a81d9b700 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f77480 0x557270fe2740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=438c04acacdeeb3f server_cookie=10078e497fff0fd7 in_seq=0 out_seq=0 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.816+0000 7f1a657fa700 10 client.25628 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.816+0000 7f1a657fa700 1 -- 172.21.15.26:0/3264679322 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557270e7a050 con 0x557270f77480 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.816+0000 7f1a657fa700 10 client.25628 handle_command_reply: tid=0 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.816+0000 7f1a9799eb80 1 client.25628 shutdown 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 msgr2=0x557270f82e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x557270f8e3b0 tx=0x7f1a78014bf0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 shutdown_connections 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x557270f77480 0x557270fe2740 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557270f6de20 0x557270f863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557270f62da0 0x557270f82e60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 --2- 172.21.15.26:0/3264679322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557270f6d4d0 0x557270f833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 >> 172.21.15.26:0/3264679322 conn(0x557270f64600 msgr2=0x557270f7a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a657fa700 0 client.25628 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:15:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 shutdown_connections 2023-07-13T22:15:20.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.817+0000 7f1a9799eb80 1 -- 172.21.15.26:0/3264679322 wait complete. 2023-07-13T22:15:20.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.818+0000 7f1a9799eb80 20 client.25628 trim_cache size 0 max 16384 2023-07-13T22:15:20.809 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 msgr2=0x7f1a680a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 0x7f1a680a3700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1a7409acf0 tx=0x7f1a740099e0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 msgr2=0x7f1a7c1b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007e10 tx=0x7f1a74004c80 comp rx=0 tx=0).stop 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 shutdown_connections 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1a680a1250 0x7f1a680a3700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a7c077210 0x7f1a7c1b66c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a7c071200 0x7f1a7c1af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 --2- 172.21.15.26:0/2072633107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a7c074850 0x7f1a7c1af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.823+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 >> 172.21.15.26:0/2072633107 conn(0x7f1a7c06ca90 msgr2=0x7f1a7c0760e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.825+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 shutdown_connections 2023-07-13T22:15:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:20.825+0000 7f1a83fff700 1 -- 172.21.15.26:0/2072633107 wait complete. 2023-07-13T22:15:20.845 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:20 smithi026 ceph-mon[95647]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 125 MiB/s wr, 49 op/s 2023-07-13T22:15:20.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:15:20.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:20 smithi117 ceph-mon[129074]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 125 MiB/s wr, 49 op/s 2023-07-13T22:15:20.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:20 smithi121 ceph-mon[129187]: pgmap v359: 97 pgs: 97 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 125 MiB/s wr, 49 op/s 2023-07-13T22:15:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.267+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/401038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8077210 msgr2=0x7f8fc8079690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.267+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/401038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8077210 0x7f8fc8079690 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000b0a0 tx=0x7f8fc000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.282+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/401038220 shutdown_connections 2023-07-13T22:15:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.282+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/401038220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8077210 0x7f8fc8079690 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.282+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/401038220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.282+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/401038220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8074850 0x7f8fc8076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.282+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/401038220 >> 172.21.15.26:0/401038220 conn(0x7f8fc806ca90 msgr2=0x7f8fc806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.301+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/401038220 shutdown_connections 2023-07-13T22:15:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.312+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/401038220 wait complete. 2023-07-13T22:15:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.313+0000 7f8fcd3ff700 1 Processor -- start 2023-07-13T22:15:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 -- start start 2023-07-13T22:15:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8082480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8074850 0x7f8fc80829c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 0x7f8fc81bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc81bd2f0 con 0x7f8fc8074850 2023-07-13T22:15:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc8129520 con 0x7f8fc8071200 2023-07-13T22:15:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.314+0000 7f8fcd3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc81296e0 con 0x7f8fc8077210 2023-07-13T22:15:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.315+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 0x7f8fc81bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.315+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 0x7f8fc81bcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42348/0 (socket says 172.21.15.26:42348) 2023-07-13T22:15:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.315+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3607809673 learned_addr learned my addr 172.21.15.26:0/3607809673 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.315+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8082480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.317+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8074850 0x7f8fc80829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.324+0000 7f8fc6ffd700 1 -- 172.21.15.26:0/3607809673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 msgr2=0x7f8fc81bcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.324+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 0x7f8fc81bcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.324+0000 7f8fc6ffd700 1 -- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8074850 msgr2=0x7f8fc80829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.324+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8074850 0x7f8fc80829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.324+0000 7f8fc6ffd700 1 -- 172.21.15.26:0/3607809673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009d00 con 0x7f8fc8071200 2023-07-13T22:15:21.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.330+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8082480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.330+0000 7f8fb7fff700 1 -- 172.21.15.26:0/3607809673 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8011830 con 0x7f8fc8071200 2023-07-13T22:15:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.330+0000 7f8fb7fff700 1 -- 172.21.15.26:0/3607809673 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fb8011e50 con 0x7f8fc8071200 2023-07-13T22:15:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.330+0000 7f8fb7fff700 1 -- 172.21.15.26:0/3607809673 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8010d50 con 0x7f8fc8071200 2023-07-13T22:15:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.330+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc81299c0 con 0x7f8fc8071200 2023-07-13T22:15:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.331+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc8129f00 con 0x7f8fc8071200 2023-07-13T22:15:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.339+0000 7f8fe0d82b80 1 Processor -- start 2023-07-13T22:15:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.339+0000 7f8fe0d82b80 1 -- start start 2023-07-13T22:15:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c1652758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c1652766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 0x55c1652770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16517acb0 con 0x55c165276c70 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1651cda40 con 0x55c165275e20 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.340+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1651c6ed0 con 0x55c1652754d0 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.350+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c1652758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.350+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c1652758e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42352/0 (socket says 172.21.15.26:42352) 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.350+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3846667803 learned_addr learned my addr 172.21.15.26:0/3846667803 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:21.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.366+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 0x55c1652770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c1652766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 -- 172.21.15.26:0/3846667803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 msgr2=0x55c1652758e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c1652758e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 -- 172.21.15.26:0/3846667803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 msgr2=0x55c1652766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c1652766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 -- 172.21.15.26:0/3846667803 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c164ff4de0 con 0x55c165276c70 2023-07-13T22:15:21.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.368+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 0x55c1652770e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b7b0 tx=0x7f8fbc00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cbdeddb7fb817f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.373+0000 7f8fb7fff700 1 -- 172.21.15.26:0/3607809673 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8fb8011990 con 0x7f8fc8071200 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.376+0000 7f8fb7fff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 0x7f8fb00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.377+0000 7f8fb7fff700 1 -- 172.21.15.26:0/3607809673 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fb80cc4e0 con 0x7f8fc8071200 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.378+0000 7f8fb57fa700 1 -- 172.21.15.26:0/3846667803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0125f0 con 0x55c165276c70 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.378+0000 7f8fb57fa700 1 -- 172.21.15.26:0/3846667803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fbc012c10 con 0x55c165276c70 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.378+0000 7f8fb57fa700 1 -- 172.21.15.26:0/3846667803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc013d50 con 0x55c165276c70 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.379+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 0x7f8fb00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3846667803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 msgr2=0x55c1652770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 0x55c1652770e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b7b0 tx=0x7f8fbc00bac0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 0x7f8fb00a3800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005bd0 tx=0x7f8fc0005a80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3846667803 shutdown_connections 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165276c70 0x55c1652770e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c1652758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3846667803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c1652766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.380+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3846667803 >> 172.21.15.26:0/3846667803 conn(0x55c16526ada0 msgr2=0x55c1652701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.382+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3846667803 shutdown_connections 2023-07-13T22:15:21.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.383+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3846667803 wait complete. 2023-07-13T22:15:21.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.386+0000 7f8fe0d82b80 1 -- start start 2023-07-13T22:15:21.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.387+0000 7f8fe0d82b80 1 Processor -- start 2023-07-13T22:15:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c16527f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 0x55c16527f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165166ac0 con 0x55c165275e20 2023-07-13T22:15:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1651667c0 con 0x55c165276c70 2023-07-13T22:15:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16504a920 con 0x55c1652754d0 2023-07-13T22:15:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.388+0000 7f8fe0d82b80 10 client.0 _ll_register_callbacks cb 0x55c165164760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:21.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.389+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.389+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48866/0 (socket says 172.21.15.26:48866) 2023-07-13T22:15:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.389+0000 7f8fc77fe700 1 -- 172.21.15.26:0/2998444748 learned_addr learned my addr 172.21.15.26:0/2998444748 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 0x55c16527f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c16527f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc77fe700 1 -- 172.21.15.26:0/2998444748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 msgr2=0x55c16527f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c16527f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc77fe700 1 -- 172.21.15.26:0/2998444748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 msgr2=0x55c16527f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 0x55c16527f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc77fe700 1 -- 172.21.15.26:0/2998444748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc00e040 con 0x55c165276c70 2023-07-13T22:15:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.390+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 0x55c16527f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:15:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.392+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0004010 tx=0x7f8fc00099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611a029ce0dc8295 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.392+0000 7f8fae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.392+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000e040 con 0x55c165276c70 2023-07-13T22:15:21.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.392+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fc0021070 con 0x55c165276c70 2023-07-13T22:15:21.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.393+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1651204f0 con 0x55c165276c70 2023-07-13T22:15:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.395+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc0009320 con 0x55c165276c70 2023-07-13T22:15:21.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.395+0000 7f8fe0d82b80 10 client.25634 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.395+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c164fddc60 con 0x55c165276c70 2023-07-13T22:15:21.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c1651204f0 con 0x55c165276c70 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8fc001b400 con 0x55c165276c70 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 10 client.25634 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 20 client.25634 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c16528af70 0x55c16528d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 4 client.25634 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:21.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.396+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1651bfeb0 con 0x55c16528af70 2023-07-13T22:15:21.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.397+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c16528af70 0x55c16528d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.398+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c16528af70 0x55c16528d380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3c01d5324b988a server_cookie=b4f840fcd90706f4 in_seq=0 out_seq=0 2023-07-13T22:15:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.398+0000 7f8fae7fc700 10 client.25634 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.399+0000 7f8fae7fc700 1 -- 172.21.15.26:0/2998444748 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c1651bfeb0 con 0x55c16528af70 2023-07-13T22:15:21.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.399+0000 7f8fae7fc700 10 client.25634 handle_command_reply: tid=0 2023-07-13T22:15:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.399+0000 7f8fe0d82b80 1 client.25634 shutdown 2023-07-13T22:15:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.399+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 msgr2=0x55c165286710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.399+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0004010 tx=0x7f8fc00099e0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 shutdown_connections 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c16528af70 0x55c16528d380 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165276c70 0x55c165286710 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c16527f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/2998444748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c165275e20 0x55c16527f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 >> 172.21.15.26:0/2998444748 conn(0x55c16526ada0 msgr2=0x55c16526f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fae7fc700 0 client.25634 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 shutdown_connections 2023-07-13T22:15:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.400+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/2998444748 wait complete. 2023-07-13T22:15:21.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.401+0000 7f8fe0d82b80 20 client.25634 trim_cache size 0 max 16384 2023-07-13T22:15:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.438+0000 7f8fe0d82b80 1 Processor -- start 2023-07-13T22:15:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.438+0000 7f8fe0d82b80 1 -- start start 2023-07-13T22:15:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c16527cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c165286630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16526a9c0 con 0x55c16526ada0 2023-07-13T22:15:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c16527ed00 con 0x55c165275e20 2023-07-13T22:15:21.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.439+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165267ef0 con 0x55c1652754d0 2023-07-13T22:15:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc67fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c165286630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48880/0 (socket says 172.21.15.26:48880) 2023-07-13T22:15:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3355368857 learned_addr learned my addr 172.21.15.26:0/3355368857 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:21.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c16527cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3355368857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 msgr2=0x55c165286630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c165286630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3355368857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 msgr2=0x55c16527cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c16527cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.440+0000 7f8fc77fe700 1 -- 172.21.15.26:0/3355368857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c16501b250 con 0x55c165275e20 2023-07-13T22:15:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.441+0000 7f8fc77fe700 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000a010 tx=0x7f8fc00050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7be0d793de9be5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.441+0000 7f8fb5ffb700 1 -- 172.21.15.26:0/3355368857 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc003d070 con 0x55c165275e20 2023-07-13T22:15:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.441+0000 7f8fb5ffb700 1 -- 172.21.15.26:0/3355368857 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc001f070 con 0x55c165275e20 2023-07-13T22:15:21.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.441+0000 7f8fb5ffb700 1 -- 172.21.15.26:0/3355368857 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc000e040 con 0x55c165275e20 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3355368857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 msgr2=0x55c165279af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000a010 tx=0x7f8fc00050d0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3355368857 shutdown_connections 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c16527cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c165275e20 0x55c165279af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/3355368857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c1652754d0 0x55c165286630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3355368857 >> 172.21.15.26:0/3355368857 conn(0x55c16526c600 msgr2=0x55c16528cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3355368857 shutdown_connections 2023-07-13T22:15:21.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.442+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/3355368857 wait complete. 2023-07-13T22:15:21.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.443+0000 7f8fe0d82b80 1 -- start start 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.443+0000 7f8fe0d82b80 1 Processor -- start 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c165281e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 0x55c165282350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 0x55c165285370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1652858b0 con 0x55c16526ada0 2023-07-13T22:15:21.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165285a20 con 0x55c1652754d0 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c165285be0 con 0x55c165275e20 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.444+0000 7f8fe0d82b80 10 client.0 _ll_register_callbacks cb 0x7f8fc0003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 0x55c165285370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 0x55c165285370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42386/0 (socket says 172.21.15.26:42386) 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc77fe700 1 -- 172.21.15.26:0/4267969393 learned_addr learned my addr 172.21.15.26:0/4267969393 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c165281e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 0x55c165282350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 -- 172.21.15.26:0/4267969393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 msgr2=0x55c165285370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 0x55c165285370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 -- 172.21.15.26:0/4267969393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 msgr2=0x55c165281e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:21.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c165281e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.445+0000 7f8fc67fc700 1 -- 172.21.15.26:0/4267969393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c164fddc60 con 0x55c1652754d0 2023-07-13T22:15:21.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.446+0000 7f8fc67fc700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 0x55c165282350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006e20 tx=0x7f8fbc012ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b214186bc2511713 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:21.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.446+0000 7f8fae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.446+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc021070 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.446+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8fbc004390 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.446+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fbc012c10 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.447+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fbc004550 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.447+0000 7f8fe0d82b80 10 client.25640 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.447+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c164ff32b0 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.447+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fbc034020 con 0x55c1652754d0 2023-07-13T22:15:21.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8fbc0137e0 con 0x55c1652754d0 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 20 client.25640 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c165276c70 0x55c1652e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 4 client.25640 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c165182050 con 0x55c165276c70 2023-07-13T22:15:21.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.448+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c165276c70 0x55c1652e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:21.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fae7fc700 10 client.25640 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:21.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fc6ffd700 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c165276c70 0x55c1652e16e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3734f94a10e1a951 server_cookie=9290a5e362b5c025 in_seq=0 out_seq=0 2023-07-13T22:15:21.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fae7fc700 1 -- 172.21.15.26:0/4267969393 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c165182050 con 0x55c165276c70 2023-07-13T22:15:21.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fae7fc700 10 client.25640 handle_command_reply: tid=0 2023-07-13T22:15:21.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fe0d82b80 1 client.25640 shutdown 2023-07-13T22:15:21.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 msgr2=0x55c165282350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.449+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 0x55c165282350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006e20 tx=0x7f8fbc012ba0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 shutdown_connections 2023-07-13T22:15:21.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c16526ada0 0x55c165281e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c165276c70 0x55c1652e16e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c165275e20 0x55c165285370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 --2- 172.21.15.26:0/4267969393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1652754d0 0x55c165282350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 >> 172.21.15.26:0/4267969393 conn(0x55c16526c600 msgr2=0x55c16527cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fae7fc700 0 client.25640 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 shutdown_connections 2023-07-13T22:15:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 1 -- 172.21.15.26:0/4267969393 wait complete. 2023-07-13T22:15:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.450+0000 7f8fe0d82b80 20 client.25640 trim_cache size 0 max 16384 2023-07-13T22:15:21.443 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:21.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.451+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 msgr2=0x7f8fb00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.451+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 0x7f8fb00a3800 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005bd0 tx=0x7f8fc0005a80 comp rx=0 tx=0).stop 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.451+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 msgr2=0x7f8fc8082480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.451+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8082480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8006fd0 tx=0x7f8fb800b730 comp rx=0 tx=0).stop 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 shutdown_connections 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fc8077210 0x7f8fc81bcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fc8071200 0x7f8fc8082480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8fb00a1350 0x7f8fb00a3800 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 --2- 172.21.15.26:0/3607809673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fc8074850 0x7f8fc80829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 >> 172.21.15.26:0/3607809673 conn(0x7f8fc806ca90 msgr2=0x7f8fc8076120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 shutdown_connections 2023-07-13T22:15:21.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:21.452+0000 7f8fcd3ff700 1 -- 172.21.15.26:0/3607809673 wait complete. 2023-07-13T22:15:22.478 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:15:22.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:15:22.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:22 smithi117 ceph-mon[129074]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 153 MiB/s wr, 56 op/s 2023-07-13T22:15:22.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:22 smithi026 ceph-mon[95647]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 153 MiB/s wr, 56 op/s 2023-07-13T22:15:22.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:22 smithi121 ceph-mon[129187]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 153 MiB/s wr, 56 op/s 2023-07-13T22:15:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.054+0000 7f6a5ffff700 1 -- 172.21.15.26:0/694144942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a5810bd10 msgr2=0x7f6a5810e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.054+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/694144942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a5810bd10 0x7f6a5810e170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009990 tx=0x7f6a48009ca0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.055+0000 7f6a5ffff700 1 -- 172.21.15.26:0/694144942 shutdown_connections 2023-07-13T22:15:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.055+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/694144942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a58070c90 0x7f6a58071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.055+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/694144942 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a5810bd10 0x7f6a5810e170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.055+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/694144942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a58071650 0x7f6a5810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.055+0000 7f6a5ffff700 1 -- 172.21.15.26:0/694144942 >> 172.21.15.26:0/694144942 conn(0x7f6a5806ca20 msgr2=0x7f6a5806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.074+0000 7f6a5ffff700 1 -- 172.21.15.26:0/694144942 shutdown_connections 2023-07-13T22:15:23.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.085+0000 7f6a5ffff700 1 -- 172.21.15.26:0/694144942 wait complete. 2023-07-13T22:15:23.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.086+0000 7f6a5ffff700 1 Processor -- start 2023-07-13T22:15:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 -- start start 2023-07-13T22:15:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a58071650 0x7f6a58114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a5810bd10 0x7f6a58115280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58116c70 con 0x7f6a58071650 2023-07-13T22:15:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58116de0 con 0x7f6a5810bd10 2023-07-13T22:15:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.087+0000 7f6a5ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58116fa0 con 0x7f6a581162c0 2023-07-13T22:15:23.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.088+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a5810bd10 0x7f6a58115280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.088+0000 7f6a5e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.088+0000 7f6a5e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53250/0 (socket says 172.21.15.26:53250) 2023-07-13T22:15:23.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.088+0000 7f6a5e59c700 1 -- 172.21.15.26:0/2086254908 learned_addr learned my addr 172.21.15.26:0/2086254908 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.089+0000 7f6a5e59c700 1 -- 172.21.15.26:0/2086254908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a5810bd10 msgr2=0x7f6a58115280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.089+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a5810bd10 0x7f6a58115280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.089+0000 7f6a5e59c700 1 -- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a58071650 msgr2=0x7f6a58114d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.089+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a58071650 0x7f6a58114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.089+0000 7f6a5e59c700 1 -- 172.21.15.26:0/2086254908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480095f0 con 0x7f6a581162c0 2023-07-13T22:15:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.116+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b5c0 tx=0x7f6a4800be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.116+0000 7f6a4effd700 1 -- 172.21.15.26:0/2086254908 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801d070 con 0x7f6a581162c0 2023-07-13T22:15:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.116+0000 7f6a4effd700 1 -- 172.21.15.26:0/2086254908 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a4800f460 con 0x7f6a581162c0 2023-07-13T22:15:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.116+0000 7f6a4effd700 1 -- 172.21.15.26:0/2086254908 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48017650 con 0x7f6a581162c0 2023-07-13T22:15:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.117+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a58117220 con 0x7f6a581162c0 2023-07-13T22:15:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.117+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581b5960 con 0x7f6a581162c0 2023-07-13T22:15:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.129+0000 7f6a73851b80 1 Processor -- start 2023-07-13T22:15:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 -- start start 2023-07-13T22:15:23.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f16927760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f16928160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1682bd30 con 0x555f16926ea0 2023-07-13T22:15:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1687eac0 con 0x555f16926550 2023-07-13T22:15:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.130+0000 7f6a73851b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16877f50 con 0x555f16927cf0 2023-07-13T22:15:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.131+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.131+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51718/0 (socket says 172.21.15.26:51718) 2023-07-13T22:15:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.131+0000 7f6a5d59a700 1 -- 172.21.15.26:0/669498953 learned_addr learned my addr 172.21.15.26:0/669498953 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.131+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f16928160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.140+0000 7f6a5d59a700 1 -- 172.21.15.26:0/669498953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 msgr2=0x555f16928160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.140+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f16928160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.140+0000 7f6a5d59a700 1 -- 172.21.15.26:0/669498953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 msgr2=0x555f16927760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.140+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f16927760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.140+0000 7f6a5d59a700 1 -- 172.21.15.26:0/669498953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f166a5e60 con 0x555f16926550 2023-07-13T22:15:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.151+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a540165d0 tx=0x7f6a5400af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7559c079d971d1e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.153+0000 7f6a4effd700 1 -- 172.21.15.26:0/2086254908 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6a4800f5d0 con 0x7f6a581162c0 2023-07-13T22:15:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.155+0000 7f6a4effd700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 0x7f6a440a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.156+0000 7f6a4effd700 1 -- 172.21.15.26:0/2086254908 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a480cdb30 con 0x7f6a581162c0 2023-07-13T22:15:23.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.165+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 0x7f6a440a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.176+0000 7f6a43fff700 1 -- 172.21.15.26:0/669498953 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54016bf0 con 0x555f16926550 2023-07-13T22:15:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.176+0000 7f6a43fff700 1 -- 172.21.15.26:0/669498953 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5400c070 con 0x555f16926550 2023-07-13T22:15:23.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.176+0000 7f6a43fff700 1 -- 172.21.15.26:0/669498953 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54012390 con 0x555f16926550 2023-07-13T22:15:23.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.180+0000 7f6a73851b80 1 -- 172.21.15.26:0/669498953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 msgr2=0x555f16926960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.180+0000 7f6a73851b80 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a540165d0 tx=0x7f6a5400af90 comp rx=0 tx=0).stop 2023-07-13T22:15:23.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.181+0000 7f6a73851b80 1 -- 172.21.15.26:0/669498953 shutdown_connections 2023-07-13T22:15:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.181+0000 7f6a73851b80 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f16928160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.181+0000 7f6a73851b80 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f16926960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.181+0000 7f6a73851b80 1 --2- 172.21.15.26:0/669498953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f16927760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.181+0000 7f6a73851b80 1 -- 172.21.15.26:0/669498953 >> 172.21.15.26:0/669498953 conn(0x555f1691be20 msgr2=0x555f16921250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.182+0000 7f6a73851b80 1 -- 172.21.15.26:0/669498953 shutdown_connections 2023-07-13T22:15:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.182+0000 7f6a73851b80 1 -- 172.21.15.26:0/669498953 wait complete. 2023-07-13T22:15:23.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.183+0000 7f6a73851b80 1 -- start start 2023-07-13T22:15:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.183+0000 7f6a73851b80 1 Processor -- start 2023-07-13T22:15:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f169302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f16937830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16817b40 con 0x555f16926550 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16817840 con 0x555f16927cf0 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f166fb9a0 con 0x555f16926ea0 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.184+0000 7f6a73851b80 10 client.0 _ll_register_callbacks cb 0x555f168157e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.185+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.185+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53262/0 (socket says 172.21.15.26:53262) 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.185+0000 7f6a5d59a700 1 -- 172.21.15.26:0/2819164569 learned_addr learned my addr 172.21.15.26:0/2819164569 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.186+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f16937830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.193+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f169302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.197+0000 7f6a5d59a700 1 -- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 msgr2=0x555f16937830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.197+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f16937830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.197+0000 7f6a5d59a700 1 -- 172.21.15.26:0/2819164569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 msgr2=0x555f169302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.197+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f169302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.197+0000 7f6a5d59a700 1 -- 172.21.15.26:0/2819164569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5400e040 con 0x555f16926ea0 2023-07-13T22:15:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.220+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ab40 tx=0x7f6a54017d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a551222451467d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.220+0000 7f6a417fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.231+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54016cb0 con 0x555f16926ea0 2023-07-13T22:15:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.231+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a54016e10 con 0x555f16926ea0 2023-07-13T22:15:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.231+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f167d1570 con 0x555f16926ea0 2023-07-13T22:15:23.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.242+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a540173e0 con 0x555f16926ea0 2023-07-13T22:15:23.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.242+0000 7f6a73851b80 10 client.25627 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:23.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.242+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f1668ece0 con 0x555f16926ea0 2023-07-13T22:15:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.248+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 0x7f6a440a3800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a581076e0 tx=0x7f6a50004be0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.249+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f167d1570 con 0x555f16926ea0 2023-07-13T22:15:23.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.249+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a5402bb70 con 0x555f16926ea0 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 10 client.25627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 20 client.25627 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f1693bda0 0x555f1693e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 4 client.25627 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.250+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f16870f30 con 0x555f1693bda0 2023-07-13T22:15:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.251+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f1693bda0 0x555f1693e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.252+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f1693bda0 0x555f1693e180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=909880afd5f3f1a server_cookie=d2f3a1eea64d4350 in_seq=0 out_seq=0 2023-07-13T22:15:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.252+0000 7f6a417fa700 10 client.25627 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.253+0000 7f6a417fa700 1 -- 172.21.15.26:0/2819164569 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f16870f30 con 0x555f1693bda0 2023-07-13T22:15:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.253+0000 7f6a417fa700 10 client.25627 handle_command_reply: tid=0 2023-07-13T22:15:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.253+0000 7f6a73851b80 1 client.25627 shutdown 2023-07-13T22:15:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 msgr2=0x555f169307f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ab40 tx=0x7f6a54017d60 comp rx=0 tx=0).stop 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 shutdown_connections 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f169302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f1693bda0 0x555f1693e180 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f16937830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 --2- 172.21.15.26:0/2819164569 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f169307f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 >> 172.21.15.26:0/2819164569 conn(0x555f1691be20 msgr2=0x555f16920930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.254+0000 7f6a417fa700 0 client.25627 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.263+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 shutdown_connections 2023-07-13T22:15:23.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.268+0000 7f6a73851b80 1 -- 172.21.15.26:0/2819164569 wait complete. 2023-07-13T22:15:23.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.269+0000 7f6a73851b80 20 client.25627 trim_cache size 0 max 16384 2023-07-13T22:15:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.312+0000 7f6a73851b80 1 Processor -- start 2023-07-13T22:15:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.312+0000 7f6a73851b80 1 -- start start 2023-07-13T22:15:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.312+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f1692acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.312+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f169377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.312+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.313+0000 7f6a73851b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16921f90 con 0x555f16926ea0 2023-07-13T22:15:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.313+0000 7f6a73851b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16922170 con 0x555f16926550 2023-07-13T22:15:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.313+0000 7f6a73851b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f1692fe20 con 0x555f16927cf0 2023-07-13T22:15:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.313+0000 7f6a5dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f1692acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.314+0000 7f6a5e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.314+0000 7f6a5e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53266/0 (socket says 172.21.15.26:53266) 2023-07-13T22:15:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.314+0000 7f6a5e59c700 1 -- 172.21.15.26:0/1936155102 learned_addr learned my addr 172.21.15.26:0/1936155102 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.316+0000 7f6a5e59c700 1 -- 172.21.15.26:0/1936155102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 msgr2=0x555f1692acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.316+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f1692acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.316+0000 7f6a5e59c700 1 -- 172.21.15.26:0/1936155102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 msgr2=0x555f169377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.316+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f169377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.316+0000 7f6a5e59c700 1 -- 172.21.15.26:0/1936155102 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f166cc2d0 con 0x555f16927cf0 2023-07-13T22:15:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.320+0000 7f6a5e59c700 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a480979f0 tx=0x7f6a4809ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8282604673dff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.323+0000 7f6a4cff9700 1 -- 172.21.15.26:0/1936155102 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4809efb0 con 0x555f16927cf0 2023-07-13T22:15:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.323+0000 7f6a4cff9700 1 -- 172.21.15.26:0/1936155102 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a480a3340 con 0x555f16927cf0 2023-07-13T22:15:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.323+0000 7f6a4cff9700 1 -- 172.21.15.26:0/1936155102 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4809b440 con 0x555f16927cf0 2023-07-13T22:15:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.324+0000 7f6a73851b80 1 -- 172.21.15.26:0/1936155102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 msgr2=0x555f1692b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.324+0000 7f6a73851b80 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a480979f0 tx=0x7f6a4809ae50 comp rx=0 tx=0).stop 2023-07-13T22:15:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.325+0000 7f6a73851b80 1 -- 172.21.15.26:0/1936155102 shutdown_connections 2023-07-13T22:15:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.325+0000 7f6a73851b80 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16927cf0 0x555f1692b230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.325+0000 7f6a73851b80 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16926550 0x555f1692acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.325+0000 7f6a73851b80 1 --2- 172.21.15.26:0/1936155102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926ea0 0x555f169377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.325+0000 7f6a73851b80 1 -- 172.21.15.26:0/1936155102 >> 172.21.15.26:0/1936155102 conn(0x555f1691be20 msgr2=0x555f1692ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.336+0000 7f6a73851b80 1 -- 172.21.15.26:0/1936155102 shutdown_connections 2023-07-13T22:15:23.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.339+0000 7f6a73851b80 1 -- 172.21.15.26:0/1936155102 wait complete. 2023-07-13T22:15:23.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.340+0000 7f6a73851b80 1 -- start start 2023-07-13T22:15:23.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.340+0000 7f6a73851b80 1 Processor -- start 2023-07-13T22:15:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f16936610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f1698fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16937230 con 0x555f16926550 2023-07-13T22:15:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f16990160 con 0x555f16927cf0 2023-07-13T22:15:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f169902a0 con 0x555f16926ea0 2023-07-13T22:15:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a73851b80 10 client.0 _ll_register_callbacks cb 0x7f6a5400ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a5d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53268/0 (socket says 172.21.15.26:53268) 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.341+0000 7f6a5d59a700 1 -- 172.21.15.26:0/188533854 learned_addr learned my addr 172.21.15.26:0/188533854 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.342+0000 7f6a5d59a700 1 -- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 msgr2=0x555f1698fc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.342+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f1698fc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.342+0000 7f6a5d59a700 1 -- 172.21.15.26:0/188533854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 msgr2=0x555f16936610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.342+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f16936610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.342+0000 7f6a5d59a700 1 -- 172.21.15.26:0/188533854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a581b5960 con 0x555f16926ea0 2023-07-13T22:15:23.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.343+0000 7f6a5d59a700 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a540155f0 tx=0x7f6a54015930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122c44d8e8074605 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.343+0000 7f6a417fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:23.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.344+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400a2a0 con 0x555f16926ea0 2023-07-13T22:15:23.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.344+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a5400c070 con 0x555f16926ea0 2023-07-13T22:15:23.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.345+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a54015360 con 0x555f16926ea0 2023-07-13T22:15:23.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.345+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a5400a4b0 con 0x555f16926ea0 2023-07-13T22:15:23.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a73851b80 10 client.25633 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:23.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555f166a4330 con 0x555f16926ea0 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a5400c070 con 0x555f16926ea0 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6a54018780 con 0x555f16926ea0 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a73851b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a73851b80 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.350+0000 7f6a73851b80 20 client.25633 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:23.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.351+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f16991960 0x555f16993e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.351+0000 7f6a73851b80 4 client.25633 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:15:23.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.351+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f168330d0 con 0x555f16991960 2023-07-13T22:15:23.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.370+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f16991960 0x555f16993e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.373+0000 7f6a5dd9b700 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f16991960 0x555f16993e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a788a997dc4c4554 server_cookie=667c2be0aa5a41c3 in_seq=0 out_seq=0 2023-07-13T22:15:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.374+0000 7f6a417fa700 10 client.25633 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.380+0000 7f6a417fa700 1 -- 172.21.15.26:0/188533854 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f168330d0 con 0x555f16991960 2023-07-13T22:15:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.380+0000 7f6a417fa700 10 client.25633 handle_command_reply: tid=0 2023-07-13T22:15:23.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.380+0000 7f6a73851b80 1 client.25633 shutdown 2023-07-13T22:15:23.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 msgr2=0x555f16936b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a540155f0 tx=0x7f6a54015930 comp rx=0 tx=0).stop 2023-07-13T22:15:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 shutdown_connections 2023-07-13T22:15:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f16926550 0x555f16936610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f16991960 0x555f16993e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f16927cf0 0x555f1698fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 --2- 172.21.15.26:0/188533854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f16926ea0 0x555f16936b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.393+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 >> 172.21.15.26:0/188533854 conn(0x555f1691be20 msgr2=0x555f1692acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.394+0000 7f6a417fa700 0 client.25633 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.394+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 shutdown_connections 2023-07-13T22:15:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.394+0000 7f6a73851b80 1 -- 172.21.15.26:0/188533854 wait complete. 2023-07-13T22:15:23.420 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:15:23.420 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:15:23.420 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "976b7fc2-9087-4ce0-aa36-e32bda3dea1b", 2023-07-13T22:15:23.421 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:15:23.421 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:15:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.416+0000 7f6a73851b80 20 client.25633 trim_cache size 0 max 16384 2023-07-13T22:15:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.443+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 msgr2=0x7f6a440a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.443+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 0x7f6a440a3800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6a581076e0 tx=0x7f6a50004be0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.443+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 msgr2=0x7f6a58116730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.443+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b5c0 tx=0x7f6a4800be70 comp rx=0 tx=0).stop 2023-07-13T22:15:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 shutdown_connections 2023-07-13T22:15:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6a440a1350 0x7f6a440a3800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a58071650 0x7f6a58114d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a581162c0 0x7f6a58116730 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 --2- 172.21.15.26:0/2086254908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a5810bd10 0x7f6a58115280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.444+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 >> 172.21.15.26:0/2086254908 conn(0x7f6a5806ca20 msgr2=0x7f6a5810ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.455+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 shutdown_connections 2023-07-13T22:15:23.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.455+0000 7f6a5ffff700 1 -- 172.21.15.26:0/2086254908 wait complete. 2023-07-13T22:15:23.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:15:23.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.975+0000 7f85a5bff700 1 -- 172.21.15.26:0/1324344677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0070ee0 msgr2=0x7f85a00712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.975+0000 7f85a5bff700 1 --2- 172.21.15.26:0/1324344677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0070ee0 0x7f85a00712f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f859000b0a0 tx=0x7f859000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.978+0000 7f85a5bff700 1 -- 172.21.15.26:0/1324344677 shutdown_connections 2023-07-13T22:15:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.978+0000 7f85a5bff700 1 --2- 172.21.15.26:0/1324344677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a0070ee0 0x7f85a00712f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.978+0000 7f85a5bff700 1 --2- 172.21.15.26:0/1324344677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a010bd60 0x7f85a010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.978+0000 7f85a5bff700 1 --2- 172.21.15.26:0/1324344677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.978+0000 7f85a5bff700 1 -- 172.21.15.26:0/1324344677 >> 172.21.15.26:0/1324344677 conn(0x7f85a006ca90 msgr2=0x7f85a006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.983+0000 7f85a5bff700 1 -- 172.21.15.26:0/1324344677 shutdown_connections 2023-07-13T22:15:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.983+0000 7f85a5bff700 1 -- 172.21.15.26:0/1324344677 wait complete. 2023-07-13T22:15:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.985+0000 7f85a5bff700 1 Processor -- start 2023-07-13T22:15:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 -- start start 2023-07-13T22:15:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a0070ee0 0x7f85a01a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a010bd60 0x7f85a01abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a01ac1f0 con 0x7f85a010bd60 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a010ec40 con 0x7f85a0070ee0 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.986+0000 7f85a5bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a010edd0 con 0x7f85a0071830 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53290/0 (socket says 172.21.15.26:53290) 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 -- 172.21.15.26:0/3758521895 learned_addr learned my addr 172.21.15.26:0/3758521895 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a010bd60 0x7f85a01abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 -- 172.21.15.26:0/3758521895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a0070ee0 msgr2=0x7f85a01a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a0070ee0 0x7f85a01a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 -- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a010bd60 msgr2=0x7f85a01abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.987+0000 7f859effd700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a010bd60 0x7f85a01abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.988+0000 7f859effd700 1 -- 172.21.15.26:0/3758521895 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009d00 con 0x7f85a0071830 2023-07-13T22:15:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.988+0000 7f859effd700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f859800e970 tx=0x7f859800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.989+0000 7f859cff9700 1 -- 172.21.15.26:0/3758521895 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009790 con 0x7f85a0071830 2023-07-13T22:15:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.989+0000 7f859cff9700 1 -- 172.21.15.26:0/3758521895 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f859800eec0 con 0x7f85a0071830 2023-07-13T22:15:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.989+0000 7f859cff9700 1 -- 172.21.15.26:0/3758521895 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598003cf0 con 0x7f85a0071830 2023-07-13T22:15:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.990+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a010f0b0 con 0x7f85a0071830 2023-07-13T22:15:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.990+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a010f570 con 0x7f85a0071830 2023-07-13T22:15:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:23.995+0000 7f859cff9700 1 -- 172.21.15.26:0/3758521895 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f859800ccd0 con 0x7f85a0071830 2023-07-13T22:15:23.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.003+0000 7f85b960ab80 1 Processor -- start 2023-07-13T22:15:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.004+0000 7f85b960ab80 1 -- start start 2023-07-13T22:15:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.007+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.008+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.008+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7f950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.008+0000 7f85b960ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7e98cb0 con 0x560cb7f93e20 2023-07-13T22:15:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.008+0000 7f85b960ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7eeba40 con 0x560cb7f934d0 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.008+0000 7f85b960ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7ee4ed0 con 0x560cb7f94c70 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.009+0000 7f859cff9700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 0x7f858c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.009+0000 7f859cff9700 1 -- 172.21.15.26:0/3758521895 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8598014070 con 0x7f85a0071830 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.009+0000 7f859f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.009+0000 7f859f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59434/0 (socket says 172.21.15.26:59434) 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.009+0000 7f859f7fe700 1 -- 172.21.15.26:0/1829584283 learned_addr learned my addr 172.21.15.26:0/1829584283 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 -- 172.21.15.26:0/1829584283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 msgr2=0x560cb7f950e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7f950e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 -- 172.21.15.26:0/1829584283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 msgr2=0x560cb7f938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 -- 172.21.15.26:0/1829584283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb7d12de0 con 0x560cb7f93e20 2023-07-13T22:15:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859ffff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 0x7f858c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.010+0000 7f859f7fe700 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f859000b780 tx=0x7f8590009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97b8d4e1709321e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.011+0000 7f859ffff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 0x7f858c0a3700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f858800acf0 tx=0x7f858800a840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.011+0000 7f8595ffb700 1 -- 172.21.15.26:0/1829584283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590021070 con 0x560cb7f93e20 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.011+0000 7f8595ffb700 1 -- 172.21.15.26:0/1829584283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8590008ce0 con 0x560cb7f93e20 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.011+0000 7f8595ffb700 1 -- 172.21.15.26:0/1829584283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000e040 con 0x560cb7f93e20 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 -- 172.21.15.26:0/1829584283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 msgr2=0x560cb7f946e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f859000b780 tx=0x7f8590009800 comp rx=0 tx=0).stop 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 -- 172.21.15.26:0/1829584283 shutdown_connections 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7f950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 --2- 172.21.15.26:0/1829584283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f946e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.012+0000 7f85b960ab80 1 -- 172.21.15.26:0/1829584283 >> 172.21.15.26:0/1829584283 conn(0x560cb7f88da0 msgr2=0x560cb7f8e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.013+0000 7f85b960ab80 1 -- 172.21.15.26:0/1829584283 shutdown_connections 2023-07-13T22:15:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.013+0000 7f85b960ab80 1 -- 172.21.15.26:0/1829584283 wait complete. 2023-07-13T22:15:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.014+0000 7f85b960ab80 1 -- start start 2023-07-13T22:15:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.014+0000 7f85b960ab80 1 Processor -- start 2023-07-13T22:15:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f9d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7e84ac0 con 0x560cb7f93e20 2023-07-13T22:15:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7e847c0 con 0x560cb7f934d0 2023-07-13T22:15:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7d68920 con 0x560cb7f94c70 2023-07-13T22:15:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.015+0000 7f85b960ab80 10 client.0 _ll_register_callbacks cb 0x560cb7e82760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.016+0000 7f859ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.016+0000 7f859effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f9d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.016+0000 7f859ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53298/0 (socket says 172.21.15.26:53298) 2023-07-13T22:15:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.016+0000 7f859ffff700 1 -- 172.21.15.26:0/226671032 learned_addr learned my addr 172.21.15.26:0/226671032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.016+0000 7f859f7fe700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.017+0000 7f859ffff700 1 -- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 msgr2=0x560cb7f9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.017+0000 7f859ffff700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.017+0000 7f859ffff700 1 -- 172.21.15.26:0/226671032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 msgr2=0x560cb7f9d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.017+0000 7f859ffff700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f9d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.017+0000 7f859ffff700 1 -- 172.21.15.26:0/226671032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a0117640 con 0x560cb7f94c70 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.018+0000 7f859ffff700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8588004820 tx=0x7f858800a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463180279ee49084 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.018+0000 7f857effd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:24.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.018+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588025670 con 0x560cb7f94c70 2023-07-13T22:15:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.019+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cb7e3e4f0 con 0x560cb7f94c70 2023-07-13T22:15:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.019+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f85880257d0 con 0x560cb7f94c70 2023-07-13T22:15:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.019+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858800ae90 con 0x560cb7f94c70 2023-07-13T22:15:24.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.019+0000 7f85b960ab80 10 client.25642 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:24.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560cb7cfbc60 con 0x560cb7f94c70 2023-07-13T22:15:24.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cb7e3e4f0 con 0x560cb7f94c70 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f858802e070 con 0x560cb7f94c70 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f85b960ab80 10 client.25642 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f85b960ab80 10 client.25642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f85b960ab80 20 client.25642 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.020+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7fa8f40 0x560cb7fab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.021+0000 7f85b960ab80 4 client.25642 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:24.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.021+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cb7eddeb0 con 0x560cb7fa8f40 2023-07-13T22:15:24.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.021+0000 7f859f7fe700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7fa8f40 0x560cb7fab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.021+0000 7f857effd700 10 client.25642 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:24.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.021+0000 7f859f7fe700 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7fa8f40 0x560cb7fab350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726fc343c53bfb49 server_cookie=1972e84028107fe7 in_seq=0 out_seq=0 2023-07-13T22:15:24.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.022+0000 7f857effd700 1 -- 172.21.15.26:0/226671032 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560cb7eddeb0 con 0x560cb7fa8f40 2023-07-13T22:15:24.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.022+0000 7f857effd700 10 client.25642 handle_command_reply: tid=0 2023-07-13T22:15:24.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.023+0000 7f85b960ab80 1 client.25642 shutdown 2023-07-13T22:15:24.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.023+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 msgr2=0x560cb7fa4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.023+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8588004820 tx=0x7f858800a250 comp rx=0 tx=0).stop 2023-07-13T22:15:24.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 shutdown_connections 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7fa8f40 0x560cb7fab350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f934d0 0x560cb7f9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 --2- 172.21.15.26:0/226671032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f93e20 0x560cb7f9d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f857effd700 0 client.25642 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 >> 172.21.15.26:0/226671032 conn(0x560cb7f88da0 msgr2=0x560cb7f8d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:24.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 shutdown_connections 2023-07-13T22:15:24.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.024+0000 7f85b960ab80 1 -- 172.21.15.26:0/226671032 wait complete. 2023-07-13T22:15:24.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.025+0000 7f85b960ab80 20 client.25642 trim_cache size 0 max 16384 2023-07-13T22:15:24.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.061+0000 7f85b960ab80 1 Processor -- start 2023-07-13T22:15:24.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.061+0000 7f85b960ab80 1 -- start start 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f93e20 0x560cb7fa4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7f88970 con 0x560cb7f934d0 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7f95420 con 0x560cb7f93e20 2023-07-13T22:15:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.062+0000 7f85b960ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7fab9b0 con 0x560cb7f94c70 2023-07-13T22:15:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.063+0000 7f859f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.063+0000 7f859f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59450/0 (socket says 172.21.15.26:59450) 2023-07-13T22:15:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.063+0000 7f859f7fe700 1 -- 172.21.15.26:0/508814773 learned_addr learned my addr 172.21.15.26:0/508814773 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.063+0000 7f859ffff700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.063+0000 7f859effd700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f93e20 0x560cb7fa4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 -- 172.21.15.26:0/508814773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f93e20 msgr2=0x560cb7fa4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f93e20 0x560cb7fa4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 -- 172.21.15.26:0/508814773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 msgr2=0x560cb7fafc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 -- 172.21.15.26:0/508814773 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb7d39250 con 0x560cb7f94c70 2023-07-13T22:15:24.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859f7fe700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:15:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.064+0000 7f859ffff700 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f858801bc60 tx=0x7f8588004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e864facc598aa37b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.065+0000 7f85967fc700 1 -- 172.21.15.26:0/508814773 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858801be80 con 0x560cb7f94c70 2023-07-13T22:15:24.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.065+0000 7f85967fc700 1 -- 172.21.15.26:0/508814773 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f858802b070 con 0x560cb7f94c70 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.065+0000 7f85967fc700 1 -- 172.21.15.26:0/508814773 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858801caf0 con 0x560cb7f94c70 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 -- 172.21.15.26:0/508814773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 msgr2=0x560cb7fa4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4ec0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f858801bc60 tx=0x7f8588004790 comp rx=0 tx=0).stop 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 -- 172.21.15.26:0/508814773 shutdown_connections 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fafc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f94c70 0x560cb7fa4ec0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 --2- 172.21.15.26:0/508814773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f93e20 0x560cb7fa4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 -- 172.21.15.26:0/508814773 >> 172.21.15.26:0/508814773 conn(0x560cb7f88da0 msgr2=0x560cb7f8e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 -- 172.21.15.26:0/508814773 shutdown_connections 2023-07-13T22:15:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.066+0000 7f85b960ab80 1 -- 172.21.15.26:0/508814773 wait complete. 2023-07-13T22:15:24.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.067+0000 7f85b960ab80 1 -- start start 2023-07-13T22:15:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.067+0000 7f85b960ab80 1 Processor -- start 2023-07-13T22:15:24.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fa3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 0x560cb7fa4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 0x560cb7fa0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7fa11b0 con 0x560cb7f934d0 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7fa1320 con 0x560cb7f94c70 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb7fa14e0 con 0x560cb7f93e20 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.068+0000 7f85b960ab80 10 client.0 _ll_register_callbacks cb 0x7f858800fed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:24.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.069+0000 7f859effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 0x560cb7fa4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.069+0000 7f859effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 0x560cb7fa4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53310/0 (socket says 172.21.15.26:53310) 2023-07-13T22:15:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.069+0000 7f859effd700 1 -- 172.21.15.26:0/4249971178 learned_addr learned my addr 172.21.15.26:0/4249971178 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.069+0000 7f859ffff700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 0x560cb7fa0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 -- 172.21.15.26:0/4249971178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 msgr2=0x560cb7fa4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 0x560cb7fa4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 -- 172.21.15.26:0/4249971178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 msgr2=0x560cb7fa3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fa3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 -- 172.21.15.26:0/4249971178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cb7d12de0 con 0x560cb7f94c70 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f859ffff700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 0x560cb7fa0ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f858800a810 tx=0x7f858801bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a45e37abb9cc3b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.070+0000 7f857effd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:24.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.071+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588041070 con 0x560cb7f94c70 2023-07-13T22:15:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.071+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8588036070 con 0x560cb7f94c70 2023-07-13T22:15:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.071+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8590008ce0 con 0x560cb7f94c70 2023-07-13T22:15:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.071+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f858801ca80 con 0x560cb7f94c70 2023-07-13T22:15:24.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.072+0000 7f85b960ab80 10 client.25661 fetch_fsmap learned FSMap version 27 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.072+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560cb7cfbc60 con 0x560cb7f94c70 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.072+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8590008ce0 con 0x560cb7f94c70 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8588044030 con 0x560cb7f94c70 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 10 client.25661 fetch_fsmap finished waiting for FSMap version 27 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 20 client.25661 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:24.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7ffdfb0 0x560cb80003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 4 client.25661 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:15:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cb7ea0050 con 0x560cb7ffdfb0 2023-07-13T22:15:24.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.073+0000 7f859f7fe700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7ffdfb0 0x560cb80003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.074+0000 7f859f7fe700 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7ffdfb0 0x560cb80003c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76502b9e398078ec server_cookie=166055dcc3c9417d in_seq=0 out_seq=0 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.074+0000 7f857effd700 10 client.25661 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.074+0000 7f857effd700 1 -- 172.21.15.26:0/4249971178 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560cb7ea0050 con 0x560cb7ffdfb0 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.074+0000 7f857effd700 10 client.25661 handle_command_reply: tid=0 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.074+0000 7f85b960ab80 1 client.25661 shutdown 2023-07-13T22:15:24.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 msgr2=0x560cb7fa0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 0x560cb7fa0ac0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f858800a810 tx=0x7f858801bfc0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 shutdown_connections 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560cb7f934d0 0x560cb7fa3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560cb7ffdfb0 0x560cb80003c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560cb7f94c70 0x560cb7fa0ac0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 --2- 172.21.15.26:0/4249971178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560cb7f93e20 0x560cb7fa4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 >> 172.21.15.26:0/4249971178 conn(0x560cb7f88da0 msgr2=0x560cb7f8e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f857effd700 0 client.25661 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 shutdown_connections 2023-07-13T22:15:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.075+0000 7f85b960ab80 1 -- 172.21.15.26:0/4249971178 wait complete. 2023-07-13T22:15:24.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.076+0000 7f85b960ab80 20 client.25661 trim_cache size 0 max 16384 2023-07-13T22:15:24.066 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "976b7fc2-9087-4ce0-aa36-e32bda3dea1b": { 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "976b7fc2-9087-4ce0-aa36-e32bda3dea1b", 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:15:24.067 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:15:24.068 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:15:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.077+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 msgr2=0x7f858c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.077+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 0x7f858c0a3700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f858800acf0 tx=0x7f858800a840 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.077+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 msgr2=0x7f85a01a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.077+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f859800e970 tx=0x7f859800ec80 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 shutdown_connections 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f858c0a1250 0x7f858c0a3700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85a010bd60 0x7f85a01abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a0070ee0 0x7f85a01a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 --2- 172.21.15.26:0/3758521895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85a0071830 0x7f85a01a4bb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:24.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 >> 172.21.15.26:0/3758521895 conn(0x7f85a006ca90 msgr2=0x7f85a006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:24.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 shutdown_connections 2023-07-13T22:15:24.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:24.078+0000 7f85a5bff700 1 -- 172.21.15.26:0/3758521895 wait complete. 2023-07-13T22:15:24.110 INFO:tasks.cephfs.filesystem:scrub status for tag:976b7fc2-9087-4ce0-aa36-e32bda3dea1b - {'path': '/', 'tag': '976b7fc2-9087-4ce0-aa36-e32bda3dea1b', 'options': 'recursive,force'} 2023-07-13T22:15:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:24 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:15:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:24 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:24 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:15:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:24 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:24 smithi117 ceph-mon[129074]: pgmap v361: 97 pgs: 97 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 231 MiB/s wr, 80 op/s 2023-07-13T22:15:24.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:24 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:15:24.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:24 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:24 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:15:24.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:24 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:24 smithi026 ceph-mon[95647]: pgmap v361: 97 pgs: 97 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 231 MiB/s wr, 80 op/s 2023-07-13T22:15:24.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:24 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:15:24.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:24 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:24 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:15:24.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:24 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:15:24.994 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:24 smithi121 ceph-mon[129187]: pgmap v361: 97 pgs: 97 active+clean; 2.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 231 MiB/s wr, 80 op/s 2023-07-13T22:15:26.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:26 smithi117 ceph-mon[129074]: pgmap v362: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 78 op/s 2023-07-13T22:15:26.943 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:26 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:15:26.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:26 smithi026 ceph-mon[95647]: pgmap v362: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 78 op/s 2023-07-13T22:15:26.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:26 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:15:26.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:26 smithi121 ceph-mon[129187]: pgmap v362: 97 pgs: 97 active+clean; 3.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 259 MiB/s wr, 78 op/s 2023-07-13T22:15:26.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:26 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:15:27.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:27 smithi121 ceph-mon[129187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:15:28.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:27 smithi117 ceph-mon[129074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:15:28.223 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:27 smithi026 ceph-mon[95647]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:15:29.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:28 smithi117 ceph-mon[129074]: pgmap v363: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 270 MiB/s wr, 84 op/s 2023-07-13T22:15:29.224 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:28 smithi026 ceph-mon[95647]: pgmap v363: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 270 MiB/s wr, 84 op/s 2023-07-13T22:15:29.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:28 smithi121 ceph-mon[129187]: pgmap v363: 97 pgs: 97 active+clean; 3.7 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 270 MiB/s wr, 84 op/s 2023-07-13T22:15:30.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:29 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:15:30.209 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:29 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:15:30.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:29 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:15:31.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:30 smithi117 ceph-mon[129074]: pgmap v364: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 79 op/s 2023-07-13T22:15:31.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:30 smithi026 ceph-mon[95647]: pgmap v364: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 79 op/s 2023-07-13T22:15:31.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:30 smithi121 ceph-mon[129187]: pgmap v364: 97 pgs: 97 active+clean; 4.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 79 op/s 2023-07-13T22:15:32.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:31 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:15:32.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:31 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:15:32.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:31 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:15:32.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:15:32.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:15:32.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:31 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:15:32.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:31 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:15:32.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:31 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:15:32.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:31 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:15:33.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:32 smithi117 ceph-mon[129074]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 261 MiB/s wr, 76 op/s 2023-07-13T22:15:33.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:32 smithi026 ceph-mon[95647]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 261 MiB/s wr, 76 op/s 2023-07-13T22:15:33.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:32 smithi121 ceph-mon[129187]: pgmap v365: 97 pgs: 97 active+clean; 4.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 261 MiB/s wr, 76 op/s 2023-07-13T22:15:34.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:33 smithi117 ceph-mon[129074]: pgmap v366: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 314 MiB/s wr, 93 op/s 2023-07-13T22:15:34.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:33 smithi026 ceph-mon[95647]: pgmap v366: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 314 MiB/s wr, 93 op/s 2023-07-13T22:15:34.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:33 smithi121 ceph-mon[129187]: pgmap v366: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 314 MiB/s wr, 93 op/s 2023-07-13T22:15:36.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:36 smithi117 ceph-mon[129074]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-07-13T22:15:36.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:36 smithi026 ceph-mon[95647]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-07-13T22:15:36.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:36 smithi121 ceph-mon[129187]: pgmap v367: 97 pgs: 97 active+clean; 6.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 280 MiB/s wr, 81 op/s 2023-07-13T22:15:38.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:38 smithi117 ceph-mon[129074]: pgmap v368: 97 pgs: 97 active+clean; 6.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 288 MiB/s wr, 85 op/s 2023-07-13T22:15:38.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:38 smithi026 ceph-mon[95647]: pgmap v368: 97 pgs: 97 active+clean; 6.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 288 MiB/s wr, 85 op/s 2023-07-13T22:15:38.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:38 smithi121 ceph-mon[129187]: pgmap v368: 97 pgs: 97 active+clean; 6.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 288 MiB/s wr, 85 op/s 2023-07-13T22:15:40.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:40 smithi117 ceph-mon[129074]: pgmap v369: 97 pgs: 97 active+clean; 7.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 85 op/s 2023-07-13T22:15:40.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:40 smithi026 ceph-mon[95647]: pgmap v369: 97 pgs: 97 active+clean; 7.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 85 op/s 2023-07-13T22:15:40.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:40 smithi121 ceph-mon[129187]: pgmap v369: 97 pgs: 97 active+clean; 7.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 85 op/s 2023-07-13T22:15:42.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:42 smithi117 ceph-mon[129074]: pgmap v370: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 81 op/s 2023-07-13T22:15:42.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:42 smithi026 ceph-mon[95647]: pgmap v370: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 81 op/s 2023-07-13T22:15:42.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:42 smithi121 ceph-mon[129187]: pgmap v370: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 283 MiB/s wr, 81 op/s 2023-07-13T22:15:44.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:44 smithi117 ceph-mon[129074]: pgmap v371: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 322 MiB/s wr, 94 op/s 2023-07-13T22:15:44.969 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:44 smithi026 ceph-mon[95647]: pgmap v371: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 322 MiB/s wr, 94 op/s 2023-07-13T22:15:44.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:44 smithi121 ceph-mon[129187]: pgmap v371: 97 pgs: 97 active+clean; 8.3 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 322 MiB/s wr, 94 op/s 2023-07-13T22:15:46.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:46 smithi117 ceph-mon[129074]: pgmap v372: 97 pgs: 97 active+clean; 8.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 82 op/s 2023-07-13T22:15:46.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:46 smithi026 ceph-mon[95647]: pgmap v372: 97 pgs: 97 active+clean; 8.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 82 op/s 2023-07-13T22:15:46.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:46 smithi121 ceph-mon[129187]: pgmap v372: 97 pgs: 97 active+clean; 8.9 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 82 op/s 2023-07-13T22:15:49.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:48 smithi117 ceph-mon[129074]: pgmap v373: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 84 op/s 2023-07-13T22:15:49.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:48 smithi026 ceph-mon[95647]: pgmap v373: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 84 op/s 2023-07-13T22:15:49.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:48 smithi121 ceph-mon[129187]: pgmap v373: 97 pgs: 97 active+clean; 9.4 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 84 op/s 2023-07-13T22:15:50.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:50 smithi117 ceph-mon[129074]: pgmap v374: 97 pgs: 97 active+clean; 9.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 87 op/s 2023-07-13T22:15:50.960 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:50 smithi026 ceph-mon[95647]: pgmap v374: 97 pgs: 97 active+clean; 9.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 87 op/s 2023-07-13T22:15:50.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:50 smithi121 ceph-mon[129187]: pgmap v374: 97 pgs: 97 active+clean; 9.8 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 87 op/s 2023-07-13T22:15:52.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:52 smithi117 ceph-mon[129074]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 81 op/s 2023-07-13T22:15:52.972 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:52 smithi026 ceph-mon[95647]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 81 op/s 2023-07-13T22:15:52.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:52 smithi121 ceph-mon[129187]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 81 op/s 2023-07-13T22:15:54.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:15:54.774 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:54 smithi026 ceph-mon[95647]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 329 MiB/s wr, 102 op/s 2023-07-13T22:15:54.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.936+0000 7fa849dff700 1 -- 172.21.15.26:0/2272346793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844103670 msgr2=0x7fa844103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:54.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.936+0000 7fa849dff700 1 --2- 172.21.15.26:0/2272346793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844103670 0x7fa844103ae0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa83c01c0d0 tx=0x7fa83c01c3e0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.938+0000 7fa849dff700 1 -- 172.21.15.26:0/2272346793 shutdown_connections 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.938+0000 7fa849dff700 1 --2- 172.21.15.26:0/2272346793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 0x7fa844100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.938+0000 7fa849dff700 1 --2- 172.21.15.26:0/2272346793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844103670 0x7fa844103ae0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.938+0000 7fa849dff700 1 --2- 172.21.15.26:0/2272346793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa844100cb0 0x7fa844103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.938+0000 7fa849dff700 1 -- 172.21.15.26:0/2272346793 >> 172.21.15.26:0/2272346793 conn(0x7fa8440fa000 msgr2=0x7fa8440fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.939+0000 7fa849dff700 1 -- 172.21.15.26:0/2272346793 shutdown_connections 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.939+0000 7fa849dff700 1 -- 172.21.15.26:0/2272346793 wait complete. 2023-07-13T22:15:54.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.940+0000 7fa849dff700 1 Processor -- start 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 -- start start 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 0x7fa844070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa844103670 0x7fa8440715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8441929b0 con 0x7fa8440fe350 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844192b20 con 0x7fa844100cb0 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.941+0000 7fa849dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844192ce0 con 0x7fa844103670 2023-07-13T22:15:54.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.942+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.942+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51576/0 (socket says 172.21.15.26:51576) 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.942+0000 7fa83affd700 1 -- 172.21.15.26:0/755821758 learned_addr learned my addr 172.21.15.26:0/755821758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa8437fe700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 0x7fa844070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa83affd700 1 -- 172.21.15.26:0/755821758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa844103670 msgr2=0x7fa8440715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa83affd700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa844103670 0x7fa8440715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa83affd700 1 -- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 msgr2=0x7fa844070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:54.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa83affd700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 0x7fa844070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.943+0000 7fa83affd700 1 -- 172.21.15.26:0/755821758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c01ad10 con 0x7fa844100cb0 2023-07-13T22:15:54.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.946+0000 7fa83affd700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa83400d950 tx=0x7fa83400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:54.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.957+0000 7fa8417fa700 1 -- 172.21.15.26:0/755821758 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834009b40 con 0x7fa844100cb0 2023-07-13T22:15:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.957+0000 7fa8417fa700 1 -- 172.21.15.26:0/755821758 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa834009ca0 con 0x7fa844100cb0 2023-07-13T22:15:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.957+0000 7fa8417fa700 1 -- 172.21.15.26:0/755821758 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400f620 con 0x7fa844100cb0 2023-07-13T22:15:54.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.957+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa844192fc0 con 0x7fa844100cb0 2023-07-13T22:15:54.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.957+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa844193480 con 0x7fa844100cb0 2023-07-13T22:15:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.970+0000 7fa85d80fb80 1 Processor -- start 2023-07-13T22:15:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.977+0000 7fa85d80fb80 1 -- start start 2023-07-13T22:15:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb63284d0 0x55dfb63288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb6328e20 0x55dfb63296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb622dcb0 con 0x55dfb6328e20 2023-07-13T22:15:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6280a40 con 0x55dfb6329c70 2023-07-13T22:15:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.978+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6279ed0 con 0x55dfb63284d0 2023-07-13T22:15:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.983+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.983+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51584/0 (socket says 172.21.15.26:51584) 2023-07-13T22:15:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.983+0000 7fa83affd700 1 -- 172.21.15.26:0/2274342547 learned_addr learned my addr 172.21.15.26:0/2274342547 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.984+0000 7fa8417fa700 1 -- 172.21.15.26:0/755821758 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa834010460 con 0x7fa844100cb0 2023-07-13T22:15:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.986+0000 7fa8417fa700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 0x7fa8300a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:54.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.986+0000 7fa8417fa700 1 -- 172.21.15.26:0/755821758 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa8340cc650 con 0x7fa844100cb0 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.989+0000 7fa83affd700 1 -- 172.21.15.26:0/2274342547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb63284d0 msgr2=0x55dfb63288e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.989+0000 7fa83affd700 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb63284d0 0x55dfb63288e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.989+0000 7fa83affd700 1 -- 172.21.15.26:0/2274342547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb6328e20 msgr2=0x55dfb63296e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.989+0000 7fa83affd700 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb6328e20 0x55dfb63296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:54.989+0000 7fa83affd700 1 -- 172.21.15.26:0/2274342547 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfb60a7de0 con 0x55dfb6329c70 2023-07-13T22:15:54.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.002+0000 7fa83affd700 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa834095f10 tx=0x7fa834095dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b366e8c584df1ecd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.009+0000 7fa839ffb700 1 -- 172.21.15.26:0/2274342547 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834099550 con 0x55dfb6329c70 2023-07-13T22:15:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.009+0000 7fa839ffb700 1 -- 172.21.15.26:0/2274342547 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa834099370 con 0x55dfb6329c70 2023-07-13T22:15:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.009+0000 7fa839ffb700 1 -- 172.21.15.26:0/2274342547 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834097ed0 con 0x55dfb6329c70 2023-07-13T22:15:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.010+0000 7fa85d80fb80 1 -- 172.21.15.26:0/2274342547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 msgr2=0x55dfb632a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.010+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa834095f10 tx=0x7fa834095dd0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.012+0000 7fa843fff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 0x7fa8300a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.013+0000 7fa843fff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 0x7fa8300a3800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa83c018bb0 tx=0x7fa83c019d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.014+0000 7fa85d80fb80 1 -- 172.21.15.26:0/2274342547 shutdown_connections 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.014+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb632a0e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.014+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb63284d0 0x55dfb63288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.014+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/2274342547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb6328e20 0x55dfb63296e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.014+0000 7fa85d80fb80 1 -- 172.21.15.26:0/2274342547 >> 172.21.15.26:0/2274342547 conn(0x55dfb631dda0 msgr2=0x55dfb63231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.034+0000 7fa85d80fb80 1 -- 172.21.15.26:0/2274342547 shutdown_connections 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.037+0000 7fa85d80fb80 1 -- 172.21.15.26:0/2274342547 wait complete. 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.037+0000 7fa85d80fb80 1 -- start start 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.038+0000 7fa85d80fb80 1 Processor -- start 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6332150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6332690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb63396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6219ac0 con 0x55dfb63284d0 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb62197c0 con 0x55dfb6329c70 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb60fd920 con 0x55dfb6328e20 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.039+0000 7fa85d80fb80 10 client.0 _ll_register_callbacks cb 0x55dfb6217760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.040+0000 7fa843fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb63396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.044+0000 7fa843fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb63396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51590/0 (socket says 172.21.15.26:51590) 2023-07-13T22:15:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.044+0000 7fa843fff700 1 -- 172.21.15.26:0/1494968511 learned_addr learned my addr 172.21.15.26:0/1494968511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.046+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6332150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.046+0000 7fa83affd700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6332690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 -- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 msgr2=0x55dfb63396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb63396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 -- 172.21.15.26:0/1494968511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 msgr2=0x55dfb6332150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6332150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 -- 172.21.15.26:0/1494968511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa844193480 con 0x55dfb6328e20 2023-07-13T22:15:55.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.060+0000 7fa83affd700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6332690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa834095f10 tx=0x7fa834099550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553b5f2e04df6b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:55.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834099100 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa83409da40 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfb61d34f0 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8340b2220 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa85d80fb80 10 client.25654 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dfb6090c60 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dfb61d34f0 con 0x55dfb6328e20 2023-07-13T22:15:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.061+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa834010710 con 0x55dfb6328e20 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 10 client.25654 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 20 client.25654 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb633da90 0x55dfb633fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 4 client.25654 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfb6272eb0 con 0x55dfb633da90 2023-07-13T22:15:55.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.062+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb633da90 0x55dfb633fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.063+0000 7fa822ffd700 10 client.25654 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.063+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb633da90 0x55dfb633fea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3501ef97502a81f7 server_cookie=339877186f04aece in_seq=0 out_seq=0 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.064+0000 7fa822ffd700 1 -- 172.21.15.26:0/1494968511 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dfb6272eb0 con 0x55dfb633da90 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.064+0000 7fa822ffd700 10 client.25654 handle_command_reply: tid=0 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.064+0000 7fa85d80fb80 1 client.25654 shutdown 2023-07-13T22:15:55.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.064+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 msgr2=0x55dfb6332690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.064+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6332690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa834095f10 tx=0x7fa834099550 comp rx=0 tx=0).stop 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 shutdown_connections 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6332150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb633da90 0x55dfb633fea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb63396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1494968511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6332690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 >> 172.21.15.26:0/1494968511 conn(0x55dfb631dda0 msgr2=0x55dfb63228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa822ffd700 0 client.25654 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:55.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 shutdown_connections 2023-07-13T22:15:55.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.065+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1494968511 wait complete. 2023-07-13T22:15:55.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.066+0000 7fa85d80fb80 20 client.25654 trim_cache size 0 max 16384 2023-07-13T22:15:55.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.104+0000 7fa85d80fb80 1 Processor -- start 2023-07-13T22:15:55.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.104+0000 7fa85d80fb80 1 -- start start 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb633fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6329c70 0x55dfb6339fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6323ed0 con 0x55dfb63284d0 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6331cc0 con 0x55dfb6328e20 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.105+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6324cc0 con 0x55dfb6329c70 2023-07-13T22:15:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.107+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.107+0000 7fa83affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51602/0 (socket says 172.21.15.26:51602) 2023-07-13T22:15:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.107+0000 7fa83affd700 1 -- 172.21.15.26:0/3786474323 learned_addr learned my addr 172.21.15.26:0/3786474323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.107+0000 7fa843fff700 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6329c70 0x55dfb6339fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.107+0000 7fa8437fe700 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb633fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.108+0000 7fa83affd700 1 -- 172.21.15.26:0/3786474323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6329c70 msgr2=0x55dfb6339fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.108+0000 7fa83affd700 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6329c70 0x55dfb6339fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.108+0000 7fa83affd700 1 -- 172.21.15.26:0/3786474323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 msgr2=0x55dfb633fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.108+0000 7fa83affd700 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb633fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.108+0000 7fa83affd700 1 -- 172.21.15.26:0/3786474323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfb60ce250 con 0x55dfb6328e20 2023-07-13T22:15:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.112+0000 7fa83affd700 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa834099710 tx=0x7fa834000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb33aca62e4c3ae8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.114+0000 7fa83a7fc700 1 -- 172.21.15.26:0/3786474323 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8340b2220 con 0x55dfb6328e20 2023-07-13T22:15:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.114+0000 7fa83a7fc700 1 -- 172.21.15.26:0/3786474323 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa83409da40 con 0x55dfb6328e20 2023-07-13T22:15:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.114+0000 7fa83a7fc700 1 -- 172.21.15.26:0/3786474323 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834095960 con 0x55dfb6328e20 2023-07-13T22:15:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 -- 172.21.15.26:0/3786474323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 msgr2=0x55dfb63395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa834099710 tx=0x7fa834000bc0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 -- 172.21.15.26:0/3786474323 shutdown_connections 2023-07-13T22:15:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb633fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6329c70 0x55dfb6339fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/3786474323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6328e20 0x55dfb63395f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.115+0000 7fa85d80fb80 1 -- 172.21.15.26:0/3786474323 >> 172.21.15.26:0/3786474323 conn(0x55dfb631dda0 msgr2=0x55dfb6330e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.116+0000 7fa85d80fb80 1 -- 172.21.15.26:0/3786474323 shutdown_connections 2023-07-13T22:15:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.116+0000 7fa85d80fb80 1 -- 172.21.15.26:0/3786474323 wait complete. 2023-07-13T22:15:55.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.116+0000 7fa85d80fb80 1 -- start start 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 Processor -- start 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6335de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb6338b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6339090 con 0x55dfb63284d0 2023-07-13T22:15:55.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6339200 con 0x55dfb6329c70 2023-07-13T22:15:55.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfb6339370 con 0x55dfb6328e20 2023-07-13T22:15:55.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.117+0000 7fa85d80fb80 10 client.0 _ll_register_callbacks cb 0x7fa834010e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:55.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.118+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.118+0000 7fa8437fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56114/0 (socket says 172.21.15.26:56114) 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.118+0000 7fa8437fe700 1 -- 172.21.15.26:0/1029837565 learned_addr learned my addr 172.21.15.26:0/1029837565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.118+0000 7fa843fff700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb6338b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 -- 172.21.15.26:0/1029837565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 msgr2=0x55dfb6335de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6335de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 -- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 msgr2=0x55dfb6338b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb6338b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 -- 172.21.15.26:0/1029837565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfb6090c60 con 0x55dfb63284d0 2023-07-13T22:15:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.119+0000 7fa8437fe700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b94f3d851e0b4c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:55.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.120+0000 7fa822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.120+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0182e0 con 0x55dfb63284d0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.120+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa82c018440 con 0x55dfb63284d0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.120+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa834099370 con 0x55dfb63284d0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.121+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82c018620 con 0x55dfb63284d0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.121+0000 7fa85d80fb80 10 client.15573 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.121+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dfb60a62b0 con 0x55dfb63284d0 2023-07-13T22:15:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.121+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa834099370 con 0x55dfb63284d0 2023-07-13T22:15:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa82c0164c0 con 0x55dfb63284d0 2023-07-13T22:15:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 10 client.15573 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 10 client.15573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 20 client.15573 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb6392a90 0x55dfb6394ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 4 client.15573 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dfb6235050 con 0x55dfb6392a90 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.122+0000 7fa83affd700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb6392a90 0x55dfb6394ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa83affd700 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb6392a90 0x55dfb6394ea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9798b66af9f679 server_cookie=678f99746b0b2f0 in_seq=0 out_seq=0 2023-07-13T22:15:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa822ffd700 10 client.15573 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa822ffd700 1 -- 172.21.15.26:0/1029837565 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dfb6235050 con 0x55dfb6392a90 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa822ffd700 10 client.15573 handle_command_reply: tid=0 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 client.15573 shutdown 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 msgr2=0x55dfb6335820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c018f90 comp rx=0 tx=0).stop 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 shutdown_connections 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dfb63284d0 0x55dfb6335820 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c018f90 comp rx=0 tx=0).stop 2023-07-13T22:15:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55dfb6392a90 0x55dfb6394ea0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dfb6329c70 0x55dfb6338b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 --2- 172.21.15.26:0/1029837565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dfb6328e20 0x55dfb6335de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 >> 172.21.15.26:0/1029837565 conn(0x55dfb631dda0 msgr2=0x55dfb6339fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.124+0000 7fa822ffd700 0 client.15573 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.125+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 shutdown_connections 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.125+0000 7fa85d80fb80 1 -- 172.21.15.26:0/1029837565 wait complete. 2023-07-13T22:15:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.125+0000 7fa85d80fb80 20 client.15573 trim_cache size 0 max 16384 2023-07-13T22:15:55.117 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:15:55.117 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:15:55.117 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:15:55.117 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:15:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 msgr2=0x7fa8300a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 0x7fa8300a3800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa83c018bb0 tx=0x7fa83c019d90 comp rx=0 tx=0).stop 2023-07-13T22:15:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 msgr2=0x7fa844071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa83400d950 tx=0x7fa83400dd10 comp rx=0 tx=0).stop 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 shutdown_connections 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8300a1350 0x7fa8300a3800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8440fe350 0x7fa844070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa844103670 0x7fa8440715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 --2- 172.21.15.26:0/755821758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa844100cb0 0x7fa844071070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.127+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 >> 172.21.15.26:0/755821758 conn(0x7fa8440fa000 msgr2=0x7fa8440fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.128+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 shutdown_connections 2023-07-13T22:15:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.128+0000 7fa849dff700 1 -- 172.21.15.26:0/755821758 wait complete. 2023-07-13T22:15:55.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:15:55.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:15:55.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:54 smithi117 ceph-mon[129074]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 329 MiB/s wr, 102 op/s 2023-07-13T22:15:55.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:54 smithi121 ceph-mon[129187]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 329 MiB/s wr, 102 op/s 2023-07-13T22:15:55.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/2599328578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94105790 msgr2=0x7f5c94107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/2599328578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94105790 0x7f5c94107bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00b0a0 tx=0x7f5c8c00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/2599328578 shutdown_connections 2023-07-13T22:15:55.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/2599328578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94105790 0x7f5c94107bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/2599328578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94100550 0x7f5c94102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/2599328578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94102e70 0x7f5c94105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.614+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/2599328578 >> 172.21.15.26:0/2599328578 conn(0x7f5c940f9fd0 msgr2=0x7f5c940fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:55.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.615+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/2599328578 shutdown_connections 2023-07-13T22:15:55.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.615+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/2599328578 wait complete. 2023-07-13T22:15:55.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.616+0000 7f5c9c1ff700 1 Processor -- start 2023-07-13T22:15:55.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 -- start start 2023-07-13T22:15:55.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 0x7f5c940ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94105790 0x7f5c940fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940ff1e0 con 0x7f5c94105790 2023-07-13T22:15:55.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940ff350 con 0x7f5c94100550 2023-07-13T22:15:55.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.617+0000 7f5c9c1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9419f960 con 0x7f5c94102e70 2023-07-13T22:15:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35878/0 (socket says 172.21.15.26:35878) 2023-07-13T22:15:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 -- 172.21.15.26:0/866792245 learned_addr learned my addr 172.21.15.26:0/866792245 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9a79c700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94105790 0x7f5c940fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c99f9b700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 0x7f5c940ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 -- 172.21.15.26:0/866792245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 msgr2=0x7f5c940ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 0x7f5c940ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 -- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94105790 msgr2=0x7f5c940fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94105790 0x7f5c940fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c9979a700 1 -- 172.21.15.26:0/866792245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009d00 con 0x7f5c94102e70 2023-07-13T22:15:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.618+0000 7f5c99f9b700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 0x7f5c940ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:15:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c9979a700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d950 tx=0x7f5c9000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90009b40 con 0x7f5c94102e70 2023-07-13T22:15:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c90009ca0 con 0x7f5c94102e70 2023-07-13T22:15:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000f620 con 0x7f5c94102e70 2023-07-13T22:15:55.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9419fc40 con 0x7f5c94102e70 2023-07-13T22:15:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.619+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c941a0100 con 0x7f5c94102e70 2023-07-13T22:15:55.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.621+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940fbb40 con 0x7f5c94102e70 2023-07-13T22:15:55.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.623+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5c90010460 con 0x7f5c94102e70 2023-07-13T22:15:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.626+0000 7f5c8affd700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 0x7f5c800a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.627+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c900cce70 con 0x7f5c94102e70 2023-07-13T22:15:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.627+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c90096570 con 0x7f5c94102e70 2023-07-13T22:15:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.630+0000 7f5c99f9b700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 0x7f5c800a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:55.631+0000 7f5c99f9b700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 0x7f5c800a36d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009cb0 tx=0x7f5c84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:56.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.290+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c9402c850 con 0x7f5c94102e70 2023-07-13T22:15:56.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.292+0000 7f5c8affd700 1 -- 172.21.15.26:0/866792245 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f5c90016090 con 0x7f5c94102e70 2023-07-13T22:15:56.297 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:15:56.297 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:15:26.683417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:15:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.309+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 msgr2=0x7f5c800a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 0x7f5c800a36d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009cb0 tx=0x7f5c84009260 comp rx=0 tx=0).stop 2023-07-13T22:15:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 msgr2=0x7f5c940fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d950 tx=0x7f5c9000dd10 comp rx=0 tx=0).stop 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 shutdown_connections 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c800a1220 0x7f5c800a36d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94105790 0x7f5c940fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c94100550 0x7f5c940ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 --2- 172.21.15.26:0/866792245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c94102e70 0x7f5c940fe500 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 >> 172.21.15.26:0/866792245 conn(0x7f5c940f9fd0 msgr2=0x7f5c941011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.310+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 shutdown_connections 2023-07-13T22:15:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:56.312+0000 7f5c9c1ff700 1 -- 172.21.15.26:0/866792245 wait complete. 2023-07-13T22:15:56.309 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2023-07-13T22:15:56.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:15:56.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:56 smithi026 ceph-mon[95647]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 301 MiB/s wr, 92 op/s 2023-07-13T22:15:56.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:56 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/866792245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.062+0000 7f732c3ff700 1 -- 172.21.15.26:0/4090774374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732410bd60 msgr2=0x7f732410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.062+0000 7f732c3ff700 1 --2- 172.21.15.26:0/4090774374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732410bd60 0x7f732410e1f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73140098f0 tx=0x7f7314009c00 comp rx=0 tx=0).stop 2023-07-13T22:15:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.063+0000 7f732c3ff700 1 -- 172.21.15.26:0/4090774374 shutdown_connections 2023-07-13T22:15:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.063+0000 7f732c3ff700 1 --2- 172.21.15.26:0/4090774374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f732410bd60 0x7f732410e1f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.063+0000 7f732c3ff700 1 --2- 172.21.15.26:0/4090774374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7324070ee0 0x7f73240712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.063+0000 7f732c3ff700 1 --2- 172.21.15.26:0/4090774374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324071830 0x7f732410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.063+0000 7f732c3ff700 1 -- 172.21.15.26:0/4090774374 >> 172.21.15.26:0/4090774374 conn(0x7f732406ca90 msgr2=0x7f732406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.064+0000 7f732c3ff700 1 -- 172.21.15.26:0/4090774374 shutdown_connections 2023-07-13T22:15:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.064+0000 7f732c3ff700 1 -- 172.21.15.26:0/4090774374 wait complete. 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.065+0000 7f732c3ff700 1 Processor -- start 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 -- start start 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 0x7f7324116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 0x7f73241167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73241b2520 0x7f73241b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241b4ed0 con 0x7f7324070ee0 2023-07-13T22:15:57.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241b5010 con 0x7f73241b2520 2023-07-13T22:15:57.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.066+0000 7f732c3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241b51d0 con 0x7f732410bd60 2023-07-13T22:15:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.071+0000 7f732a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 0x7f7324116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.071+0000 7f732a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 0x7f7324116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56152/0 (socket says 172.21.15.26:56152) 2023-07-13T22:15:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.071+0000 7f732a19b700 1 -- 172.21.15.26:0/1822382067 learned_addr learned my addr 172.21.15.26:0/1822382067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.071+0000 7f732999a700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 0x7f73241167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.072+0000 7f732a99c700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73241b2520 0x7f73241b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.073+0000 7f732999a700 1 -- 172.21.15.26:0/1822382067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73241b2520 msgr2=0x7f73241b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.073+0000 7f732999a700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73241b2520 0x7f73241b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.073+0000 7f732999a700 1 -- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 msgr2=0x7f7324116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.073+0000 7f732999a700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 0x7f7324116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.073+0000 7f732999a700 1 -- 172.21.15.26:0/1822382067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009520 con 0x7f732410bd60 2023-07-13T22:15:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.074+0000 7f732999a700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 0x7f73241167b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7324071ce0 tx=0x7f732000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.075+0000 7f731b7fe700 1 -- 172.21.15.26:0/1822382067 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320004440 con 0x7f732410bd60 2023-07-13T22:15:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.075+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73241b54b0 con 0x7f732410bd60 2023-07-13T22:15:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.075+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73241b5a40 con 0x7f732410bd60 2023-07-13T22:15:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.078+0000 7f731b7fe700 1 -- 172.21.15.26:0/1822382067 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7320003b40 con 0x7f732410bd60 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.079+0000 7f731b7fe700 1 -- 172.21.15.26:0/1822382067 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000fc30 con 0x7f732410bd60 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 Processor -- start 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 -- start start 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d6558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d6570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d55acb0 con 0x55d60d655e20 2023-07-13T22:15:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d5ada40 con 0x55d60d656c70 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.082+0000 7f733fd12b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d5a6ed0 con 0x55d60d6554d0 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.084+0000 7f732a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.084+0000 7f732a19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56160/0 (socket says 172.21.15.26:56160) 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.084+0000 7f732a19b700 1 -- 172.21.15.26:0/2927428275 learned_addr learned my addr 172.21.15.26:0/2927428275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.084+0000 7f732a99c700 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d6558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.085+0000 7f732a99c700 1 -- 172.21.15.26:0/2927428275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 msgr2=0x55d60d6570e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.085+0000 7f732a99c700 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d6570e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.085+0000 7f732a99c700 1 -- 172.21.15.26:0/2927428275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 msgr2=0x55d60d6566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.085+0000 7f732a99c700 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.085+0000 7f732a99c700 1 -- 172.21.15.26:0/2927428275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d60d3d4de0 con 0x55d60d6554d0 2023-07-13T22:15:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.086+0000 7f732a99c700 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d6558e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f731400fad0 tx=0x7f731400bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7064685c5e6f3bc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.094+0000 7f7318bfe700 1 -- 172.21.15.26:0/2927428275 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401c040 con 0x55d60d6554d0 2023-07-13T22:15:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.094+0000 7f7318bfe700 1 -- 172.21.15.26:0/2927428275 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f731400f720 con 0x55d60d6554d0 2023-07-13T22:15:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.094+0000 7f7318bfe700 1 -- 172.21.15.26:0/2927428275 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314005570 con 0x55d60d6554d0 2023-07-13T22:15:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.112+0000 7f733fd12b80 1 -- 172.21.15.26:0/2927428275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 msgr2=0x55d60d6558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.113+0000 7f733fd12b80 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d6558e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f731400fad0 tx=0x7f731400bf70 comp rx=0 tx=0).stop 2023-07-13T22:15:57.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.123+0000 7f733fd12b80 1 -- 172.21.15.26:0/2927428275 shutdown_connections 2023-07-13T22:15:57.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.123+0000 7f733fd12b80 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d6570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.123+0000 7f733fd12b80 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d6558e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.123+0000 7f733fd12b80 1 --2- 172.21.15.26:0/2927428275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.123+0000 7f733fd12b80 1 -- 172.21.15.26:0/2927428275 >> 172.21.15.26:0/2927428275 conn(0x55d60d64ada0 msgr2=0x55d60d6501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.149+0000 7f731b7fe700 1 -- 172.21.15.26:0/1822382067 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f732000f480 con 0x7f732410bd60 2023-07-13T22:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.151+0000 7f731b7fe700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 0x7f73100a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.152+0000 7f731b7fe700 1 -- 172.21.15.26:0/1822382067 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f73200cc700 con 0x7f732410bd60 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.162+0000 7f733fd12b80 1 -- 172.21.15.26:0/2927428275 shutdown_connections 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.162+0000 7f733fd12b80 1 -- 172.21.15.26:0/2927428275 wait complete. 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.163+0000 7f733fd12b80 1 -- start start 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.163+0000 7f733fd12b80 1 Processor -- start 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d655e20 0x55d60d65f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d666720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d546ac0 con 0x55d60d656c70 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d5467c0 con 0x55d60d655e20 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d42a920 con 0x55d60d6554d0 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.164+0000 7f733fd12b80 10 client.0 _ll_register_callbacks cb 0x55d60d544760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.165+0000 7f732a19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.165+0000 7f732a19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35908/0 (socket says 172.21.15.26:35908) 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.165+0000 7f732a19b700 1 -- 172.21.15.26:0/881633216 learned_addr learned my addr 172.21.15.26:0/881633216 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 0x7f73100a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 -- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d655e20 msgr2=0x55d60d65f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d655e20 0x55d60d65f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 -- 172.21.15.26:0/881633216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 msgr2=0x55d60d666720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d666720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 -- 172.21.15.26:0/881633216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7324117640 con 0x55d60d6554d0 2023-07-13T22:15:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.171+0000 7f732a19b700 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f731c01a790 tx=0x7f731c01ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f27bb0bb63c806 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.183+0000 7f732a19b700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 0x7f73100a3700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f731c022000 tx=0x7f731c013000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.189+0000 7f730dffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.189+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00e040 con 0x55d60d6554d0 2023-07-13T22:15:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.189+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f731c01b920 con 0x55d60d6554d0 2023-07-13T22:15:57.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:56 smithi117 ceph-mon[129074]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 301 MiB/s wr, 92 op/s 2023-07-13T22:15:57.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:56 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/866792245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:57.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.197+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d60d5004f0 con 0x55d60d6554d0 2023-07-13T22:15:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.235+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f731c01bc00 con 0x55d60d6554d0 2023-07-13T22:15:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.245+0000 7f733fd12b80 10 client.25675 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.245+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d60d3bdc60 con 0x55d60d6554d0 2023-07-13T22:15:57.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:56 smithi121 ceph-mon[129187]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 301 MiB/s wr, 92 op/s 2023-07-13T22:15:57.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:56 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/866792245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:15:57.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.247+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d60d5004f0 con 0x55d60d6554d0 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.247+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f731c0233d0 con 0x55d60d6554d0 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 10 client.25675 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 10 client.25675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 20 client.25675 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d68d0c0 0x55d60d68f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 4 client.25675 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:57.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.257+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d60d59feb0 con 0x55d60d68d0c0 2023-07-13T22:15:57.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.258+0000 7f732999a700 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d68d0c0 0x55d60d68f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.283+0000 7f732999a700 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d68d0c0 0x55d60d68f4d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4b8f5c43b3a88a1d server_cookie=fde9638dd36c47a2 in_seq=0 out_seq=0 2023-07-13T22:15:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.286+0000 7f730dffb700 10 client.25675 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.295+0000 7f730dffb700 1 -- 172.21.15.26:0/881633216 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d60d59feb0 con 0x55d60d68d0c0 2023-07-13T22:15:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.295+0000 7f730dffb700 10 client.25675 handle_command_reply: tid=0 2023-07-13T22:15:57.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.295+0000 7f733fd12b80 1 client.25675 shutdown 2023-07-13T22:15:57.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.310+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 msgr2=0x55d60d65f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.310+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f731c01a790 tx=0x7f731c01ab50 comp rx=0 tx=0).stop 2023-07-13T22:15:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 shutdown_connections 2023-07-13T22:15:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d68d0c0 0x55d60d68f4d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d666720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65f1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 --2- 172.21.15.26:0/881633216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d655e20 0x55d60d65f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.339+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 >> 172.21.15.26:0/881633216 conn(0x55d60d64ada0 msgr2=0x55d60d64f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.340+0000 7f730dffb700 0 client.25675 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.340+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 shutdown_connections 2023-07-13T22:15:57.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.340+0000 7f733fd12b80 1 -- 172.21.15.26:0/881633216 wait complete. 2023-07-13T22:15:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.341+0000 7f733fd12b80 20 client.25675 trim_cache size 0 max 16384 2023-07-13T22:15:57.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.371+0000 7f733fd12b80 1 Processor -- start 2023-07-13T22:15:57.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.378+0000 7f733fd12b80 1 -- start start 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d64aa00 con 0x55d60d655e20 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d6514f0 con 0x55d60d656c70 2023-07-13T22:15:57.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.379+0000 7f733fd12b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d64f490 con 0x55d60d6554d0 2023-07-13T22:15:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.380+0000 7f732a19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.380+0000 7f732a99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.380+0000 7f732a99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51662/0 (socket says 172.21.15.26:51662) 2023-07-13T22:15:57.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.380+0000 7f732a99c700 1 -- 172.21.15.26:0/4237312049 learned_addr learned my addr 172.21.15.26:0/4237312049 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.382+0000 7f732999a700 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 -- 172.21.15.26:0/4237312049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 msgr2=0x55d60d65d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:57.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 -- 172.21.15.26:0/4237312049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 msgr2=0x55d60d6666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:57.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 -- 172.21.15.26:0/4237312049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d60d3fb250 con 0x55d60d656c70 2023-07-13T22:15:57.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.383+0000 7f732a99c700 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7314009810 tx=0x7f7314005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e286bdaa6e1015d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.384+0000 7f73193ff700 1 -- 172.21.15.26:0/4237312049 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401c070 con 0x55d60d656c70 2023-07-13T22:15:57.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.384+0000 7f73193ff700 1 -- 172.21.15.26:0/4237312049 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f731400ba20 con 0x55d60d656c70 2023-07-13T22:15:57.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.384+0000 7f73193ff700 1 -- 172.21.15.26:0/4237312049 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314004930 con 0x55d60d656c70 2023-07-13T22:15:57.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.385+0000 7f733fd12b80 1 -- 172.21.15.26:0/4237312049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 msgr2=0x55d60d659b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.385+0000 7f733fd12b80 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7314009810 tx=0x7f7314005b40 comp rx=0 tx=0).stop 2023-07-13T22:15:57.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.392+0000 7f733fd12b80 1 -- 172.21.15.26:0/4237312049 shutdown_connections 2023-07-13T22:15:57.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.392+0000 7f733fd12b80 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d656c70 0x55d60d659b70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.392+0000 7f733fd12b80 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d6554d0 0x55d60d65d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.392+0000 7f733fd12b80 1 --2- 172.21.15.26:0/4237312049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d655e20 0x55d60d6666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.392+0000 7f733fd12b80 1 -- 172.21.15.26:0/4237312049 >> 172.21.15.26:0/4237312049 conn(0x55d60d64ada0 msgr2=0x55d60d65de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.396+0000 7f733fd12b80 1 -- 172.21.15.26:0/4237312049 shutdown_connections 2023-07-13T22:15:57.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.397+0000 7f733fd12b80 1 -- 172.21.15.26:0/4237312049 wait complete. 2023-07-13T22:15:57.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.398+0000 7f733fd12b80 1 -- start start 2023-07-13T22:15:57.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.399+0000 7f733fd12b80 1 Processor -- start 2023-07-13T22:15:57.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.401+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d6554d0 0x55d60d662910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d655e20 0x55d60d662e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d6661d0 con 0x55d60d656c70 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d666340 con 0x55d60d6554d0 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d60d666500 con 0x55d60d655e20 2023-07-13T22:15:57.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.402+0000 7f733fd12b80 10 client.0 _ll_register_callbacks cb 0x7f731c0079e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:57.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.406+0000 7f732a99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.407+0000 7f732a99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56194/0 (socket says 172.21.15.26:56194) 2023-07-13T22:15:57.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.407+0000 7f732a99c700 1 -- 172.21.15.26:0/3585562307 learned_addr learned my addr 172.21.15.26:0/3585562307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.408+0000 7f732a99c700 1 -- 172.21.15.26:0/3585562307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d655e20 msgr2=0x55d60d662e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:57.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.408+0000 7f732a99c700 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d655e20 0x55d60d662e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.408+0000 7f732a99c700 1 -- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d6554d0 msgr2=0x55d60d662910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:57.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.408+0000 7f732a99c700 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d6554d0 0x55d60d662910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.408+0000 7f732a99c700 1 -- 172.21.15.26:0/3585562307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d60d3d4de0 con 0x55d60d656c70 2023-07-13T22:15:57.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.409+0000 7f732a99c700 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f731400b3b0 tx=0x7f731400b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031c794478824e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.410+0000 7f730dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:15:57.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.413+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401c050 con 0x55d60d656c70 2023-07-13T22:15:57.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.413+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f731402bb30 con 0x55d60d656c70 2023-07-13T22:15:57.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.417+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f731400f720 con 0x55d60d656c70 2023-07-13T22:15:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.434+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7314005720 con 0x55d60d656c70 2023-07-13T22:15:57.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.453+0000 7f733fd12b80 10 client.15594 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:57.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.453+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d60d3d32b0 con 0x55d60d656c70 2023-07-13T22:15:57.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.466+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f731400f720 con 0x55d60d656c70 2023-07-13T22:15:57.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.466+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f731403b070 con 0x55d60d656c70 2023-07-13T22:15:57.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 10 client.15594 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:15:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 20 client.15594 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d6e1cf0 0x55d60d6e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 4 client.15594 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.476+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d60d562050 con 0x55d60d6e1cf0 2023-07-13T22:15:57.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.497+0000 7f732a19b700 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d6e1cf0 0x55d60d6e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.504+0000 7f732a19b700 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d6e1cf0 0x55d60d6e40d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=984b50028fb9d6a server_cookie=8b351da76b89a341 in_seq=0 out_seq=0 2023-07-13T22:15:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.518+0000 7f730dffb700 10 client.15594 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.518+0000 7f730dffb700 1 -- 172.21.15.26:0/3585562307 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d60d562050 con 0x55d60d6e1cf0 2023-07-13T22:15:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.518+0000 7f730dffb700 10 client.15594 handle_command_reply: tid=0 2023-07-13T22:15:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.518+0000 7f733fd12b80 1 client.15594 shutdown 2023-07-13T22:15:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 msgr2=0x55d60d665c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f731400b3b0 tx=0x7f731400b9b0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 shutdown_connections 2023-07-13T22:15:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55d60d6e1cf0 0x55d60d6e40d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d60d656c70 0x55d60d665c90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d60d6554d0 0x55d60d662910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 --2- 172.21.15.26:0/3585562307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d60d655e20 0x55d60d662e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 >> 172.21.15.26:0/3585562307 conn(0x55d60d64ada0 msgr2=0x55d60d65de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.570+0000 7f730dffb700 0 client.15594 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:15:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.571+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 shutdown_connections 2023-07-13T22:15:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.571+0000 7f733fd12b80 1 -- 172.21.15.26:0/3585562307 wait complete. 2023-07-13T22:15:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.572+0000 7f733fd12b80 20 client.15594 trim_cache size 0 max 16384 2023-07-13T22:15:57.561 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 msgr2=0x7f73100a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 0x7f73100a3700 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f731c022000 tx=0x7f731c013000 comp rx=0 tx=0).stop 2023-07-13T22:15:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 msgr2=0x7f73241167b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 0x7f73241167b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7324071ce0 tx=0x7f732000da60 comp rx=0 tx=0).stop 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 shutdown_connections 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f73100a1250 0x7f73100a3700 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7324070ee0 0x7f7324116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f73241b2520 0x7f73241b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 --2- 172.21.15.26:0/1822382067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f732410bd60 0x7f73241167b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.582+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 >> 172.21.15.26:0/1822382067 conn(0x7f732406ca90 msgr2=0x7f732406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.583+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 shutdown_connections 2023-07-13T22:15:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.585+0000 7f732c3ff700 1 -- 172.21.15.26:0/1822382067 wait complete. 2023-07-13T22:15:57.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:15:57.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 -- 172.21.15.26:0/696217915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e01061a0 msgr2=0x7f69e01065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 --2- 172.21.15.26:0/696217915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e01061a0 0x7f69e01065b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009990 tx=0x7f69d0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 -- 172.21.15.26:0/696217915 shutdown_connections 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 --2- 172.21.15.26:0/696217915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107d20 0x7f69e010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 --2- 172.21.15.26:0/696217915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e01061a0 0x7f69e01065b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 --2- 172.21.15.26:0/696217915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e0107390 0x7f69e01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.986+0000 7f69e49ff700 1 -- 172.21.15.26:0/696217915 >> 172.21.15.26:0/696217915 conn(0x7f69e00751c0 msgr2=0x7f69e0077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.987+0000 7f69e49ff700 1 -- 172.21.15.26:0/696217915 shutdown_connections 2023-07-13T22:15:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.987+0000 7f69e49ff700 1 -- 172.21.15.26:0/696217915 wait complete. 2023-07-13T22:15:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.989+0000 7f69e49ff700 1 Processor -- start 2023-07-13T22:15:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.990+0000 7f69e49ff700 1 -- start start 2023-07-13T22:15:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.990+0000 7f69e49ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.991+0000 7f69e49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 0x7f69e006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.991+0000 7f69e49ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e0107d20 0x7f69e019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.991+0000 7f69e49ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00723b0 con 0x7f69e01061a0 2023-07-13T22:15:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.991+0000 7f69e49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0072520 con 0x7f69e0107390 2023-07-13T22:15:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.991+0000 7f69e49ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e019b560 con 0x7f69e0107d20 2023-07-13T22:15:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.993+0000 7f69de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.993+0000 7f69de59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56206/0 (socket says 172.21.15.26:56206) 2023-07-13T22:15:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.993+0000 7f69de59c700 1 -- 172.21.15.26:0/3682318758 learned_addr learned my addr 172.21.15.26:0/3682318758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 0x7f69e006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69ded9d700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e0107d20 0x7f69e019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69de59c700 1 -- 172.21.15.26:0/3682318758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e0107d20 msgr2=0x7f69e019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69de59c700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e0107d20 0x7f69e019b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69de59c700 1 -- 172.21.15.26:0/3682318758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 msgr2=0x7f69e006f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69de59c700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 0x7f69e006f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.994+0000 7f69de59c700 1 -- 172.21.15.26:0/3682318758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00095f0 con 0x7f69e01061a0 2023-07-13T22:15:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.995+0000 7f69de59c700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f69d000b5c0 tx=0x7f69d0003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.997+0000 7f69cf7fe700 1 -- 172.21.15.26:0/3682318758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d001d070 con 0x7f69e01061a0 2023-07-13T22:15:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.997+0000 7f69cf7fe700 1 -- 172.21.15.26:0/3682318758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d000f460 con 0x7f69e01061a0 2023-07-13T22:15:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.997+0000 7f69cf7fe700 1 -- 172.21.15.26:0/3682318758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0017650 con 0x7f69e01061a0 2023-07-13T22:15:57.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.998+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 0x7f69e006f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:15:57.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.999+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e019b7e0 con 0x7f69e01061a0 2023-07-13T22:15:57.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:57.999+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e019bca0 con 0x7f69e01061a0 2023-07-13T22:15:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.024+0000 7f69f83bcb80 1 Processor -- start 2023-07-13T22:15:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.029+0000 7f69cf7fe700 1 -- 172.21.15.26:0/3682318758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f69d00052d0 con 0x7f69e01061a0 2023-07-13T22:15:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.031+0000 7f69cf7fe700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 0x7f69c80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.032+0000 7f69cf7fe700 1 -- 172.21.15.26:0/3682318758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69d00cd9a0 con 0x7f69e01061a0 2023-07-13T22:15:58.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.036+0000 7f69f83bcb80 1 -- start start 2023-07-13T22:15:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.036+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.036+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec14e20 0x5575eec156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.037+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec15c70 0x5575eec160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.037+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eeb19cb0 con 0x5575eec14e20 2023-07-13T22:15:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.037+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eeb6ca40 con 0x5575eec15c70 2023-07-13T22:15:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.037+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eeb65ed0 con 0x5575eec144d0 2023-07-13T22:15:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35958/0 (socket says 172.21.15.26:35958) 2023-07-13T22:15:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 -- 172.21.15.26:0/689280013 learned_addr learned my addr 172.21.15.26:0/689280013 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69ded9d700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 0x7f69c80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 -- 172.21.15.26:0/689280013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec15c70 msgr2=0x5575eec160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec15c70 0x5575eec160e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 -- 172.21.15.26:0/689280013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec14e20 msgr2=0x5575eec156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec14e20 0x5575eec156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 -- 172.21.15.26:0/689280013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ee993de0 con 0x5575eec144d0 2023-07-13T22:15:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.048+0000 7f69de59c700 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f69d0011040 tx=0x7f69d0097240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f31fa46d7c1f6b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.049+0000 7f69ccff9700 1 -- 172.21.15.26:0/689280013 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d00b2070 con 0x5575eec144d0 2023-07-13T22:15:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.049+0000 7f69ccff9700 1 -- 172.21.15.26:0/689280013 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d009ee20 con 0x5575eec144d0 2023-07-13T22:15:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.049+0000 7f69ccff9700 1 -- 172.21.15.26:0/689280013 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0020c10 con 0x5575eec144d0 2023-07-13T22:15:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.050+0000 7f69f83bcb80 1 -- 172.21.15.26:0/689280013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 msgr2=0x5575eec148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.050+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f69d0011040 tx=0x7f69d0097240 comp rx=0 tx=0).stop 2023-07-13T22:15:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.054+0000 7f69f83bcb80 1 -- 172.21.15.26:0/689280013 shutdown_connections 2023-07-13T22:15:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.054+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec15c70 0x5575eec160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.054+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec144d0 0x5575eec148e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.054+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/689280013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec14e20 0x5575eec156e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.054+0000 7f69f83bcb80 1 -- 172.21.15.26:0/689280013 >> 172.21.15.26:0/689280013 conn(0x5575eec09da0 msgr2=0x5575eec0f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.060+0000 7f69ded9d700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 0x7f69c80a3800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69d8008fc0 tx=0x7f69d8008580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.061+0000 7f69f83bcb80 1 -- 172.21.15.26:0/689280013 shutdown_connections 2023-07-13T22:15:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.061+0000 7f69f83bcb80 1 -- 172.21.15.26:0/689280013 wait complete. 2023-07-13T22:15:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.062+0000 7f69f83bcb80 1 -- start start 2023-07-13T22:15:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.062+0000 7f69f83bcb80 1 Processor -- start 2023-07-13T22:15:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec25350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eeb05ac0 con 0x5575eec15c70 2023-07-13T22:15:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eeb057c0 con 0x5575eec144d0 2023-07-13T22:15:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575ee9e9920 con 0x5575eec14e20 2023-07-13T22:15:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.063+0000 7f69f83bcb80 10 client.0 _ll_register_callbacks cb 0x5575eeb03760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.066+0000 7f69de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.066+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.066+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35974/0 (socket says 172.21.15.26:35974) 2023-07-13T22:15:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.066+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1561193088 learned_addr learned my addr 172.21.15.26:0/1561193088 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.068+0000 7f69ded9d700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec25350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.077+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1561193088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 msgr2=0x5575eec1ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.077+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec1ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.077+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1561193088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 msgr2=0x5575eec25350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.077+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec25350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.078+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1561193088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e019bca0 con 0x5575eec14e20 2023-07-13T22:15:58.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.079+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e3f0 tx=0x7f69d400e7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c866b5667dea6d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.080+0000 7f69c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.080+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d400eee0 con 0x5575eec14e20 2023-07-13T22:15:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.080+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69d4011880 con 0x5575eec14e20 2023-07-13T22:15:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.080+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575eeabf4f0 con 0x5575eec14e20 2023-07-13T22:15:58.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.091+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d4011a40 con 0x5575eec14e20 2023-07-13T22:15:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.091+0000 7f69f83bcb80 10 client.25687 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:58.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.091+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5575ee97cc60 con 0x5575eec14e20 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575eeabf4f0 con 0x5575eec14e20 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f69d400c070 con 0x5575eec14e20 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 10 client.25687 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 10 client.25687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 20 client.25687 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:58.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec3ac60 0x5575eec3d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 4 client.25687 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:58.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.103+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575eeb5eeb0 con 0x5575eec3ac60 2023-07-13T22:15:58.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.110+0000 7f69de59c700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec3ac60 0x5575eec3d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.126+0000 7f69de59c700 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec3ac60 0x5575eec3d070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=269a83f5ba31705c server_cookie=d629720f1d5c1180 in_seq=0 out_seq=0 2023-07-13T22:15:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.128+0000 7f69c5ffb700 10 client.25687 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:58.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.128+0000 7f69c5ffb700 1 -- 172.21.15.26:0/1561193088 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575eeb5eeb0 con 0x5575eec3ac60 2023-07-13T22:15:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.128+0000 7f69c5ffb700 10 client.25687 handle_command_reply: tid=0 2023-07-13T22:15:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.129+0000 7f69f83bcb80 1 client.25687 shutdown 2023-07-13T22:15:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.135+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 msgr2=0x5575eec1e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.135+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e3f0 tx=0x7f69d400e7b0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 shutdown_connections 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec3ac60 0x5575eec3d070 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec25350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1561193088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec1e310 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 >> 172.21.15.26:0/1561193088 conn(0x5575eec09da0 msgr2=0x5575eec0e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69c5ffb700 0 client.25687 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.147+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 shutdown_connections 2023-07-13T22:15:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.148+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1561193088 wait complete. 2023-07-13T22:15:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.171+0000 7f69f83bcb80 20 client.25687 trim_cache size 0 max 16384 2023-07-13T22:15:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.202+0000 7f69f83bcb80 1 Processor -- start 2023-07-13T22:15:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.202+0000 7f69f83bcb80 1 -- start start 2023-07-13T22:15:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.202+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec144d0 0x5575eec1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.202+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.203+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec15c70 0x5575eec1c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.203+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec15570 con 0x5575eec144d0 2023-07-13T22:15:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.203+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec0e520 con 0x5575eec14e20 2023-07-13T22:15:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.203+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec1d940 con 0x5575eec15c70 2023-07-13T22:15:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.203+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.204+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51722/0 (socket says 172.21.15.26:51722) 2023-07-13T22:15:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.204+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1574678753 learned_addr learned my addr 172.21.15.26:0/1574678753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.204+0000 7f69ded9d700 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec15c70 0x5575eec1c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.205+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1574678753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec15c70 msgr2=0x5575eec1c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.205+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec15c70 0x5575eec1c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.205+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1574678753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec144d0 msgr2=0x5575eec1bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.205+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec144d0 0x5575eec1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.205+0000 7f69ddd9b700 1 -- 172.21.15.26:0/1574678753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575ee9ba250 con 0x5575eec14e20 2023-07-13T22:15:58.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.206+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5005ac4a08cf0412 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.206+0000 7f69cd7fa700 1 -- 172.21.15.26:0/1574678753 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40095d0 con 0x5575eec14e20 2023-07-13T22:15:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.207+0000 7f69cd7fa700 1 -- 172.21.15.26:0/1574678753 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d40108a0 con 0x5575eec14e20 2023-07-13T22:15:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.207+0000 7f69cd7fa700 1 -- 172.21.15.26:0/1574678753 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40052a0 con 0x5575eec14e20 2023-07-13T22:15:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.208+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1574678753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 msgr2=0x5575eec25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.208+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d400ec60 comp rx=0 tx=0).stop 2023-07-13T22:15:58.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1574678753 shutdown_connections 2023-07-13T22:15:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec144d0 0x5575eec1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec15c70 0x5575eec1c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/1574678753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec14e20 0x5575eec25310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1574678753 >> 172.21.15.26:0/1574678753 conn(0x5575eec09da0 msgr2=0x5575eec0de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1574678753 shutdown_connections 2023-07-13T22:15:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.231+0000 7f69f83bcb80 1 -- 172.21.15.26:0/1574678753 wait complete. 2023-07-13T22:15:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.232+0000 7f69f83bcb80 1 -- start start 2023-07-13T22:15:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.232+0000 7f69f83bcb80 1 Processor -- start 2023-07-13T22:15:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec8eb10 con 0x5575eec15c70 2023-07-13T22:15:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec8ec50 con 0x5575eec144d0 2023-07-13T22:15:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575eec8ee10 con 0x5575eec14e20 2023-07-13T22:15:58.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.233+0000 7f69f83bcb80 10 client.0 _ll_register_callbacks cb 0x7f69d4009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.243+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.244+0000 7f69de59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.244+0000 7f69ddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:35996/0 (socket says 172.21.15.26:35996) 2023-07-13T22:15:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.244+0000 7f69ddd9b700 1 -- 172.21.15.26:0/2009537112 learned_addr learned my addr 172.21.15.26:0/2009537112 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:58.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.247+0000 7f69ddd9b700 1 -- 172.21.15.26:0/2009537112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 msgr2=0x5575eec252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.247+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.247+0000 7f69ddd9b700 1 -- 172.21.15.26:0/2009537112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 msgr2=0x5575eec223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.247+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.247+0000 7f69ddd9b700 1 -- 172.21.15.26:0/2009537112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575ee97cc60 con 0x5575eec14e20 2023-07-13T22:15:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.264+0000 7f69ddd9b700 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d4010d20 tx=0x7f69d4004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474da1b15632307 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.264+0000 7f69c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:15:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.264+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4010510 con 0x5575eec14e20 2023-07-13T22:15:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.264+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f69d4000e40 con 0x5575eec14e20 2023-07-13T22:15:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.265+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69d009ee20 con 0x5575eec14e20 2023-07-13T22:15:58.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.276+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d4010670 con 0x5575eec14e20 2023-07-13T22:15:58.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.278+0000 7f69f83bcb80 10 client.25693 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:58.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.278+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5575ee9922b0 con 0x5575eec14e20 2023-07-13T22:15:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.280+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d009ee20 con 0x5575eec14e20 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.280+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f69d403f070 con 0x5575eec14e20 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 10 client.25693 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 10 client.25693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 20 client.25693 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec90250 0x5575eec92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 4 client.25693 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:58.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.281+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575eeb21050 con 0x5575eec90250 2023-07-13T22:15:58.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.285+0000 7f69de59c700 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec90250 0x5575eec92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:58.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.316+0000 7f69de59c700 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec90250 0x5575eec92630 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9351dfa6fb1f7dee server_cookie=f9856da267afaa8e in_seq=0 out_seq=0 2023-07-13T22:15:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.316+0000 7f69c5ffb700 10 client.25693 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.317+0000 7f69c5ffb700 1 -- 172.21.15.26:0/2009537112 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575eeb21050 con 0x5575eec90250 2023-07-13T22:15:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.317+0000 7f69c5ffb700 10 client.25693 handle_command_reply: tid=0 2023-07-13T22:15:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.318+0000 7f69f83bcb80 1 client.25693 shutdown 2023-07-13T22:15:58.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.321+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 msgr2=0x5575eec21d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.321+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d4010d20 tx=0x7f69d4004d60 comp rx=0 tx=0).stop 2023-07-13T22:15:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 shutdown_connections 2023-07-13T22:15:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5575eec90250 0x5575eec92630 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5575eec15c70 0x5575eec223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5575eec144d0 0x5575eec252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 --2- 172.21.15.26:0/2009537112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5575eec14e20 0x5575eec21d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 >> 172.21.15.26:0/2009537112 conn(0x5575eec09da0 msgr2=0x5575eec1bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.323+0000 7f69c5ffb700 0 client.25693 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:15:58.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.330+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 shutdown_connections 2023-07-13T22:15:58.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.330+0000 7f69f83bcb80 1 -- 172.21.15.26:0/2009537112 wait complete. 2023-07-13T22:15:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.342+0000 7f69f83bcb80 20 client.25693 trim_cache size 0 max 16384 2023-07-13T22:15:58.331 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 msgr2=0x7f69c80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 0x7f69c80a3800 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69d8008fc0 tx=0x7f69d8008580 comp rx=0 tx=0).stop 2023-07-13T22:15:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 msgr2=0x7f69e006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f69d000b5c0 tx=0x7f69d0003cb0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 shutdown_connections 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f69c80a1350 0x7f69c80a3800 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69e01061a0 0x7f69e006ed20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69e0107d20 0x7f69e019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 --2- 172.21.15.26:0/3682318758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69e0107390 0x7f69e006f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 >> 172.21.15.26:0/3682318758 conn(0x7f69e00751c0 msgr2=0x7f69e00773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.346+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 shutdown_connections 2023-07-13T22:15:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:58.347+0000 7f69e49ff700 1 -- 172.21.15.26:0/3682318758 wait complete. 2023-07-13T22:15:58.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:15:58.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:15:58 smithi026 ceph-mon[95647]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 96 op/s 2023-07-13T22:15:59.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:15:58 smithi117 ceph-mon[129074]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 96 op/s 2023-07-13T22:15:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.243+0000 7fb8883ff700 1 -- 172.21.15.26:0/1512185830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 msgr2=0x7fb88010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.243+0000 7fb8883ff700 1 --2- 172.21.15.26:0/1512185830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb88010e1f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009990 tx=0x7fb87c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.244+0000 7fb8883ff700 1 -- 172.21.15.26:0/1512185830 shutdown_connections 2023-07-13T22:15:59.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.244+0000 7fb8883ff700 1 --2- 172.21.15.26:0/1512185830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb88010e1f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.244+0000 7fb8883ff700 1 --2- 172.21.15.26:0/1512185830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb880070ee0 0x7fb8800712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.244+0000 7fb8883ff700 1 --2- 172.21.15.26:0/1512185830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb880071830 0x7fb88010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.244+0000 7fb8883ff700 1 -- 172.21.15.26:0/1512185830 >> 172.21.15.26:0/1512185830 conn(0x7fb88006ca90 msgr2=0x7fb88006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.246+0000 7fb8883ff700 1 -- 172.21.15.26:0/1512185830 shutdown_connections 2023-07-13T22:15:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.246+0000 7fb8883ff700 1 -- 172.21.15.26:0/1512185830 wait complete. 2023-07-13T22:15:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.248+0000 7fb8883ff700 1 Processor -- start 2023-07-13T22:15:59.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.248+0000 7fb8883ff700 1 -- start start 2023-07-13T22:15:59.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb880071830 0x7fb880113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8801b2970 0x7fb8801b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880114660 con 0x7fb880071830 2023-07-13T22:15:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801147d0 con 0x7fb8801b2970 2023-07-13T22:15:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.250+0000 7fb8883ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801b5290 con 0x7fb88010bd60 2023-07-13T22:15:59.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.251+0000 7fb88599a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.251+0000 7fb88599a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36012/0 (socket says 172.21.15.26:36012) 2023-07-13T22:15:59.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.251+0000 7fb88599a700 1 -- 172.21.15.26:0/4261155597 learned_addr learned my addr 172.21.15.26:0/4261155597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.252+0000 7fb88599a700 1 -- 172.21.15.26:0/4261155597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8801b2970 msgr2=0x7fb8801b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.252+0000 7fb88599a700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8801b2970 0x7fb8801b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.252+0000 7fb88599a700 1 -- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb880071830 msgr2=0x7fb880113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.252+0000 7fb88599a700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb880071830 0x7fb880113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.252+0000 7fb88599a700 1 -- 172.21.15.26:0/4261155597 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0095f0 con 0x7fb88010bd60 2023-07-13T22:15:59.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.253+0000 7fb88599a700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b9c0 tx=0x7fb87800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.253+0000 7fb8777fe700 1 -- 172.21.15.26:0/4261155597 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800c9d0 con 0x7fb88010bd60 2023-07-13T22:15:59.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:15:58 smithi121 ceph-mon[129187]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 96 op/s 2023-07-13T22:15:59.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.253+0000 7fb8777fe700 1 -- 172.21.15.26:0/4261155597 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb878014440 con 0x7fb88010bd60 2023-07-13T22:15:59.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.253+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801b5570 con 0x7fb88010bd60 2023-07-13T22:15:59.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.253+0000 7fb8777fe700 1 -- 172.21.15.26:0/4261155597 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780125a0 con 0x7fb88010bd60 2023-07-13T22:15:59.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.254+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8801b5b50 con 0x7fb88010bd60 2023-07-13T22:15:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.269+0000 7fb8777fe700 1 -- 172.21.15.26:0/4261155597 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb8780145b0 con 0x7fb88010bd60 2023-07-13T22:15:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.271+0000 7fb8777fe700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 0x7fb86c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.272+0000 7fb8777fe700 1 -- 172.21.15.26:0/4261155597 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb878006e40 con 0x7fb88010bd60 2023-07-13T22:15:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.273+0000 7fb89bd7fb80 1 Processor -- start 2023-07-13T22:15:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.273+0000 7fb88619b700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 0x7fb86c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 -- start start 2023-07-13T22:15:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23ce91cb0 con 0x55b23cf8ce20 2023-07-13T22:15:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cee4a40 con 0x55b23cf8dc70 2023-07-13T22:15:59.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.274+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cedded0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51742/0 (socket says 172.21.15.26:51742) 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 -- 172.21.15.26:0/3365564202 learned_addr learned my addr 172.21.15.26:0/3365564202 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 -- 172.21.15.26:0/3365564202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 msgr2=0x55b23cf8c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf8c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 -- 172.21.15.26:0/3365564202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 msgr2=0x55b23cf8d6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf8d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.275+0000 7fb88699c700 1 -- 172.21.15.26:0/3365564202 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b23cd0bde0 con 0x55b23cf8dc70 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.276+0000 7fb88699c700 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0092e0 tx=0x7fb87c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c4f5153c5b41fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.276+0000 7fb874bfe700 1 -- 172.21.15.26:0/3365564202 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c01d070 con 0x55b23cf8dc70 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.276+0000 7fb874bfe700 1 -- 172.21.15.26:0/3365564202 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87c00bec0 con 0x55b23cf8dc70 2023-07-13T22:15:59.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.276+0000 7fb874bfe700 1 -- 172.21.15.26:0/3365564202 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0173f0 con 0x55b23cf8dc70 2023-07-13T22:15:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3365564202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 msgr2=0x55b23cf8e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0092e0 tx=0x7fb87c009f60 comp rx=0 tx=0).stop 2023-07-13T22:15:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3365564202 shutdown_connections 2023-07-13T22:15:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf8e0e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3365564202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf8d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.278+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3365564202 >> 172.21.15.26:0/3365564202 conn(0x55b23cf81da0 msgr2=0x55b23cf871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.279+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3365564202 shutdown_connections 2023-07-13T22:15:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.279+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3365564202 wait complete. 2023-07-13T22:15:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.280+0000 7fb89bd7fb80 1 -- start start 2023-07-13T22:15:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.280+0000 7fb89bd7fb80 1 Processor -- start 2023-07-13T22:15:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf961c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 0x55b23cf96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8dc70 0x55b23cf9d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23ce7dac0 con 0x55b23cf8dc70 2023-07-13T22:15:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23ce7d7c0 con 0x55b23cf8ce20 2023-07-13T22:15:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.281+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cd61920 con 0x55b23cf8c4d0 2023-07-13T22:15:59.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.282+0000 7fb89bd7fb80 10 client.0 _ll_register_callbacks cb 0x55b23ce7b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:59.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.284+0000 7fb88599a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 0x55b23cf96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.284+0000 7fb88619b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.284+0000 7fb88619b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36034/0 (socket says 172.21.15.26:36034) 2023-07-13T22:15:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.284+0000 7fb88619b700 1 -- 172.21.15.26:0/4195031846 learned_addr learned my addr 172.21.15.26:0/4195031846 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88699c700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8dc70 0x55b23cf9d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88619b700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 0x7fb86c0a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8801076e0 tx=0x7fb870006cd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88599a700 1 -- 172.21.15.26:0/4195031846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 msgr2=0x55b23cf961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88599a700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88599a700 1 -- 172.21.15.26:0/4195031846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8dc70 msgr2=0x55b23cf9d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88599a700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8dc70 0x55b23cf9d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.289+0000 7fb88599a700 1 -- 172.21.15.26:0/4195031846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880117640 con 0x55b23cf8ce20 2023-07-13T22:15:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.290+0000 7fb88599a700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 0x55b23cf96700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb878014860 tx=0x7fb878099830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfa8c96365f1d63 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.290+0000 7fb869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780c9ea0 con 0x55b23cf8ce20 2023-07-13T22:15:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb87809d320 con 0x55b23cf8ce20 2023-07-13T22:15:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b23ced6eb0 con 0x55b23cf8ce20 2023-07-13T22:15:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb87809d130 con 0x55b23cf8ce20 2023-07-13T22:15:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb89bd7fb80 10 client.25709 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.291+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b23ccf4c60 con 0x55b23cf8ce20 2023-07-13T22:15:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b23ced6eb0 con 0x55b23cf8ce20 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb8780aed80 con 0x55b23cf8ce20 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 10 client.25709 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 10 client.25709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 20 client.25709 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cfa1e50 0x55b23cfa4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 4 client.25709 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:15:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.292+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b23ce99050 con 0x55b23cfa1e50 2023-07-13T22:15:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.293+0000 7fb88619b700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cfa1e50 0x55b23cfa4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.294+0000 7fb88619b700 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cfa1e50 0x55b23cfa4260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c10b112876ff4e0 server_cookie=165eed220ac94a6d in_seq=0 out_seq=0 2023-07-13T22:15:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.294+0000 7fb869ffb700 10 client.25709 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.294+0000 7fb869ffb700 1 -- 172.21.15.26:0/4195031846 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b23ce99050 con 0x55b23cfa1e50 2023-07-13T22:15:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb869ffb700 10 client.25709 handle_command_reply: tid=0 2023-07-13T22:15:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 client.25709 shutdown 2023-07-13T22:15:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 msgr2=0x55b23cf96700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 0x55b23cf96700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb878014860 tx=0x7fb878099830 comp rx=0 tx=0).stop 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 shutdown_connections 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8dc70 0x55b23cf9d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cfa1e50 0x55b23cfa4260 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8c4d0 0x55b23cf961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/4195031846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8ce20 0x55b23cf96700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 >> 172.21.15.26:0/4195031846 conn(0x55b23cf81da0 msgr2=0x55b23cf868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.295+0000 7fb869ffb700 0 client.25709 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:59.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.296+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 shutdown_connections 2023-07-13T22:15:59.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.296+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/4195031846 wait complete. 2023-07-13T22:15:59.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.296+0000 7fb89bd7fb80 20 client.25709 trim_cache size 0 max 16384 2023-07-13T22:15:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.327+0000 7fb89bd7fb80 1 Processor -- start 2023-07-13T22:15:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.327+0000 7fb89bd7fb80 1 -- start start 2023-07-13T22:15:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8c4d0 0x55b23cfa3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 0x55b23cf9d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf90c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf973b0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf88c50 con 0x55b23cf8dc70 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.328+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf9f430 con 0x55b23cf8ce20 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.329+0000 7fb88599a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 0x55b23cf9d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.329+0000 7fb88599a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 0x55b23cf9d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36042/0 (socket says 172.21.15.26:36042) 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.329+0000 7fb88599a700 1 -- 172.21.15.26:0/3324285683 learned_addr learned my addr 172.21.15.26:0/3324285683 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.329+0000 7fb88699c700 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf90c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.330+0000 7fb88699c700 1 -- 172.21.15.26:0/3324285683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 msgr2=0x55b23cf9d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.330+0000 7fb88699c700 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 0x55b23cf9d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.330+0000 7fb88699c700 1 -- 172.21.15.26:0/3324285683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8c4d0 msgr2=0x55b23cfa3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.330+0000 7fb88699c700 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8c4d0 0x55b23cfa3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.330+0000 7fb88699c700 1 -- 172.21.15.26:0/3324285683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b23cf8e560 con 0x55b23cf8dc70 2023-07-13T22:15:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.331+0000 7fb88699c700 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf90c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0098e0 tx=0x7fb87c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cbb35e79ab3ac30 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.355+0000 7fb8753ff700 1 -- 172.21.15.26:0/3324285683 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c01d040 con 0x55b23cf8dc70 2023-07-13T22:15:59.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.355+0000 7fb8753ff700 1 -- 172.21.15.26:0/3324285683 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb87c01e040 con 0x55b23cf8dc70 2023-07-13T22:15:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.355+0000 7fb8753ff700 1 -- 172.21.15.26:0/3324285683 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c022680 con 0x55b23cf8dc70 2023-07-13T22:15:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.357+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3324285683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 msgr2=0x55b23cf90c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.357+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf90c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0098e0 tx=0x7fb87c009310 comp rx=0 tx=0).stop 2023-07-13T22:15:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.358+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3324285683 shutdown_connections 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.358+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8c4d0 0x55b23cfa3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.358+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8dc70 0x55b23cf90c70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.358+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/3324285683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8ce20 0x55b23cf9d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.358+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3324285683 >> 172.21.15.26:0/3324285683 conn(0x55b23cf81da0 msgr2=0x55b23cf94170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.359+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3324285683 shutdown_connections 2023-07-13T22:15:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.359+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/3324285683 wait complete. 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.359+0000 7fb89bd7fb80 1 -- start start 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 Processor -- start 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8dc70 0x55b23cf9c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf9ce30 con 0x55b23cf8ce20 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf9cfa0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b23cf9d160 con 0x55b23cf8dc70 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.360+0000 7fb89bd7fb80 10 client.0 _ll_register_callbacks cb 0x7fb878098eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.361+0000 7fb88619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.361+0000 7fb88619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51774/0 (socket says 172.21.15.26:51774) 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.361+0000 7fb88619b700 1 -- 172.21.15.26:0/2132615727 learned_addr learned my addr 172.21.15.26:0/2132615727 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88699c700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8dc70 0x55b23cf9c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88619b700 1 -- 172.21.15.26:0/2132615727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8dc70 msgr2=0x55b23cf9c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88619b700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8dc70 0x55b23cf9c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88619b700 1 -- 172.21.15.26:0/2132615727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 msgr2=0x55b23cf9b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88619b700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.362+0000 7fb88619b700 1 -- 172.21.15.26:0/2132615727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b23cd0bde0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.373+0000 7fb88619b700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb870009110 tx=0x7fb870023ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ead0faacd13cf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.373+0000 7fb869ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.373+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870023350 con 0x55b23cf8c4d0 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.373+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8700234b0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.373+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b23ce374f0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.375+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb870004020 con 0x55b23cf8c4d0 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.375+0000 7fb89bd7fb80 10 client.25715 fetch_fsmap learned FSMap version 28 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.375+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b23cd32250 con 0x55b23cf8c4d0 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.376+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b23ce374f0 con 0x55b23cf8c4d0 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.376+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb87001f420 con 0x55b23cf8c4d0 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 10 client.25715 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 10 client.25715 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:15:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 20 client.25715 populate_metadata read hostname 'smithi026' 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cff6c00 0x55b23cff9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 4 client.25715 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.377+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b23cd2cf70 con 0x55b23cff6c00 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.382+0000 7fb88599a700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cff6c00 0x55b23cff9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.383+0000 7fb88599a700 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cff6c00 0x55b23cff9010 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38eeb3061a9c8f6e server_cookie=913db3640ea4676e in_seq=0 out_seq=0 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.389+0000 7fb869ffb700 10 client.25715 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.389+0000 7fb869ffb700 1 -- 172.21.15.26:0/2132615727 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b23cd2cf70 con 0x55b23cff6c00 2023-07-13T22:15:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.389+0000 7fb869ffb700 10 client.25715 handle_command_reply: tid=0 2023-07-13T22:15:59.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.389+0000 7fb89bd7fb80 1 client.25715 shutdown 2023-07-13T22:15:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.393+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 msgr2=0x55b23cf9b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.393+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb870009110 tx=0x7fb870023ef0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 shutdown_connections 2023-07-13T22:15:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b23cff6c00 0x55b23cff9010 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b23cf8dc70 0x55b23cf9c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b23cf8c4d0 0x55b23cf9b3a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 --2- 172.21.15.26:0/2132615727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b23cf8ce20 0x55b23cf9b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 >> 172.21.15.26:0/2132615727 conn(0x55b23cf81da0 msgr2=0x55b23cfa3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.398+0000 7fb869ffb700 0 client.25715 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:15:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.401+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 shutdown_connections 2023-07-13T22:15:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.401+0000 7fb89bd7fb80 1 -- 172.21.15.26:0/2132615727 wait complete. 2023-07-13T22:15:59.403 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:15:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.407+0000 7fb89bd7fb80 20 client.25715 trim_cache size 0 max 16384 2023-07-13T22:15:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.415+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 msgr2=0x7fb86c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.415+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 0x7fb86c0a3750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8801076e0 tx=0x7fb870006cd0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.415+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 msgr2=0x7fb880114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.415+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb87800b9c0 tx=0x7fb87800bd80 comp rx=0 tx=0).stop 2023-07-13T22:15:59.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 shutdown_connections 2023-07-13T22:15:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb86c0a12a0 0x7fb86c0a3750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb880071830 0x7fb880113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8801b2970 0x7fb8801b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 --2- 172.21.15.26:0/4261155597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb88010bd60 0x7fb880114090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.420+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 >> 172.21.15.26:0/4261155597 conn(0x7fb88006ca90 msgr2=0x7fb88006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.421+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 shutdown_connections 2023-07-13T22:15:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.421+0000 7fb8883ff700 1 -- 172.21.15.26:0/4261155597 wait complete. 2023-07-13T22:15:59.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:15:59.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 -- 172.21.15.26:0/3224519504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c070ee0 msgr2=0x7f402c0712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 --2- 172.21.15.26:0/3224519504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c070ee0 0x7f402c0712f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4028009990 tx=0x7f4028009ca0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 -- 172.21.15.26:0/3224519504 shutdown_connections 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 --2- 172.21.15.26:0/3224519504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c070ee0 0x7f402c0712f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 --2- 172.21.15.26:0/3224519504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c10bd60 0x7f402c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 --2- 172.21.15.26:0/3224519504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 -- 172.21.15.26:0/3224519504 >> 172.21.15.26:0/3224519504 conn(0x7f402c06ca90 msgr2=0x7f402c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.944+0000 7f4033fff700 1 -- 172.21.15.26:0/3224519504 shutdown_connections 2023-07-13T22:15:59.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.945+0000 7f4033fff700 1 -- 172.21.15.26:0/3224519504 wait complete. 2023-07-13T22:15:59.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.946+0000 7f4033fff700 1 Processor -- start 2023-07-13T22:15:59.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 -- start start 2023-07-13T22:15:59.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c070ee0 0x7f402c1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 0x7f402c1abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1ac1d0 con 0x7f402c10bd60 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1ac310 con 0x7f402c071830 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.947+0000 7f4033fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1ac4a0 con 0x7f402c070ee0 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.948+0000 7f403159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.948+0000 7f403159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51798/0 (socket says 172.21.15.26:51798) 2023-07-13T22:15:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.948+0000 7f403159a700 1 -- 172.21.15.26:0/1024935467 learned_addr learned my addr 172.21.15.26:0/1024935467 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.948+0000 7f4031d9b700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c070ee0 0x7f402c1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 -- 172.21.15.26:0/1024935467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c070ee0 msgr2=0x7f402c1a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c070ee0 0x7f402c1a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 -- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 msgr2=0x7f402c1abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403259c700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 0x7f402c1abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 0x7f402c1abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 -- 172.21.15.26:0/1024935467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40280095f0 con 0x7f402c071830 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403259c700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 0x7f402c1abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:15:59.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.949+0000 7f403159a700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d900 tx=0x7f401c00dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.951+0000 7f4022ffd700 1 -- 172.21.15.26:0/1024935467 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c00b950 con 0x7f402c071830 2023-07-13T22:15:59.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.951+0000 7f4022ffd700 1 -- 172.21.15.26:0/1024935467 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c007620 con 0x7f402c071830 2023-07-13T22:15:59.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.951+0000 7f4022ffd700 1 -- 172.21.15.26:0/1024935467 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c00f3c0 con 0x7f402c071830 2023-07-13T22:15:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.951+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c1ac780 con 0x7f402c071830 2023-07-13T22:15:59.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.951+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c1acc40 con 0x7f402c071830 2023-07-13T22:15:59.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.966+0000 7f40479a0b80 1 Processor -- start 2023-07-13T22:15:59.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.969+0000 7f40479a0b80 1 -- start start 2023-07-13T22:15:59.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.969+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4337454d0 0x55e4337458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.969+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433746c70 0x55e4337470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f40479a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43364acb0 con 0x55e4337454d0 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f40479a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43369da40 con 0x55e433746c70 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f40479a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e433696ed0 con 0x55e433745e20 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36068/0 (socket says 172.21.15.26:36068) 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f4031d9b700 1 -- 172.21.15.26:0/4157106317 learned_addr learned my addr 172.21.15.26:0/4157106317 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.970+0000 7f403259c700 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4337454d0 0x55e4337458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 -- 172.21.15.26:0/4157106317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433746c70 msgr2=0x55e4337470e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433746c70 0x55e4337470e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 -- 172.21.15.26:0/4157106317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4337454d0 msgr2=0x55e4337458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4337454d0 0x55e4337458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 -- 172.21.15.26:0/4157106317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4334c4de0 con 0x55e433745e20 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4031d9b700 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f402800b680 tx=0x7f402800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a1183d87fbe1cb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.971+0000 7f4022ffd700 1 -- 172.21.15.26:0/1024935467 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f401c00bab0 con 0x7f402c071830 2023-07-13T22:15:59.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.973+0000 7f4017fff700 1 -- 172.21.15.26:0/4157106317 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402801d070 con 0x55e433745e20 2023-07-13T22:15:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.973+0000 7f4017fff700 1 -- 172.21.15.26:0/4157106317 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f402801e040 con 0x55e433745e20 2023-07-13T22:15:59.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.973+0000 7f4017fff700 1 -- 172.21.15.26:0/4157106317 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028004a40 con 0x55e433745e20 2023-07-13T22:15:59.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.975+0000 7f4022ffd700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 0x7f40180a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.976+0000 7f4022ffd700 1 -- 172.21.15.26:0/1024935467 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f401c0cc760 con 0x7f402c071830 2023-07-13T22:15:59.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.977+0000 7f40479a0b80 1 -- 172.21.15.26:0/4157106317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 msgr2=0x55e4337466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:15:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.977+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f402800b680 tx=0x7f402800b850 comp rx=0 tx=0).stop 2023-07-13T22:15:59.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.980+0000 7f403259c700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 0x7f40180a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.981+0000 7f403259c700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 0x7f40180a3780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40240073e0 tx=0x7f402400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:15:59.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.992+0000 7f40479a0b80 1 -- 172.21.15.26:0/4157106317 shutdown_connections 2023-07-13T22:15:59.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.992+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4337454d0 0x55e4337458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.992+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433746c70 0x55e4337470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.992+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4157106317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337466e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.992+0000 7f40479a0b80 1 -- 172.21.15.26:0/4157106317 >> 172.21.15.26:0/4157106317 conn(0x55e43373ada0 msgr2=0x55e4337401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:15:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.995+0000 7f40479a0b80 1 -- 172.21.15.26:0/4157106317 shutdown_connections 2023-07-13T22:15:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.996+0000 7f40479a0b80 1 -- 172.21.15.26:0/4157106317 wait complete. 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.997+0000 7f40479a0b80 1 -- start start 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.998+0000 7f40479a0b80 1 Processor -- start 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.998+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.998+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.999+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433767730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.999+0000 7f40479a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e433636ac0 con 0x55e433746c70 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.999+0000 7f40479a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4336367c0 con 0x55e4337454d0 2023-07-13T22:15:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.999+0000 7f40479a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43351a920 con 0x55e433745e20 2023-07-13T22:15:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:15:59.999+0000 7f40479a0b80 10 client.0 _ll_register_callbacks cb 0x55e433634760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:15:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.001+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.001+0000 7f4031d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51810/0 (socket says 172.21.15.26:51810) 2023-07-13T22:15:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.001+0000 7f4031d9b700 1 -- 172.21.15.26:0/2170999366 learned_addr learned my addr 172.21.15.26:0/2170999366 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:15:59.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.001+0000 7f403159a700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:15:59.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.007+0000 7f4031d9b700 1 -- 172.21.15.26:0/2170999366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 msgr2=0x55e4337606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.007+0000 7f4031d9b700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.007+0000 7f4031d9b700 1 -- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 msgr2=0x55e433767730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:15:59.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.007+0000 7f4031d9b700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433767730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:15:59.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.008+0000 7f4031d9b700 1 -- 172.21.15.26:0/2170999366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c117640 con 0x55e4337454d0 2023-07-13T22:15:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.008+0000 7f4031d9b700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ce0 tx=0x7f4028023b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f90c183330d489 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.009+0000 7f40157fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.011+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402801d070 con 0x55e4337454d0 2023-07-13T22:16:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.012+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f402800f460 con 0x55e4337454d0 2023-07-13T22:16:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.013+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4335f04f0 con 0x55e4337454d0 2023-07-13T22:16:00.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.013+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f402800f620 con 0x55e4337454d0 2023-07-13T22:16:00.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.015+0000 7f40479a0b80 10 client.25724 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:00.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.015+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e4334adc60 con 0x55e4337454d0 2023-07-13T22:16:00.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4335f04f0 con 0x55e4337454d0 2023-07-13T22:16:00.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4028035070 con 0x55e4337454d0 2023-07-13T22:16:00.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:00.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 10 client.25724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:16:00.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 20 client.25724 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:00.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e43376ba40 0x55e43376de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 4 client.25724 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:00.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.016+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e43368feb0 con 0x55e43376ba40 2023-07-13T22:16:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.029+0000 7f403159a700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e43376ba40 0x55e43376de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.030+0000 7f40157fa700 10 client.25724 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.030+0000 7f403159a700 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e43376ba40 0x55e43376de20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb6812e10dc2ffd server_cookie=ad03ffdf0f19d21f in_seq=0 out_seq=0 2023-07-13T22:16:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.032+0000 7f40157fa700 1 -- 172.21.15.26:0/2170999366 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e43368feb0 con 0x55e43376ba40 2023-07-13T22:16:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.032+0000 7f40157fa700 10 client.25724 handle_command_reply: tid=0 2023-07-13T22:16:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.032+0000 7f40479a0b80 1 client.25724 shutdown 2023-07-13T22:16:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.033+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 msgr2=0x55e4337601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.033+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4028009ce0 tx=0x7f4028023b70 comp rx=0 tx=0).stop 2023-07-13T22:16:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 shutdown_connections 2023-07-13T22:16:00.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e43376ba40 0x55e43376de20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433767730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e4337601b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 --2- 172.21.15.26:0/2170999366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433745e20 0x55e4337606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 >> 172.21.15.26:0/2170999366 conn(0x55e43373ada0 msgr2=0x55e43373f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40157fa700 0 client.25724 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 shutdown_connections 2023-07-13T22:16:00.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.035+0000 7f40479a0b80 1 -- 172.21.15.26:0/2170999366 wait complete. 2023-07-13T22:16:00.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.036+0000 7f40479a0b80 20 client.25724 trim_cache size 0 max 16384 2023-07-13T22:16:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 Processor -- start 2023-07-13T22:16:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 -- start start 2023-07-13T22:16:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4337454d0 0x55e43375ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433749c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43375fd80 con 0x55e433746c70 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43375ec20 con 0x55e433745e20 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.065+0000 7f40479a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4337611e0 con 0x55e4337454d0 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.066+0000 7f403159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.066+0000 7f403159a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51812/0 (socket says 172.21.15.26:51812) 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.066+0000 7f403159a700 1 -- 172.21.15.26:0/4075947452 learned_addr learned my addr 172.21.15.26:0/4075947452 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.067+0000 7f4031d9b700 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4337454d0 0x55e43375ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.068+0000 7f403159a700 1 -- 172.21.15.26:0/4075947452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4337454d0 msgr2=0x55e43375ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.068+0000 7f403159a700 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4337454d0 0x55e43375ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.068+0000 7f403159a700 1 -- 172.21.15.26:0/4075947452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 msgr2=0x55e433749c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.068+0000 7f403159a700 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433749c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.068+0000 7f403159a700 1 -- 172.21.15.26:0/4075947452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4334eb250 con 0x55e433745e20 2023-07-13T22:16:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.069+0000 7f403159a700 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f401c010fd0 tx=0x7f401c095f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dd8acebc120898d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.070+0000 7f4020ff9700 1 -- 172.21.15.26:0/4075947452 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c014040 con 0x55e433745e20 2023-07-13T22:16:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.070+0000 7f4020ff9700 1 -- 172.21.15.26:0/4075947452 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f401c097cc0 con 0x55e433745e20 2023-07-13T22:16:00.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.070+0000 7f4020ff9700 1 -- 172.21.15.26:0/4075947452 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c0992d0 con 0x55e433745e20 2023-07-13T22:16:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.071+0000 7f40479a0b80 1 -- 172.21.15.26:0/4075947452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 msgr2=0x55e4337676f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.071+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f401c010fd0 tx=0x7f401c095f70 comp rx=0 tx=0).stop 2023-07-13T22:16:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.078+0000 7f40479a0b80 1 -- 172.21.15.26:0/4075947452 shutdown_connections 2023-07-13T22:16:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.078+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433746c70 0x55e433749c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.078+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4337454d0 0x55e43375ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.078+0000 7f40479a0b80 1 --2- 172.21.15.26:0/4075947452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e433745e20 0x55e4337676f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.078+0000 7f40479a0b80 1 -- 172.21.15.26:0/4075947452 >> 172.21.15.26:0/4075947452 conn(0x55e43373ada0 msgr2=0x55e43376da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:00.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.079+0000 7f40479a0b80 1 -- 172.21.15.26:0/4075947452 shutdown_connections 2023-07-13T22:16:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.079+0000 7f40479a0b80 1 -- 172.21.15.26:0/4075947452 wait complete. 2023-07-13T22:16:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.080+0000 7f40479a0b80 1 -- start start 2023-07-13T22:16:00.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.080+0000 7f40479a0b80 1 Processor -- start 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e433765300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433745e20 0x55e433765840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e433766d90 con 0x55e433745e20 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e433766f00 con 0x55e4337454d0 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4337670c0 con 0x55e433746c70 2023-07-13T22:16:00.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.081+0000 7f40479a0b80 10 client.0 _ll_register_callbacks cb 0x7f4024009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.082+0000 7f403259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.082+0000 7f403259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36102/0 (socket says 172.21.15.26:36102) 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.082+0000 7f403259c700 1 -- 172.21.15.26:0/825503239 learned_addr learned my addr 172.21.15.26:0/825503239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.082+0000 7f4031d9b700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e433765300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.083+0000 7f403259c700 1 -- 172.21.15.26:0/825503239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 msgr2=0x55e433765300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.083+0000 7f403259c700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e433765300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.083+0000 7f403259c700 1 -- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433745e20 msgr2=0x55e433765840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:00.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.083+0000 7f403259c700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433745e20 0x55e433765840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.083+0000 7f403259c700 1 -- 172.21.15.26:0/825503239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c1acc40 con 0x55e433746c70 2023-07-13T22:16:00.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.084+0000 7f403259c700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4024018a20 tx=0x7f4024018d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd3d1276941d5cc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.086+0000 7f40157fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.086+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024017c60 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.086+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4024017dc0 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.086+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f402801e040 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.087+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4024018e50 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.087+0000 7f40479a0b80 10 client.25726 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.087+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e4334c32b0 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.091+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f402801e040 con 0x55e433746c70 2023-07-13T22:16:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.091+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f402402b070 con 0x55e433746c70 2023-07-13T22:16:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.094+0000 7f40479a0b80 10 client.25726 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.094+0000 7f40479a0b80 10 client.25726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:16:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.095+0000 7f40479a0b80 20 client.25726 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.095+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e4337c06e0 0x55e4337c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.095+0000 7f40479a0b80 4 client.25726 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.095+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e433652050 con 0x55e4337c06e0 2023-07-13T22:16:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.101+0000 7f4031d9b700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e4337c06e0 0x55e4337c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:00.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.131+0000 7f4031d9b700 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e4337c06e0 0x55e4337c2af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=680bd3d6a0d3b279 server_cookie=ce284c2a36a3697 in_seq=0 out_seq=0 2023-07-13T22:16:00.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.131+0000 7f40157fa700 10 client.25726 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.142+0000 7f40157fa700 1 -- 172.21.15.26:0/825503239 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e433652050 con 0x55e4337c06e0 2023-07-13T22:16:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.142+0000 7f40157fa700 10 client.25726 handle_command_reply: tid=0 2023-07-13T22:16:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.142+0000 7f40479a0b80 1 client.25726 shutdown 2023-07-13T22:16:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 msgr2=0x55e433766850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4024018a20 tx=0x7f4024018d30 comp rx=0 tx=0).stop 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 shutdown_connections 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e433746c70 0x55e433766850 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4337454d0 0x55e433765300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55e4337c06e0 0x55e4337c2af0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 --2- 172.21.15.26:0/825503239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e433745e20 0x55e433765840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.148+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 >> 172.21.15.26:0/825503239 conn(0x55e43373ada0 msgr2=0x55e43375ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.153+0000 7f40157fa700 0 client.25726 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.153+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 shutdown_connections 2023-07-13T22:16:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.153+0000 7f40479a0b80 1 -- 172.21.15.26:0/825503239 wait complete. 2023-07-13T22:16:00.160 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.171+0000 7f40479a0b80 20 client.25726 trim_cache size 0 max 16384 2023-07-13T22:16:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.173+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 msgr2=0x7f40180a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.173+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 0x7f40180a3780 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f40240073e0 tx=0x7f402400b040 comp rx=0 tx=0).stop 2023-07-13T22:16:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.173+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 msgr2=0x7f402c1a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.173+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f401c00d900 tx=0x7f401c00dc10 comp rx=0 tx=0).stop 2023-07-13T22:16:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 shutdown_connections 2023-07-13T22:16:00.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f40180a12d0 0x7f40180a3780 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f402c10bd60 0x7f402c1abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402c070ee0 0x7f402c1a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 --2- 172.21.15.26:0/1024935467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f402c071830 0x7f402c1a4b90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:00.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.174+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 >> 172.21.15.26:0/1024935467 conn(0x7f402c06ca90 msgr2=0x7f402c06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.186+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 shutdown_connections 2023-07-13T22:16:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:00.187+0000 7f4033fff700 1 -- 172.21.15.26:0/1024935467 wait complete. 2023-07-13T22:16:00.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:16:00.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:00 smithi117 ceph-mon[129074]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:16:00.976 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:00 smithi026 ceph-mon[95647]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:16:00.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:00 smithi121 ceph-mon[129187]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:16:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.113+0000 7f23bedff700 1 -- 172.21.15.26:0/77218824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8100cb0 msgr2=0x7f23b8103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.113+0000 7f23bedff700 1 --2- 172.21.15.26:0/77218824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8100cb0 0x7f23b8103130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.121+0000 7f23bedff700 1 -- 172.21.15.26:0/77218824 shutdown_connections 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.121+0000 7f23bedff700 1 --2- 172.21.15.26:0/77218824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23b8103670 0x7f23b8103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.121+0000 7f23bedff700 1 --2- 172.21.15.26:0/77218824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.121+0000 7f23bedff700 1 --2- 172.21.15.26:0/77218824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8100cb0 0x7f23b8103130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.121+0000 7f23bedff700 1 -- 172.21.15.26:0/77218824 >> 172.21.15.26:0/77218824 conn(0x7f23b80fa000 msgr2=0x7f23b80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.132+0000 7f23bedff700 1 -- 172.21.15.26:0/77218824 shutdown_connections 2023-07-13T22:16:01.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.132+0000 7f23bedff700 1 -- 172.21.15.26:0/77218824 wait complete. 2023-07-13T22:16:01.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 Processor -- start 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 -- start start 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23b8100cb0 0x7f23b80703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8103670 0x7f23b80713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80719b0 con 0x7f23b8100cb0 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8071b20 con 0x7f23b8103670 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.134+0000 7f23bedff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b8071ce0 con 0x7f23b80fe350 2023-07-13T22:16:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36112/0 (socket says 172.21.15.26:36112) 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 -- 172.21.15.26:0/3361182093 learned_addr learned my addr 172.21.15.26:0/3361182093 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bd39c700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8103670 0x7f23b80713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 -- 172.21.15.26:0/3361182093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8103670 msgr2=0x7f23b80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8103670 0x7f23b80713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 -- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23b8100cb0 msgr2=0x7f23b80703a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:16:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.135+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23b8100cb0 0x7f23b80703a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.136+0000 7f23bcb9b700 1 -- 172.21.15.26:0/3361182093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80095f0 con 0x7f23b80fe350 2023-07-13T22:16:01.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.144+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23b00060b0 tx=0x7f23b000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.157+0000 7f23b5ffb700 1 -- 172.21.15.26:0/3361182093 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b000ddd0 con 0x7f23b80fe350 2023-07-13T22:16:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.157+0000 7f23b5ffb700 1 -- 172.21.15.26:0/3361182093 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b000f040 con 0x7f23b80fe350 2023-07-13T22:16:01.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.157+0000 7f23b5ffb700 1 -- 172.21.15.26:0/3361182093 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0014690 con 0x7f23b80fe350 2023-07-13T22:16:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.157+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b8071f60 con 0x7f23b80fe350 2023-07-13T22:16:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.157+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8072420 con 0x7f23b80fe350 2023-07-13T22:16:01.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.164+0000 7f23d270cb80 1 Processor -- start 2023-07-13T22:16:01.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.166+0000 7f23d270cb80 1 -- start start 2023-07-13T22:16:01.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e462f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e46306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4534cb0 con 0x55c5e462f4d0 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4587a40 con 0x55c5e462fe20 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.167+0000 7f23d270cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4580ed0 con 0x55c5e4630c70 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.168+0000 7f23b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.168+0000 7f23b7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36124/0 (socket says 172.21.15.26:36124) 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.168+0000 7f23b7fff700 1 -- 172.21.15.26:0/436162849 learned_addr learned my addr 172.21.15.26:0/436162849 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:01.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.172+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e46306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.173+0000 7f23b7fff700 1 -- 172.21.15.26:0/436162849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 msgr2=0x55c5e46306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.173+0000 7f23b7fff700 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e46306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.173+0000 7f23b7fff700 1 -- 172.21.15.26:0/436162849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 msgr2=0x55c5e462f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.173+0000 7f23b7fff700 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e462f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.173+0000 7f23b7fff700 1 -- 172.21.15.26:0/436162849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e43aede0 con 0x55c5e4630c70 2023-07-13T22:16:01.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.175+0000 7f23b7fff700 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f23a8005910 tx=0x7f23a8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184f560b1ff71c6a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.176+0000 7f23b5ffb700 1 -- 172.21.15.26:0/3361182093 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f23b00147f0 con 0x7f23b80fe350 2023-07-13T22:16:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.179+0000 7f23b5ffb700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 0x7f23a00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.180+0000 7f23b5ffb700 1 -- 172.21.15.26:0/3361182093 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23b00cc2a0 con 0x7f23b80fe350 2023-07-13T22:16:01.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.183+0000 7f239effd700 1 -- 172.21.15.26:0/436162849 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x55c5e4630c70 2023-07-13T22:16:01.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.183+0000 7f239effd700 1 -- 172.21.15.26:0/436162849 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23a800be00 con 0x55c5e4630c70 2023-07-13T22:16:01.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.183+0000 7f239effd700 1 -- 172.21.15.26:0/436162849 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004970 con 0x55c5e4630c70 2023-07-13T22:16:01.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.184+0000 7f23d270cb80 1 -- 172.21.15.26:0/436162849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 msgr2=0x55c5e46310e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.184+0000 7f23d270cb80 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f23a8005910 tx=0x7f23a8005c00 comp rx=0 tx=0).stop 2023-07-13T22:16:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.185+0000 7f23d270cb80 1 -- 172.21.15.26:0/436162849 shutdown_connections 2023-07-13T22:16:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.185+0000 7f23d270cb80 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e462f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.185+0000 7f23d270cb80 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e4630c70 0x55c5e46310e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.185+0000 7f23d270cb80 1 --2- 172.21.15.26:0/436162849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e46306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.185+0000 7f23d270cb80 1 -- 172.21.15.26:0/436162849 >> 172.21.15.26:0/436162849 conn(0x55c5e4624da0 msgr2=0x55c5e462a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.186+0000 7f23d270cb80 1 -- 172.21.15.26:0/436162849 shutdown_connections 2023-07-13T22:16:01.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.186+0000 7f23d270cb80 1 -- 172.21.15.26:0/436162849 wait complete. 2023-07-13T22:16:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.187+0000 7f23bd39c700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 0x7f23a00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.187+0000 7f23d270cb80 1 -- start start 2023-07-13T22:16:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.188+0000 7f23d270cb80 1 Processor -- start 2023-07-13T22:16:01.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e4639170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462fe20 0x55c5e46396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4520ac0 con 0x55c5e462f4d0 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e45207c0 con 0x55c5e4630c70 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4404920 con 0x55c5e462fe20 2023-07-13T22:16:01.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.196+0000 7f23d270cb80 10 client.0 _ll_register_callbacks cb 0x55c5e451e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51856/0 (socket says 172.21.15.26:51856) 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 -- 172.21.15.26:0/2993979730 learned_addr learned my addr 172.21.15.26:0/2993979730 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e4639170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 -- 172.21.15.26:0/2993979730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462fe20 msgr2=0x55c5e46396b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462fe20 0x55c5e46396b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 -- 172.21.15.26:0/2993979730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 msgr2=0x55c5e4639170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e4639170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.208+0000 7f23bd39c700 1 -- 172.21.15.26:0/2993979730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8104610 con 0x55c5e4630c70 2023-07-13T22:16:01.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.220+0000 7f23bd39c700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0189b0 tx=0x7f23ac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46dc1dfb1f3258d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.225+0000 7f239c7f8700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.229+0000 7f23bd39c700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 0x7f23a00a3800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009e70 tx=0x7f23ac0155e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.229+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac017900 con 0x55c5e4630c70 2023-07-13T22:16:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.229+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23ac017a60 con 0x55c5e4630c70 2023-07-13T22:16:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.230+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5e44da4f0 con 0x55c5e4630c70 2023-07-13T22:16:01.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.232+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac017c20 con 0x55c5e4630c70 2023-07-13T22:16:01.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.232+0000 7f23d270cb80 10 client.25742 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.232+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5e4397c60 con 0x55c5e4630c70 2023-07-13T22:16:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5e44da4f0 con 0x55c5e4630c70 2023-07-13T22:16:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f23ac021030 con 0x55c5e4630c70 2023-07-13T22:16:01.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 10 client.25742 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 20 client.25742 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e46450b0 0x55c5e46474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 4 client.25742 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:01.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.244+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5e4579eb0 con 0x55c5e46450b0 2023-07-13T22:16:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.245+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e46450b0 0x55c5e46474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.246+0000 7f239c7f8700 10 client.25742 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.246+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e46450b0 0x55c5e46474c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b2d56be2e80031a server_cookie=846de04b018b8a10 in_seq=0 out_seq=0 2023-07-13T22:16:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.256+0000 7f239c7f8700 1 -- 172.21.15.26:0/2993979730 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5e4579eb0 con 0x55c5e46450b0 2023-07-13T22:16:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.256+0000 7f239c7f8700 10 client.25742 handle_command_reply: tid=0 2023-07-13T22:16:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.257+0000 7f23d270cb80 1 client.25742 shutdown 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 msgr2=0x55c5e46406f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0189b0 tx=0x7f23ac004750 comp rx=0 tx=0).stop 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 shutdown_connections 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462f4d0 0x55c5e4639170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e46450b0 0x55c5e46474c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.258+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e46406f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.259+0000 7f23d270cb80 1 --2- 172.21.15.26:0/2993979730 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462fe20 0x55c5e46396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.259+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 >> 172.21.15.26:0/2993979730 conn(0x55c5e4624da0 msgr2=0x55c5e46298b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.259+0000 7f239c7f8700 0 client.25742 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.259+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 shutdown_connections 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.259+0000 7f23d270cb80 1 -- 172.21.15.26:0/2993979730 wait complete. 2023-07-13T22:16:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.260+0000 7f23d270cb80 20 client.25742 trim_cache size 0 max 16384 2023-07-13T22:16:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.313+0000 7f23d270cb80 1 Processor -- start 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 -- start start 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e46470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e4640610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 0x55c5e4636ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4638620 con 0x55c5e4630c70 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4638ce0 con 0x55c5e462fe20 2023-07-13T22:16:01.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.314+0000 7f23d270cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e4630350 con 0x55c5e462f4d0 2023-07-13T22:16:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.315+0000 7f23bcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e46470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.315+0000 7f23bcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e46470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:36132/0 (socket says 172.21.15.26:36132) 2023-07-13T22:16:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.315+0000 7f23bcb9b700 1 -- 172.21.15.26:0/1912896069 learned_addr learned my addr 172.21.15.26:0/1912896069 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:01.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.315+0000 7f23b7fff700 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e4640610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.315+0000 7f23bd39c700 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 0x55c5e4636ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 -- 172.21.15.26:0/1912896069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 msgr2=0x55c5e46470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e46470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 -- 172.21.15.26:0/1912896069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 msgr2=0x55c5e4640610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e4640610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 -- 172.21.15.26:0/1912896069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e43d5250 con 0x55c5e4630c70 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.316+0000 7f23bd39c700 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 0x55c5e4636ff0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23ac012420 tx=0x7f23ac015aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63ebae1c7c67375 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.321+0000 7f239f7fe700 1 -- 172.21.15.26:0/1912896069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac009180 con 0x55c5e4630c70 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.321+0000 7f239f7fe700 1 -- 172.21.15.26:0/1912896069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23ac00d040 con 0x55c5e4630c70 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.321+0000 7f239f7fe700 1 -- 172.21.15.26:0/1912896069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac012a70 con 0x55c5e4630c70 2023-07-13T22:16:01.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.322+0000 7f23d270cb80 1 -- 172.21.15.26:0/1912896069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 msgr2=0x55c5e4636ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.322+0000 7f23d270cb80 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 0x55c5e4636ff0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23ac012420 tx=0x7f23ac015aa0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.324+0000 7f23d270cb80 1 -- 172.21.15.26:0/1912896069 shutdown_connections 2023-07-13T22:16:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.324+0000 7f23d270cb80 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e4630c70 0x55c5e4636ff0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.324+0000 7f23d270cb80 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e46470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.324+0000 7f23d270cb80 1 --2- 172.21.15.26:0/1912896069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e462fe20 0x55c5e4640610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.324+0000 7f23d270cb80 1 -- 172.21.15.26:0/1912896069 >> 172.21.15.26:0/1912896069 conn(0x55c5e4624da0 msgr2=0x55c5e4637e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.335+0000 7f23d270cb80 1 -- 172.21.15.26:0/1912896069 shutdown_connections 2023-07-13T22:16:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.335+0000 7f23d270cb80 1 -- 172.21.15.26:0/1912896069 wait complete. 2023-07-13T22:16:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.336+0000 7f23d270cb80 1 -- start start 2023-07-13T22:16:01.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.336+0000 7f23d270cb80 1 Processor -- start 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e463c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462fe20 0x55c5e463c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e463fbe0 con 0x55c5e462fe20 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e463fd50 con 0x55c5e4630c70 2023-07-13T22:16:01.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e463ff10 con 0x55c5e462f4d0 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.337+0000 7f23d270cb80 10 client.0 _ll_register_callbacks cb 0x55c5e451e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.343+0000 7f23bd39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.343+0000 7f23bcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e463c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.346+0000 7f23bd39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:51868/0 (socket says 172.21.15.26:51868) 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.346+0000 7f23bd39c700 1 -- 172.21.15.26:0/4217406632 learned_addr learned my addr 172.21.15.26:0/4217406632 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.355+0000 7f23bd39c700 1 -- 172.21.15.26:0/4217406632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 msgr2=0x55c5e463c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.355+0000 7f23bd39c700 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e463c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.355+0000 7f23bd39c700 1 -- 172.21.15.26:0/4217406632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462fe20 msgr2=0x55c5e463c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.355+0000 7f23bd39c700 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462fe20 0x55c5e463c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.355+0000 7f23bd39c700 1 -- 172.21.15.26:0/4217406632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5e4397c60 con 0x55c5e4630c70 2023-07-13T22:16:01.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.360+0000 7f23bd39c700 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0176c0 tx=0x7f23ac012770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b645c636649bf769 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:01.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.374+0000 7f239c7f8700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:01.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.374+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac017b10 con 0x55c5e4630c70 2023-07-13T22:16:01.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.374+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23ac017c70 con 0x55c5e4630c70 2023-07-13T22:16:01.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.374+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5e453c050 con 0x55c5e4630c70 2023-07-13T22:16:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.383+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23ac0129b0 con 0x55c5e4630c70 2023-07-13T22:16:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.383+0000 7f23d270cb80 10 client.25748 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.383+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c5e43ad2b0 con 0x55c5e4630c70 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.393+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5e453c050 con 0x55c5e4630c70 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.393+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f23ac049070 con 0x55c5e4630c70 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 10 client.25748 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 10 client.25748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 20 client.25748 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e4699860 0x55c5e469bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 4 client.25748 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:01.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.394+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c5e43cff70 con 0x55c5e4699860 2023-07-13T22:16:01.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.407+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e4699860 0x55c5e469bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.432+0000 7f23bcb9b700 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e4699860 0x55c5e469bc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8493150d964c51ef server_cookie=5d0b0fb78e631 in_seq=0 out_seq=0 2023-07-13T22:16:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.433+0000 7f239c7f8700 10 client.25748 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.433+0000 7f239c7f8700 1 -- 172.21.15.26:0/4217406632 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c5e43cff70 con 0x55c5e4699860 2023-07-13T22:16:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.433+0000 7f239c7f8700 10 client.25748 handle_command_reply: tid=0 2023-07-13T22:16:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.434+0000 7f23d270cb80 1 client.25748 shutdown 2023-07-13T22:16:01.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.435+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 msgr2=0x55c5e463f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.435+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0176c0 tx=0x7f23ac012770 comp rx=0 tx=0).stop 2023-07-13T22:16:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 shutdown_connections 2023-07-13T22:16:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c5e4699860 0x55c5e469bc40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e4630c70 0x55c5e463f6a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c5e462f4d0 0x55c5e463c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 --2- 172.21.15.26:0/4217406632 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c5e462fe20 0x55c5e463c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 >> 172.21.15.26:0/4217406632 conn(0x55c5e4624da0 msgr2=0x55c5e46470c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.436+0000 7f239c7f8700 0 client.25748 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:01.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.437+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 shutdown_connections 2023-07-13T22:16:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.437+0000 7f23d270cb80 1 -- 172.21.15.26:0/4217406632 wait complete. 2023-07-13T22:16:01.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.439+0000 7f23d270cb80 20 client.25748 trim_cache size 0 max 16384 2023-07-13T22:16:01.428 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 msgr2=0x7f23a00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 0x7f23a00a3800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009e70 tx=0x7f23ac0155e0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 msgr2=0x7f23b806fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23b00060b0 tx=0x7f23b000d3b0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 shutdown_connections 2023-07-13T22:16:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23b8103670 0x7f23b80713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23b80fe350 0x7f23b806fe60 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23b00060b0 tx=0x7f23b000d3b0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23a00a1350 0x7f23a00a3800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 --2- 172.21.15.26:0/3361182093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23b8100cb0 0x7f23b80703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.441+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 >> 172.21.15.26:0/3361182093 conn(0x7f23b80fa000 msgr2=0x7f23b80feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.443+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 shutdown_connections 2023-07-13T22:16:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:01.443+0000 7f23bedff700 1 -- 172.21.15.26:0/3361182093 wait complete. 2023-07-13T22:16:02.463 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:16:02.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:16:02.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:02 smithi121 ceph-mon[129187]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:16:03.027 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:02 smithi026 ceph-mon[95647]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:16:03.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:02 smithi117 ceph-mon[129074]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:16:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.397+0000 7f599efff700 1 -- 172.21.15.26:0/487588132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 msgr2=0x7f599810bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.397+0000 7f599efff700 1 --2- 172.21.15.26:0/487588132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f599810bab0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f598c0098f0 tx=0x7f598c009c00 comp rx=0 tx=0).stop 2023-07-13T22:16:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.408+0000 7f599efff700 1 -- 172.21.15.26:0/487588132 shutdown_connections 2023-07-13T22:16:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.408+0000 7f599efff700 1 --2- 172.21.15.26:0/487588132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998070fb0 0x7f59980713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.408+0000 7f599efff700 1 --2- 172.21.15.26:0/487588132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f599810bff0 0x7f599810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.408+0000 7f599efff700 1 --2- 172.21.15.26:0/487588132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f599810bab0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.408+0000 7f599efff700 1 -- 172.21.15.26:0/487588132 >> 172.21.15.26:0/487588132 conn(0x7f599806ca30 msgr2=0x7f599806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.409+0000 7f599efff700 1 -- 172.21.15.26:0/487588132 shutdown_connections 2023-07-13T22:16:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.409+0000 7f599efff700 1 -- 172.21.15.26:0/487588132 wait complete. 2023-07-13T22:16:03.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.415+0000 7f599efff700 1 Processor -- start 2023-07-13T22:16:03.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.415+0000 7f599efff700 1 -- start start 2023-07-13T22:16:03.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 0x7f59981a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f59981a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599810bff0 0x7f5998078550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998078b50 con 0x7f599810bff0 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998078cc0 con 0x7f5998070fb0 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.416+0000 7f599efff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998078e30 con 0x7f5998109630 2023-07-13T22:16:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.417+0000 7f5997fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f59981a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.420+0000 7f5997fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f59981a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49654/0 (socket says 172.21.15.26:49654) 2023-07-13T22:16:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.420+0000 7f5997fff700 1 -- 172.21.15.26:0/4294666396 learned_addr learned my addr 172.21.15.26:0/4294666396 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.420+0000 7f599cd9b700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 0x7f59981a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.421+0000 7f599d59c700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599810bff0 0x7f5998078550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.424+0000 7f599cd9b700 1 -- 172.21.15.26:0/4294666396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 msgr2=0x7f59981a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.424+0000 7f599cd9b700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f59981a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.424+0000 7f599cd9b700 1 -- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599810bff0 msgr2=0x7f5998078550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.424+0000 7f599cd9b700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599810bff0 0x7f5998078550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.424+0000 7f599cd9b700 1 -- 172.21.15.26:0/4294666396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c009520 con 0x7f5998070fb0 2023-07-13T22:16:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.425+0000 7f599cd9b700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 0x7f59981a4630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f599000eee0 tx=0x7f599000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.425+0000 7f5995ffb700 1 -- 172.21.15.26:0/4294666396 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990018070 con 0x7f5998070fb0 2023-07-13T22:16:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.425+0000 7f5995ffb700 1 -- 172.21.15.26:0/4294666396 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f599000f040 con 0x7f5998070fb0 2023-07-13T22:16:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.425+0000 7f5995ffb700 1 -- 172.21.15.26:0/4294666396 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990004280 con 0x7f5998070fb0 2023-07-13T22:16:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.426+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59980790b0 con 0x7f5998070fb0 2023-07-13T22:16:03.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.426+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5998079570 con 0x7f5998070fb0 2023-07-13T22:16:03.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.434+0000 7f59b2917b80 1 Processor -- start 2023-07-13T22:16:03.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 -- start start 2023-07-13T22:16:03.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 0x55863fe67960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fd6cd30 con 0x55863fe68cf0 2023-07-13T22:16:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fdbfac0 con 0x55863fe67550 2023-07-13T22:16:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.435+0000 7f59b2917b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fdb8f50 con 0x55863fe67ea0 2023-07-13T22:16:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.438+0000 7f5997fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.438+0000 7f5997fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36604/0 (socket says 172.21.15.26:36604) 2023-07-13T22:16:03.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.438+0000 7f5997fff700 1 -- 172.21.15.26:0/3845468732 learned_addr learned my addr 172.21.15.26:0/3845468732 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.440+0000 7f599d59c700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 0x55863fe67960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.445+0000 7f599cd9b700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.445+0000 7f5997fff700 1 -- 172.21.15.26:0/3845468732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 msgr2=0x55863fe68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.445+0000 7f5995ffb700 1 -- 172.21.15.26:0/4294666396 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5990003a00 con 0x7f5998070fb0 2023-07-13T22:16:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.447+0000 7f5997fff700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.447+0000 7f5997fff700 1 -- 172.21.15.26:0/3845468732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 msgr2=0x55863fe67960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.447+0000 7f5997fff700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 0x55863fe67960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.447+0000 7f5997fff700 1 -- 172.21.15.26:0/3845468732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863fbe6e60 con 0x55863fe68cf0 2023-07-13T22:16:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.448+0000 7f599d59c700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 0x55863fe67960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:16:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.448+0000 7f599cd9b700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:16:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.449+0000 7f5997fff700 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f598c0094f0 tx=0x7f598c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839bbe5cbd70aec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.450+0000 7f597effd700 1 -- 172.21.15.26:0/3845468732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c01c040 con 0x55863fe68cf0 2023-07-13T22:16:03.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.451+0000 7f597effd700 1 -- 172.21.15.26:0/3845468732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c00f6e0 con 0x55863fe68cf0 2023-07-13T22:16:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.451+0000 7f597effd700 1 -- 172.21.15.26:0/3845468732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c005700 con 0x55863fe68cf0 2023-07-13T22:16:03.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f5995ffb700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 0x7f59800a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 -- 172.21.15.26:0/3845468732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 msgr2=0x55863fe69160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f598c0094f0 tx=0x7f598c005bc0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f5995ffb700 1 -- 172.21.15.26:0/4294666396 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59900ce4b0 con 0x7f5998070fb0 2023-07-13T22:16:03.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 -- 172.21.15.26:0/3845468732 shutdown_connections 2023-07-13T22:16:03.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe68cf0 0x55863fe69160 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67550 0x55863fe67960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.452+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3845468732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.453+0000 7f59b2917b80 1 -- 172.21.15.26:0/3845468732 >> 172.21.15.26:0/3845468732 conn(0x55863fe5ce20 msgr2=0x55863fe62250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.453+0000 7f59b2917b80 1 -- 172.21.15.26:0/3845468732 shutdown_connections 2023-07-13T22:16:03.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.453+0000 7f59b2917b80 1 -- 172.21.15.26:0/3845468732 wait complete. 2023-07-13T22:16:03.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.454+0000 7f599d59c700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 0x7f59800a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.454+0000 7f59b2917b80 1 -- start start 2023-07-13T22:16:03.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.454+0000 7f59b2917b80 1 Processor -- start 2023-07-13T22:16:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67550 0x55863fe71180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67ea0 0x55863fe716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fd58b40 con 0x55863fe67ea0 2023-07-13T22:16:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fd58840 con 0x55863fe68cf0 2023-07-13T22:16:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fc3c9a0 con 0x55863fe67550 2023-07-13T22:16:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f59b2917b80 10 client.0 _ll_register_callbacks cb 0x55863fd567e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.455+0000 7f599d59c700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 0x7f59800a3800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5988005fd0 tx=0x7f5988009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.456+0000 7f599d59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.456+0000 7f599d59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36384/0 (socket says 172.21.15.26:36384) 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.456+0000 7f599d59c700 1 -- 172.21.15.26:0/1797407018 learned_addr learned my addr 172.21.15.26:0/1797407018 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.456+0000 7f5997fff700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67ea0 0x55863fe716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 -- 172.21.15.26:0/1797407018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67550 msgr2=0x55863fe71180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67550 0x55863fe71180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 -- 172.21.15.26:0/1797407018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67ea0 msgr2=0x55863fe716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67ea0 0x55863fe716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 -- 172.21.15.26:0/1797407018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5988009900 con 0x55863fe68cf0 2023-07-13T22:16:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f599d59c700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5988004620 tx=0x7f5988004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afeb7386d6303baa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.457+0000 7f5977fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988017070 con 0x55863fe68cf0 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f598801f430 con 0x55863fe68cf0 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55863fd12570 con 0x55863fe68cf0 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5988003ea0 con 0x55863fe68cf0 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f59b2917b80 10 client.25760 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.458+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5998117750 con 0x55863fe68cf0 2023-07-13T22:16:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55863fd12570 con 0x55863fe68cf0 2023-07-13T22:16:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5988033870 con 0x55863fe68cf0 2023-07-13T22:16:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 10 client.25760 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 20 client.25760 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fe7cf90 0x55863fe7f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 4 client.25760 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.459+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55863fdb1f30 con 0x55863fe7cf90 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.460+0000 7f599cd9b700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fe7cf90 0x55863fe7f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.460+0000 7f599cd9b700 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fe7cf90 0x55863fe7f3a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae917b99e85048c5 server_cookie=bbb036e06d078c1b in_seq=0 out_seq=0 2023-07-13T22:16:03.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.460+0000 7f5977fff700 10 client.25760 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:03.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.461+0000 7f5977fff700 1 -- 172.21.15.26:0/1797407018 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55863fdb1f30 con 0x55863fe7cf90 2023-07-13T22:16:03.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.461+0000 7f5977fff700 10 client.25760 handle_command_reply: tid=0 2023-07-13T22:16:03.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.461+0000 7f59b2917b80 1 client.25760 shutdown 2023-07-13T22:16:03.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 msgr2=0x55863fe78700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5988004620 tx=0x7f5988004750 comp rx=0 tx=0).stop 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 shutdown_connections 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fe7cf90 0x55863fe7f3a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe78700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67550 0x55863fe71180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 --2- 172.21.15.26:0/1797407018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67ea0 0x55863fe716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f5977fff700 0 client.25760 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:03.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 >> 172.21.15.26:0/1797407018 conn(0x55863fe5ce20 msgr2=0x55863fe61930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.462+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 shutdown_connections 2023-07-13T22:16:03.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.463+0000 7f59b2917b80 1 -- 172.21.15.26:0/1797407018 wait complete. 2023-07-13T22:16:03.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.463+0000 7f59b2917b80 20 client.25760 trim_cache size 0 max 16384 2023-07-13T22:16:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.500+0000 7f59b2917b80 1 Processor -- start 2023-07-13T22:16:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.500+0000 7f59b2917b80 1 -- start start 2023-07-13T22:16:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.500+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe7efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.500+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe78620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.500+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.501+0000 7f59b2917b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fe62f50 con 0x55863fe67550 2023-07-13T22:16:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.501+0000 7f59b2917b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fe63d80 con 0x55863fe68cf0 2023-07-13T22:16:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.501+0000 7f59b2917b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fe70610 con 0x55863fe67ea0 2023-07-13T22:16:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36390/0 (socket says 172.21.15.26:36390) 2023-07-13T22:16:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 -- 172.21.15.26:0/3473391718 learned_addr learned my addr 172.21.15.26:0/3473391718 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f5997fff700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe78620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599cd9b700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe7efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 -- 172.21.15.26:0/3473391718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 msgr2=0x55863fe78620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe78620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 -- 172.21.15.26:0/3473391718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 msgr2=0x55863fe7efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe7efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f599d59c700 1 -- 172.21.15.26:0/3473391718 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55863fe5e660 con 0x55863fe68cf0 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.502+0000 7f5997fff700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe78620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.503+0000 7f599d59c700 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5988033c50 tx=0x7f5988012f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8f97a9064a10 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.503+0000 7f597f7fe700 1 -- 172.21.15.26:0/3473391718 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988038070 con 0x55863fe68cf0 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.503+0000 7f597f7fe700 1 -- 172.21.15.26:0/3473391718 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5988033930 con 0x55863fe68cf0 2023-07-13T22:16:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.503+0000 7f597f7fe700 1 -- 172.21.15.26:0/3473391718 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59880047c0 con 0x55863fe68cf0 2023-07-13T22:16:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.504+0000 7f59b2917b80 1 -- 172.21.15.26:0/3473391718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 msgr2=0x55863fe6f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.504+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5988033c50 tx=0x7f5988012f30 comp rx=0 tx=0).stop 2023-07-13T22:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.506+0000 7f59b2917b80 1 -- 172.21.15.26:0/3473391718 shutdown_connections 2023-07-13T22:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.506+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe7efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.506+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe68cf0 0x55863fe6f100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.506+0000 7f59b2917b80 1 --2- 172.21.15.26:0/3473391718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe67ea0 0x55863fe78620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.506+0000 7f59b2917b80 1 -- 172.21.15.26:0/3473391718 >> 172.21.15.26:0/3473391718 conn(0x55863fe5ce20 msgr2=0x55863fe6ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.511+0000 7f59b2917b80 1 -- 172.21.15.26:0/3473391718 shutdown_connections 2023-07-13T22:16:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.511+0000 7f59b2917b80 1 -- 172.21.15.26:0/3473391718 wait complete. 2023-07-13T22:16:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.512+0000 7f59b2917b80 1 -- start start 2023-07-13T22:16:03.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.512+0000 7f59b2917b80 1 Processor -- start 2023-07-13T22:16:03.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 0x55863fe73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 0x55863fee1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fe78750 con 0x55863fe67550 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fe788c0 con 0x55863fe67ea0 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55863fee23b0 con 0x55863fe68cf0 2023-07-13T22:16:03.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.513+0000 7f59b2917b80 10 client.0 _ll_register_callbacks cb 0x7f59880043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.514+0000 7f599cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.514+0000 7f599cd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36626/0 (socket says 172.21.15.26:36626) 2023-07-13T22:16:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.514+0000 7f599cd9b700 1 -- 172.21.15.26:0/405291827 learned_addr learned my addr 172.21.15.26:0/405291827 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:03.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.515+0000 7f599d59c700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 0x55863fee1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.515+0000 7f599d59c700 1 -- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 msgr2=0x55863fe73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.515+0000 7f5997fff700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 0x55863fe73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.515+0000 7f599d59c700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 0x55863fe73720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.516+0000 7f599d59c700 1 -- 172.21.15.26:0/405291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 msgr2=0x55863fe731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.516+0000 7f599d59c700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.516+0000 7f599d59c700 1 -- 172.21.15.26:0/405291827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5998117750 con 0x55863fe68cf0 2023-07-13T22:16:03.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.516+0000 7f5997fff700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 0x55863fe73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:16:03.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.516+0000 7f599d59c700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 0x55863fee1e70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5988005380 tx=0x7f5988003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f617bc35495993 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.527+0000 7f5977fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.527+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988040070 con 0x55863fe68cf0 2023-07-13T22:16:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.528+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5988033680 con 0x55863fe68cf0 2023-07-13T22:16:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.528+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f598c00f6e0 con 0x55863fe68cf0 2023-07-13T22:16:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.529+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5988038070 con 0x55863fe68cf0 2023-07-13T22:16:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.529+0000 7f59b2917b80 10 client.25750 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.529+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55863fbcfce0 con 0x55863fe68cf0 2023-07-13T22:16:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598c00f6e0 con 0x55863fe68cf0 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5988043030 con 0x55863fe68cf0 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 10 client.25750 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 10 client.25750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 20 client.25750 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fee3830 0x55863fee5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 4 client.25750 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:16:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.543+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55863fd740d0 con 0x55863fee3830 2023-07-13T22:16:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.553+0000 7f599cd9b700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fee3830 0x55863fee5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.554+0000 7f599cd9b700 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fee3830 0x55863fee5c40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98da4ba863d3808a server_cookie=b8690a390e9ae9d3 in_seq=0 out_seq=0 2023-07-13T22:16:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.554+0000 7f5977fff700 10 client.25750 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.555+0000 7f5977fff700 1 -- 172.21.15.26:0/405291827 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55863fd740d0 con 0x55863fee3830 2023-07-13T22:16:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.555+0000 7f5977fff700 10 client.25750 handle_command_reply: tid=0 2023-07-13T22:16:03.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.555+0000 7f59b2917b80 1 client.25750 shutdown 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 msgr2=0x55863fee1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 0x55863fee1e70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5988005380 tx=0x7f5988003f80 comp rx=0 tx=0).stop 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 shutdown_connections 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55863fe67550 0x55863fe731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55863fee3830 0x55863fee5c40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55863fe68cf0 0x55863fee1e70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 --2- 172.21.15.26:0/405291827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55863fe67ea0 0x55863fe73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 >> 172.21.15.26:0/405291827 conn(0x55863fe5ce20 msgr2=0x55863fe7efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.556+0000 7f5977fff700 0 client.25750 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.557+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 shutdown_connections 2023-07-13T22:16:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.557+0000 7f59b2917b80 1 -- 172.21.15.26:0/405291827 wait complete. 2023-07-13T22:16:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.558+0000 7f59b2917b80 20 client.25750 trim_cache size 0 max 16384 2023-07-13T22:16:03.547 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:16:03.547 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:16:03.548 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "e2aed3b8-9605-4741-9cac-65011e9cec93", 2023-07-13T22:16:03.548 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:16:03.548 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:16:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.559+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 msgr2=0x7f59800a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.559+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 0x7f59800a3800 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5988005fd0 tx=0x7f5988009310 comp rx=0 tx=0).stop 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.559+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 msgr2=0x7f59981a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.559+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 0x7f59981a4630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f599000eee0 tx=0x7f599000c3a0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 shutdown_connections 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f59800a1350 0x7f59800a3800 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599810bff0 0x7f5998078550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5998070fb0 0x7f59981a4630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 --2- 172.21.15.26:0/4294666396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5998109630 0x7f59981a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 >> 172.21.15.26:0/4294666396 conn(0x7f599806ca30 msgr2=0x7f599810d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 shutdown_connections 2023-07-13T22:16:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:03.560+0000 7f599efff700 1 -- 172.21.15.26:0/4294666396 wait complete. 2023-07-13T22:16:03.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:16:03.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:16:03.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:03.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:16:03.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:16:03.936 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:03.937 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:16:04.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:16:04.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:16:04.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.193 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:16:04.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:16:04.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:16:04.244 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.244 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:04.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 -- 172.21.15.26:0/690054318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec10bd60 msgr2=0x7f08ec10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 --2- 172.21.15.26:0/690054318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec10bd60 0x7f08ec10e1f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009990 tx=0x7f08dc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 -- 172.21.15.26:0/690054318 shutdown_connections 2023-07-13T22:16:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 --2- 172.21.15.26:0/690054318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec10bd60 0x7f08ec10e1f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 --2- 172.21.15.26:0/690054318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec070ee0 0x7f08ec0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 --2- 172.21.15.26:0/690054318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08ec071830 0x7f08ec10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.277+0000 7f08f0bff700 1 -- 172.21.15.26:0/690054318 >> 172.21.15.26:0/690054318 conn(0x7f08ec06ca90 msgr2=0x7f08ec06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.281+0000 7f08f0bff700 1 -- 172.21.15.26:0/690054318 shutdown_connections 2023-07-13T22:16:04.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.281+0000 7f08f0bff700 1 -- 172.21.15.26:0/690054318 wait complete. 2023-07-13T22:16:04.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.283+0000 7f08f0bff700 1 Processor -- start 2023-07-13T22:16:04.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.283+0000 7f08f0bff700 1 -- start start 2023-07-13T22:16:04.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 0x7f08ec1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 0x7f08ec1a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08ec10bd60 0x7f08ec1abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1ac340 con 0x7f08ec10bd60 2023-07-13T22:16:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec10ec40 con 0x7f08ec070ee0 2023-07-13T22:16:04.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.284+0000 7f08f0bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec10edd0 con 0x7f08ec071830 2023-07-13T22:16:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.289+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 0x7f08ec1a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.290+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 0x7f08ec1a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49710/0 (socket says 172.21.15.26:49710) 2023-07-13T22:16:04.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.290+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1135715193 learned_addr learned my addr 172.21.15.26:0/1135715193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:04.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.290+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 0x7f08ec1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.292+0000 7f08ea59c700 1 -- 172.21.15.26:0/1135715193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 msgr2=0x7f08ec1a4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.292+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 0x7f08ec1a4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.292+0000 7f08ea59c700 1 -- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08ec10bd60 msgr2=0x7f08ec1abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.292+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08ec10bd60 0x7f08ec1abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.292+0000 7f08ea59c700 1 -- 172.21.15.26:0/1135715193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0095f0 con 0x7f08ec070ee0 2023-07-13T22:16:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.293+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 0x7f08ec1a47c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006fd0 tx=0x7f08e000b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.299+0000 7f08db7fe700 1 -- 172.21.15.26:0/1135715193 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0011830 con 0x7f08ec070ee0 2023-07-13T22:16:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.299+0000 7f08db7fe700 1 -- 172.21.15.26:0/1135715193 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e0011e50 con 0x7f08ec070ee0 2023-07-13T22:16:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.299+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec10f0b0 con 0x7f08ec070ee0 2023-07-13T22:16:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.300+0000 7f08db7fe700 1 -- 172.21.15.26:0/1135715193 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0010e00 con 0x7f08ec070ee0 2023-07-13T22:16:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.300+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec10f640 con 0x7f08ec070ee0 2023-07-13T22:16:04.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.309+0000 7f090445eb80 1 Processor -- start 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.309+0000 7f090445eb80 1 -- start start 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd474d0 0x56114cd478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cc4ccb0 con 0x56114cd47e20 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cc9fa40 con 0x56114cd48c70 2023-07-13T22:16:04.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f090445eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cc98ed0 con 0x56114cd474d0 2023-07-13T22:16:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36444/0 (socket says 172.21.15.26:36444) 2023-07-13T22:16:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.310+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1296176539 learned_addr learned my addr 172.21.15.26:0/1296176539 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:04.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08ead9d700 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd474d0 0x56114cd478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1296176539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd474d0 msgr2=0x56114cd478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd474d0 0x56114cd478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1296176539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 msgr2=0x56114cd486e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd486e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1296176539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114cac6de0 con 0x56114cd48c70 2023-07-13T22:16:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.311+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08e40073f0 tx=0x7f08e4007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f32ecdd5405eab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.312+0000 7f08db7fe700 1 -- 172.21.15.26:0/1135715193 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f08e0011990 con 0x7f08ec070ee0 2023-07-13T22:16:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.313+0000 7f08d8ff9700 1 -- 172.21.15.26:0/1296176539 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4012710 con 0x56114cd48c70 2023-07-13T22:16:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.313+0000 7f08d8ff9700 1 -- 172.21.15.26:0/1296176539 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e4007e30 con 0x56114cd48c70 2023-07-13T22:16:04.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.313+0000 7f08d8ff9700 1 -- 172.21.15.26:0/1296176539 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4014c90 con 0x56114cd48c70 2023-07-13T22:16:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f08db7fe700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 0x7f08d40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 -- 172.21.15.26:0/1296176539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 msgr2=0x56114cd490e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f08e40073f0 tx=0x7f08e4007700 comp rx=0 tx=0).stop 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f08db7fe700 1 -- 172.21.15.26:0/1135715193 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08e00cc270 con 0x7f08ec070ee0 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 -- 172.21.15.26:0/1296176539 shutdown_connections 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd48c70 0x56114cd490e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd474d0 0x56114cd478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 --2- 172.21.15.26:0/1296176539 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f090445eb80 1 -- 172.21.15.26:0/1296176539 >> 172.21.15.26:0/1296176539 conn(0x56114cd3cda0 msgr2=0x56114cd421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.314+0000 7f08ead9d700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 0x7f08d40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.315+0000 7f090445eb80 1 -- 172.21.15.26:0/1296176539 shutdown_connections 2023-07-13T22:16:04.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.315+0000 7f090445eb80 1 -- 172.21.15.26:0/1296176539 wait complete. 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.315+0000 7f08ead9d700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 0x7f08d40a3800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b5c0 tx=0x7f08dc005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.315+0000 7f090445eb80 1 -- start start 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 Processor -- start 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd51740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cc38ac0 con 0x56114cd47e20 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cc387c0 con 0x56114cd474d0 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cb1c920 con 0x56114cd48c70 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f090445eb80 10 client.0 _ll_register_callbacks cb 0x56114cc36760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f08ea59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f08ea59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36458/0 (socket says 172.21.15.26:36458) 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.316+0000 7f08ea59c700 1 -- 172.21.15.26:0/1705667967 learned_addr learned my addr 172.21.15.26:0/1705667967 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:04.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd51740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ead9d700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 -- 172.21.15.26:0/1705667967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 msgr2=0x56114cd58780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 -- 172.21.15.26:0/1705667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 msgr2=0x56114cd51740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd51740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 -- 172.21.15.26:0/1705667967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e4009cc0 con 0x56114cd474d0 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08e00995c0 tx=0x7f08e0099630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62fb4fe8b68ac07b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a1a80 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08e00979d0 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.317+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56114cbf24f0 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e00c9e30 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 10 client.25775 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56114caafc60 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e00c7080 con 0x56114cd474d0 2023-07-13T22:16:04.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f08e00a16d0 con 0x56114cd474d0 2023-07-13T22:16:04.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 10 client.25775 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 20 client.25775 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cd5cfc0 0x56114cd5f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 4 client.25775 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.318+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56114cc91eb0 con 0x56114cd5cfc0 2023-07-13T22:16:04.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.319+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cd5cfc0 0x56114cd5f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.319+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cd5cfc0 0x56114cd5f3d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9c3ef925643d92 server_cookie=a0dd10911f63af1c in_seq=0 out_seq=0 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.319+0000 7f08d1ffb700 10 client.25775 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1705667967 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56114cc91eb0 con 0x56114cd5cfc0 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f08d1ffb700 10 client.25775 handle_command_reply: tid=0 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 client.25775 shutdown 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 msgr2=0x56114cd51200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08e00995c0 tx=0x7f08e0099630 comp rx=0 tx=0).stop 2023-07-13T22:16:04.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 shutdown_connections 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cd5cfc0 0x56114cd5f3d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd474d0 0x56114cd51200 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 --2- 172.21.15.26:0/1705667967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd47e20 0x56114cd51740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 >> 172.21.15.26:0/1705667967 conn(0x56114cd3cda0 msgr2=0x56114cd418b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f08d1ffb700 0 client.25775 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 shutdown_connections 2023-07-13T22:16:04.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 1 -- 172.21.15.26:0/1705667967 wait complete. 2023-07-13T22:16:04.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.320+0000 7f090445eb80 20 client.25775 trim_cache size 0 max 16384 2023-07-13T22:16:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 Processor -- start 2023-07-13T22:16:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 -- start start 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd4bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cd50dd0 con 0x56114cd474d0 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.346+0000 7f090445eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cd5a440 con 0x56114cd47e20 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f090445eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cd430a0 con 0x56114cd48c70 2023-07-13T22:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36466/0 (socket says 172.21.15.26:36466) 2023-07-13T22:16:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1723932348 learned_addr learned my addr 172.21.15.26:0/1723932348 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1723932348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 msgr2=0x56114cd58710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1723932348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 msgr2=0x56114cd4bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd4bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.347+0000 7f08e9d9b700 1 -- 172.21.15.26:0/1723932348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56114caed250 con 0x56114cd47e20 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.348+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009780 tx=0x7f08e4007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78de55d2b65ef00 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.348+0000 7f08d97fa700 1 -- 172.21.15.26:0/1723932348 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4007260 con 0x56114cd47e20 2023-07-13T22:16:04.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.348+0000 7f08d97fa700 1 -- 172.21.15.26:0/1723932348 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e4014400 con 0x56114cd47e20 2023-07-13T22:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.348+0000 7f08d97fa700 1 -- 172.21.15.26:0/1723932348 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4014710 con 0x56114cd47e20 2023-07-13T22:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 -- 172.21.15.26:0/1723932348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 msgr2=0x56114cd4c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009780 tx=0x7f08e4007700 comp rx=0 tx=0).stop 2023-07-13T22:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 -- 172.21.15.26:0/1723932348 shutdown_connections 2023-07-13T22:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd4bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd58710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 --2- 172.21.15.26:0/1723932348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd4c140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 -- 172.21.15.26:0/1723932348 >> 172.21.15.26:0/1723932348 conn(0x56114cd3cda0 msgr2=0x56114cd4fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 -- 172.21.15.26:0/1723932348 shutdown_connections 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.349+0000 7f090445eb80 1 -- 172.21.15.26:0/1723932348 wait complete. 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.350+0000 7f090445eb80 1 -- start start 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.350+0000 7f090445eb80 1 Processor -- start 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd54fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cd55a20 con 0x56114cd474d0 2023-07-13T22:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cd55b90 con 0x56114cd47e20 2023-07-13T22:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56114cdb0e70 con 0x56114cd48c70 2023-07-13T22:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f090445eb80 10 client.0 _ll_register_callbacks cb 0x56114cc36760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.351+0000 7f08ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36660/0 (socket says 172.21.15.26:36660) 2023-07-13T22:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 -- 172.21.15.26:0/1544788536 learned_addr learned my addr 172.21.15.26:0/1544788536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08e9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd54fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ead9d700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 -- 172.21.15.26:0/1544788536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 msgr2=0x56114cd554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 -- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 msgr2=0x56114cd54fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd54fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 -- 172.21.15.26:0/1544788536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56114cac6de0 con 0x56114cd474d0 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08ea59c700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08e00960a0 tx=0x7f08e009d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f92d40602f2687c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:04.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.352+0000 7f08d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00a1a80 con 0x56114cd474d0 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08e0098b20 con 0x56114cd474d0 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08e4007e30 con 0x56114cd474d0 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08e0098ce0 con 0x56114cd474d0 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f090445eb80 10 client.15642 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:04.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56114cac52b0 con 0x56114cd474d0 2023-07-13T22:16:04.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.353+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08e4007e30 con 0x56114cd474d0 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.364+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f08e009aa90 con 0x56114cd474d0 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 10 client.15642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 20 client.15642 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cdb25f0 0x56114cdb4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 4 client.15642 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:16:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.366+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56114cc54050 con 0x56114cdb25f0 2023-07-13T22:16:04.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.377+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cdb25f0 0x56114cdb4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:04.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.417+0000 7f08e9d9b700 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cdb25f0 0x56114cdb4a00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eaa19b9885715e1 server_cookie=1696cc0431fd9fa0 in_seq=0 out_seq=0 2023-07-13T22:16:04.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.433+0000 7f08d1ffb700 10 client.15642 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:04.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.439+0000 7f08d1ffb700 1 -- 172.21.15.26:0/1544788536 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56114cc54050 con 0x56114cdb25f0 2023-07-13T22:16:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.439+0000 7f08d1ffb700 10 client.15642 handle_command_reply: tid=0 2023-07-13T22:16:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.439+0000 7f090445eb80 1 client.15642 shutdown 2023-07-13T22:16:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.473+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 msgr2=0x56114cd58340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.473+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08e00960a0 tx=0x7f08e009d7e0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 shutdown_connections 2023-07-13T22:16:04.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56114cd474d0 0x56114cd58340 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56114cdb25f0 0x56114cdb4a00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56114cd48c70 0x56114cd554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 --2- 172.21.15.26:0/1544788536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56114cd47e20 0x56114cd54fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 >> 172.21.15.26:0/1544788536 conn(0x56114cd3cda0 msgr2=0x56114cd4bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.498+0000 7f08d1ffb700 0 client.15642 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.499+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 shutdown_connections 2023-07-13T22:16:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.499+0000 7f090445eb80 1 -- 172.21.15.26:0/1544788536 wait complete. 2023-07-13T22:16:04.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.509+0000 7f090445eb80 20 client.15642 trim_cache size 0 max 16384 2023-07-13T22:16:04.498 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:16:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:16:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:16:04.498 INFO:teuthology.orchestra.run.smithi026.stdout: "e2aed3b8-9605-4741-9cac-65011e9cec93": { 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e2aed3b8-9605-4741-9cac-65011e9cec93", 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:16:04.499 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:16:04.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.510+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 msgr2=0x7f08d40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.510+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 0x7f08d40a3800 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b5c0 tx=0x7f08dc005de0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.510+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 msgr2=0x7f08ec1a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:04.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.510+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 0x7f08ec1a47c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006fd0 tx=0x7f08e000b7e0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 shutdown_connections 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08d40a1350 0x7f08d40a3800 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08ec10bd60 0x7f08ec1abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08ec070ee0 0x7f08ec1a47c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 --2- 172.21.15.26:0/1135715193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08ec071830 0x7f08ec1a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:04.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.512+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 >> 172.21.15.26:0/1135715193 conn(0x7f08ec06ca90 msgr2=0x7f08ec06e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.513+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 shutdown_connections 2023-07-13T22:16:04.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:04.513+0000 7f08f0bff700 1 -- 172.21.15.26:0/1135715193 wait complete. 2023-07-13T22:16:04.547 INFO:tasks.cephfs.filesystem:scrub status for tag:e2aed3b8-9605-4741-9cac-65011e9cec93 - {'path': '/', 'tag': 'e2aed3b8-9605-4741-9cac-65011e9cec93', 'options': 'recursive,force'} 2023-07-13T22:16:04.859 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:04 smithi026 ceph-mon[95647]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 334 MiB/s wr, 108 op/s 2023-07-13T22:16:05.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:04 smithi117 ceph-mon[129074]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 334 MiB/s wr, 108 op/s 2023-07-13T22:16:05.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:04 smithi121 ceph-mon[129187]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 334 MiB/s wr, 108 op/s 2023-07-13T22:16:07.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:06 smithi117 ceph-mon[129074]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:16:07.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:06 smithi026 ceph-mon[95647]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:16:07.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:06 smithi121 ceph-mon[129187]: pgmap v382: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:16:09.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:08 smithi121 ceph-mon[129187]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:16:09.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:08 smithi117 ceph-mon[129074]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:16:09.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:08 smithi026 ceph-mon[95647]: pgmap v383: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:16:11.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:11 smithi117 ceph-mon[129074]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:16:11.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:11 smithi026 ceph-mon[95647]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:16:11.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:11 smithi121 ceph-mon[129187]: pgmap v384: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:16:12.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:12 smithi117 ceph-mon[129074]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2023-07-13T22:16:12.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:12 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:16:12.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:12 smithi026 ceph-mon[95647]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2023-07-13T22:16:12.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:12 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:16:12.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:12 smithi121 ceph-mon[129187]: pgmap v385: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2023-07-13T22:16:12.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:12 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:16:13.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:13 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:16:13.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:13 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:16:13.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:13 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:16:14.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:14 smithi117 ceph-mon[129074]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 338 MiB/s wr, 112 op/s 2023-07-13T22:16:14.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:14 smithi026 ceph-mon[95647]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 338 MiB/s wr, 112 op/s 2023-07-13T22:16:14.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:14 smithi121 ceph-mon[129187]: pgmap v386: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 338 MiB/s wr, 112 op/s 2023-07-13T22:16:16.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:16 smithi121 ceph-mon[129187]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 97 op/s 2023-07-13T22:16:17.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:16 smithi117 ceph-mon[129074]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 97 op/s 2023-07-13T22:16:17.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:16 smithi026 ceph-mon[95647]: pgmap v387: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 97 op/s 2023-07-13T22:16:19.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:18 smithi117 ceph-mon[129074]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:16:19.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:18 smithi026 ceph-mon[95647]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:16:19.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:18 smithi121 ceph-mon[129187]: pgmap v388: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:16:21.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:20 smithi117 ceph-mon[129074]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 295 MiB/s wr, 97 op/s 2023-07-13T22:16:21.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:20 smithi026 ceph-mon[95647]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 295 MiB/s wr, 97 op/s 2023-07-13T22:16:21.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:20 smithi121 ceph-mon[129187]: pgmap v389: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 295 MiB/s wr, 97 op/s 2023-07-13T22:16:23.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:22 smithi117 ceph-mon[129074]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 87 op/s 2023-07-13T22:16:23.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:22 smithi026 ceph-mon[95647]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 87 op/s 2023-07-13T22:16:23.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:22 smithi121 ceph-mon[129187]: pgmap v390: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 266 MiB/s wr, 87 op/s 2023-07-13T22:16:25.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:24 smithi117 ceph-mon[129074]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 326 MiB/s wr, 109 op/s 2023-07-13T22:16:25.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:24 smithi026 ceph-mon[95647]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 326 MiB/s wr, 109 op/s 2023-07-13T22:16:25.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:24 smithi121 ceph-mon[129187]: pgmap v391: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 326 MiB/s wr, 109 op/s 2023-07-13T22:16:27.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:26 smithi117 ceph-mon[129074]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:16:27.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:26 smithi026 ceph-mon[95647]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:16:27.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:26 smithi121 ceph-mon[129187]: pgmap v392: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:16:29.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:28 smithi117 ceph-mon[129074]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 287 MiB/s wr, 97 op/s 2023-07-13T22:16:29.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:28 smithi026 ceph-mon[95647]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 287 MiB/s wr, 97 op/s 2023-07-13T22:16:29.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:28 smithi121 ceph-mon[129187]: pgmap v393: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 287 MiB/s wr, 97 op/s 2023-07-13T22:16:31.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:30 smithi121 ceph-mon[129187]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 291 MiB/s wr, 95 op/s 2023-07-13T22:16:31.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:30 smithi117 ceph-mon[129074]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 291 MiB/s wr, 95 op/s 2023-07-13T22:16:31.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:30 smithi026 ceph-mon[95647]: pgmap v394: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 291 MiB/s wr, 95 op/s 2023-07-13T22:16:33.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 86 op/s 2023-07-13T22:16:33.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:16:33.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:33 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.468 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 86 op/s 2023-07-13T22:16:33.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:16:33.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.469 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:33 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: pgmap v395: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 86 op/s 2023-07-13T22:16:33.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:16:33.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:33.493 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:33 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:16:34.864 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:34 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.865 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:34 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.865 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:34 smithi026 ceph-mon[95647]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 318 MiB/s wr, 105 op/s 2023-07-13T22:16:34.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:34 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:34 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:34 smithi117 ceph-mon[129074]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 318 MiB/s wr, 105 op/s 2023-07-13T22:16:34.992 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:34 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:34 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:34.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:34 smithi121 ceph-mon[129187]: pgmap v396: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 318 MiB/s wr, 105 op/s 2023-07-13T22:16:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.288+0000 7f80863ff700 1 -- 172.21.15.26:0/1803666282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080070ee0 msgr2=0x7f80800712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.288+0000 7f80863ff700 1 --2- 172.21.15.26:0/1803666282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080070ee0 0x7f80800712f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8070009990 tx=0x7f8070009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 -- 172.21.15.26:0/1803666282 shutdown_connections 2023-07-13T22:16:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 --2- 172.21.15.26:0/1803666282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080070ee0 0x7f80800712f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 --2- 172.21.15.26:0/1803666282 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f808010bd60 0x7f808010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 --2- 172.21.15.26:0/1803666282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080071830 0x7f808010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 -- 172.21.15.26:0/1803666282 >> 172.21.15.26:0/1803666282 conn(0x7f808006ca90 msgr2=0x7f808006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.289+0000 7f80863ff700 1 -- 172.21.15.26:0/1803666282 shutdown_connections 2023-07-13T22:16:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.290+0000 7f80863ff700 1 -- 172.21.15.26:0/1803666282 wait complete. 2023-07-13T22:16:35.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 Processor -- start 2023-07-13T22:16:35.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 -- start start 2023-07-13T22:16:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080070ee0 0x7f8080113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f808010bd60 0x7f80801b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80801170f0 con 0x7f808010bd60 2023-07-13T22:16:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8080117260 con 0x7f8080071830 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.292+0000 7f80863ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80801b4dd0 con 0x7f8080070ee0 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.293+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.293+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:41958/0 (socket says 172.21.15.26:41958) 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.293+0000 7f807f7fe700 1 -- 172.21.15.26:0/1785136843 learned_addr learned my addr 172.21.15.26:0/1785136843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807ffff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080070ee0 0x7f8080113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807f7fe700 1 -- 172.21.15.26:0/1785136843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080070ee0 msgr2=0x7f8080113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080070ee0 0x7f8080113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807f7fe700 1 -- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f808010bd60 msgr2=0x7f80801b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f808010bd60 0x7f80801b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.294+0000 7f807f7fe700 1 -- 172.21.15.26:0/1785136843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700095f0 con 0x7f8080071830 2023-07-13T22:16:35.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.295+0000 7f807f7fe700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f807400b550 tx=0x7f807400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.296+0000 7f807d7fa700 1 -- 172.21.15.26:0/1785136843 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074011830 con 0x7f8080071830 2023-07-13T22:16:35.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.296+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80801b4fd0 con 0x7f8080071830 2023-07-13T22:16:35.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.296+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801b54d0 con 0x7f8080071830 2023-07-13T22:16:35.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.297+0000 7f807d7fa700 1 -- 172.21.15.26:0/1785136843 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8074011e50 con 0x7f8080071830 2023-07-13T22:16:35.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.297+0000 7f807d7fa700 1 -- 172.21.15.26:0/1785136843 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074010e50 con 0x7f8080071830 2023-07-13T22:16:35.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.308+0000 7f8099cabb80 1 Processor -- start 2023-07-13T22:16:35.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.309+0000 7f8099cabb80 1 -- start start 2023-07-13T22:16:35.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe1a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe1c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd1fcb0 con 0x558f5fe1a4d0 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd72a40 con 0x558f5fe1bc70 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f8099cabb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd6bed0 con 0x558f5fe1ae20 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f807ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.310+0000 7f807ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:39994/0 (socket says 172.21.15.26:39994) 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.311+0000 7f807ffff700 1 -- 172.21.15.26:0/2935920471 learned_addr learned my addr 172.21.15.26:0/2935920471 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:35.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.311+0000 7f808499c700 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe1a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.312+0000 7f807ffff700 1 -- 172.21.15.26:0/2935920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 msgr2=0x558f5fe1c0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.312+0000 7f807ffff700 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe1c0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.312+0000 7f807ffff700 1 -- 172.21.15.26:0/2935920471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 msgr2=0x558f5fe1a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.312+0000 7f807ffff700 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe1a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.312+0000 7f807ffff700 1 -- 172.21.15.26:0/2935920471 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5fb99de0 con 0x558f5fe1ae20 2023-07-13T22:16:35.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.313+0000 7f807d7fa700 1 -- 172.21.15.26:0/1785136843 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8074011990 con 0x7f8080071830 2023-07-13T22:16:35.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.316+0000 7f807d7fa700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 0x7f80680a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.316+0000 7f807d7fa700 1 -- 172.21.15.26:0/1785136843 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80740cc2c0 con 0x7f8080071830 2023-07-13T22:16:35.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.318+0000 7f808499c700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 0x7f80680a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.320+0000 7f807ffff700 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f80700098e0 tx=0x7f8070009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=befa41cc44057bb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.321+0000 7f80667fc700 1 -- 172.21.15.26:0/2935920471 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807001d030 con 0x558f5fe1ae20 2023-07-13T22:16:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.321+0000 7f80667fc700 1 -- 172.21.15.26:0/2935920471 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807001e040 con 0x558f5fe1ae20 2023-07-13T22:16:35.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.321+0000 7f80667fc700 1 -- 172.21.15.26:0/2935920471 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070005620 con 0x558f5fe1ae20 2023-07-13T22:16:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.334+0000 7f8099cabb80 1 -- 172.21.15.26:0/2935920471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 msgr2=0x558f5fe1b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.334+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f80700098e0 tx=0x7f8070009ee0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f808499c700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 0x7f80680a3700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8078009690 tx=0x7f8078009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f8099cabb80 1 -- 172.21.15.26:0/2935920471 shutdown_connections 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe1a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe1c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2935920471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe1b6e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.345+0000 7f8099cabb80 1 -- 172.21.15.26:0/2935920471 >> 172.21.15.26:0/2935920471 conn(0x558f5fe0fda0 msgr2=0x558f5fe151d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.353+0000 7f8099cabb80 1 -- 172.21.15.26:0/2935920471 shutdown_connections 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.356+0000 7f8099cabb80 1 -- 172.21.15.26:0/2935920471 wait complete. 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.356+0000 7f8099cabb80 1 -- start start 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.357+0000 7f8099cabb80 1 Processor -- start 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.364+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 0x558f5fe24200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.364+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.364+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 0x558f5fe2b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.364+0000 7f8099cabb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd0bac0 con 0x558f5fe1bc70 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.365+0000 7f8099cabb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd0b7c0 con 0x558f5fe1ae20 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.365+0000 7f8099cabb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fbef920 con 0x558f5fe1a4d0 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.365+0000 7f8099cabb80 10 client.0 _ll_register_callbacks cb 0x558f5fd09760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.366+0000 7f808499c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 0x558f5fe2b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.366+0000 7f808499c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 0x558f5fe2b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43394/0 (socket says 172.21.15.26:43394) 2023-07-13T22:16:35.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.366+0000 7f808499c700 1 -- 172.21.15.26:0/2584062419 learned_addr learned my addr 172.21.15.26:0/2584062419 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:35.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.373+0000 7f807f7fe700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.373+0000 7f807ffff700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 0x558f5fe24200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.384+0000 7f807ffff700 1 -- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 msgr2=0x558f5fe24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.384+0000 7f807ffff700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.384+0000 7f807ffff700 1 -- 172.21.15.26:0/2584062419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 msgr2=0x558f5fe2b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.384+0000 7f807ffff700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 0x558f5fe2b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.384+0000 7f807ffff700 1 -- 172.21.15.26:0/2584062419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8080117640 con 0x558f5fe1a4d0 2023-07-13T22:16:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.396+0000 7f807ffff700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 0x558f5fe24200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8070003790 tx=0x7f80700055d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1242f0710863a80c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.396+0000 7f805f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:35.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.399+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807001d070 con 0x558f5fe1a4d0 2023-07-13T22:16:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.399+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80700056c0 con 0x558f5fe1a4d0 2023-07-13T22:16:35.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.399+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f5fcc54f0 con 0x558f5fe1a4d0 2023-07-13T22:16:35.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.400+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8070005920 con 0x558f5fe1a4d0 2023-07-13T22:16:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.408+0000 7f8099cabb80 10 client.25771 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.408+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f5fb82c60 con 0x558f5fe1a4d0 2023-07-13T22:16:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.409+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f5fcc54f0 con 0x558f5fe1a4d0 2023-07-13T22:16:35.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.409+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8070032050 con 0x558f5fe1a4d0 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 10 client.25771 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 10 client.25771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 20 client.25771 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe2fa20 0x558f5fe31e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 4 client.25771 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:35.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.410+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f5fd64eb0 con 0x558f5fe2fa20 2023-07-13T22:16:35.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.413+0000 7f807f7fe700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe2fa20 0x558f5fe31e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.419+0000 7f807f7fe700 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe2fa20 0x558f5fe31e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5e720e1badd847 server_cookie=7f8acf55e63abbee in_seq=0 out_seq=0 2023-07-13T22:16:35.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.419+0000 7f805f7fe700 10 client.25771 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:35.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.420+0000 7f805f7fe700 1 -- 172.21.15.26:0/2584062419 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f5fd64eb0 con 0x558f5fe2fa20 2023-07-13T22:16:35.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.420+0000 7f805f7fe700 10 client.25771 handle_command_reply: tid=0 2023-07-13T22:16:35.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.420+0000 7f8099cabb80 1 client.25771 shutdown 2023-07-13T22:16:35.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.421+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 msgr2=0x558f5fe24200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.421+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 0x558f5fe24200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8070003790 tx=0x7f80700055d0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.421+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 shutdown_connections 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.421+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1bc70 0x558f5fe2b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.421+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe2fa20 0x558f5fe31e30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1a4d0 0x558f5fe24200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2584062419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 >> 172.21.15.26:0/2584062419 conn(0x558f5fe0fda0 msgr2=0x558f5fe148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f805f7fe700 0 client.25771 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:35.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 shutdown_connections 2023-07-13T22:16:35.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.422+0000 7f8099cabb80 1 -- 172.21.15.26:0/2584062419 wait complete. 2023-07-13T22:16:35.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.424+0000 7f8099cabb80 20 client.25771 trim_cache size 0 max 16384 2023-07-13T22:16:35.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.466+0000 7f8099cabb80 1 Processor -- start 2023-07-13T22:16:35.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.466+0000 7f8099cabb80 1 -- start start 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe22e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1bc70 0x558f5fe1ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fe23dd0 con 0x558f5fe1a4d0 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fe160f0 con 0x558f5fe1ae20 2023-07-13T22:16:35.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.467+0000 7f8099cabb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fd64eb0 con 0x558f5fe1bc70 2023-07-13T22:16:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.468+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.468+0000 7f807f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:41976/0 (socket says 172.21.15.26:41976) 2023-07-13T22:16:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.468+0000 7f807f7fe700 1 -- 172.21.15.26:0/315551299 learned_addr learned my addr 172.21.15.26:0/315551299 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:35.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.468+0000 7f808499c700 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1bc70 0x558f5fe1ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.468+0000 7f807ffff700 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe22e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.471+0000 7f807f7fe700 1 -- 172.21.15.26:0/315551299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1bc70 msgr2=0x558f5fe1ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.471+0000 7f807f7fe700 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1bc70 0x558f5fe1ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.471+0000 7f807f7fe700 1 -- 172.21.15.26:0/315551299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 msgr2=0x558f5fe22e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.471+0000 7f807f7fe700 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe22e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.471+0000 7f807f7fe700 1 -- 172.21.15.26:0/315551299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f5fbc0250 con 0x558f5fe1ae20 2023-07-13T22:16:35.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.479+0000 7f807f7fe700 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8074009430 tx=0x7f8074084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6134bab61501e90 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.482+0000 7f8066ffd700 1 -- 172.21.15.26:0/315551299 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740a1ad0 con 0x558f5fe1ae20 2023-07-13T22:16:35.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.482+0000 7f8066ffd700 1 -- 172.21.15.26:0/315551299 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80740977c0 con 0x558f5fe1ae20 2023-07-13T22:16:35.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.482+0000 7f8066ffd700 1 -- 172.21.15.26:0/315551299 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074099b40 con 0x558f5fe1ae20 2023-07-13T22:16:35.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.483+0000 7f8099cabb80 1 -- 172.21.15.26:0/315551299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 msgr2=0x558f5fe2b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.483+0000 7f8099cabb80 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8074009430 tx=0x7f8074084040 comp rx=0 tx=0).stop 2023-07-13T22:16:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.491+0000 7f8099cabb80 1 -- 172.21.15.26:0/315551299 shutdown_connections 2023-07-13T22:16:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.492+0000 7f8099cabb80 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe22e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.492+0000 7f8099cabb80 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1bc70 0x558f5fe1ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.492+0000 7f8099cabb80 1 --2- 172.21.15.26:0/315551299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.492+0000 7f8099cabb80 1 -- 172.21.15.26:0/315551299 >> 172.21.15.26:0/315551299 conn(0x558f5fe0fda0 msgr2=0x558f5fe31a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.497+0000 7f8099cabb80 1 -- 172.21.15.26:0/315551299 shutdown_connections 2023-07-13T22:16:35.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.508+0000 7f8099cabb80 1 -- 172.21.15.26:0/315551299 wait complete. 2023-07-13T22:16:35.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.509+0000 7f8099cabb80 1 -- start start 2023-07-13T22:16:35.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.509+0000 7f8099cabb80 1 Processor -- start 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fe29d40 con 0x558f5fe1a4d0 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fe29eb0 con 0x558f5fe1bc70 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f5fe2a070 con 0x558f5fe1ae20 2023-07-13T22:16:35.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.510+0000 7f8099cabb80 10 client.0 _ll_register_callbacks cb 0x7f8070005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:35.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.511+0000 7f808499c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.511+0000 7f808499c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:41990/0 (socket says 172.21.15.26:41990) 2023-07-13T22:16:35.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.511+0000 7f808499c700 1 -- 172.21.15.26:0/2945234652 learned_addr learned my addr 172.21.15.26:0/2945234652 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:35.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.518+0000 7f807f7fe700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 -- 172.21.15.26:0/2945234652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 msgr2=0x558f5fe2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 -- 172.21.15.26:0/2945234652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 msgr2=0x558f5fe2b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:35.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 -- 172.21.15.26:0/2945234652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801b54d0 con 0x558f5fe1bc70 2023-07-13T22:16:35.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.522+0000 7f808499c700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80780169b0 tx=0x7f8078009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cff7f7f281922969 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.523+0000 7f805f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.523+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078014a70 con 0x558f5fe1bc70 2023-07-13T22:16:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.523+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f807001e040 con 0x558f5fe1bc70 2023-07-13T22:16:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.523+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8078014bd0 con 0x558f5fe1bc70 2023-07-13T22:16:35.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.524+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8078014d90 con 0x558f5fe1bc70 2023-07-13T22:16:35.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.524+0000 7f8099cabb80 10 client.25796 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:35.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.524+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f5fb982b0 con 0x558f5fe1bc70 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.532+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f807001e040 con 0x558f5fe1bc70 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8078021050 con 0x558f5fe1bc70 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 10 client.25796 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 20 client.25796 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe84de0 0x558f5fe871f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 4 client.25796 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:16:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.533+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f5fd27050 con 0x558f5fe84de0 2023-07-13T22:16:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.542+0000 7f807ffff700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe84de0 0x558f5fe871f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.551+0000 7f807ffff700 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe84de0 0x558f5fe871f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955363a8d61cae91 server_cookie=e89bd82f7cb08cf7 in_seq=0 out_seq=0 2023-07-13T22:16:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.551+0000 7f805f7fe700 10 client.25796 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.554+0000 7f805f7fe700 1 -- 172.21.15.26:0/2945234652 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f5fd27050 con 0x558f5fe84de0 2023-07-13T22:16:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.554+0000 7f805f7fe700 10 client.25796 handle_command_reply: tid=0 2023-07-13T22:16:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.554+0000 7f8099cabb80 1 client.25796 shutdown 2023-07-13T22:16:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 msgr2=0x558f5fe29660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80780169b0 tx=0x7f8078009620 comp rx=0 tx=0).stop 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 shutdown_connections 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558f5fe1a4d0 0x558f5fe2b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558f5fe84de0 0x558f5fe871f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f5fe1bc70 0x558f5fe29660 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 --2- 172.21.15.26:0/2945234652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558f5fe1ae20 0x558f5fe2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 >> 172.21.15.26:0/2945234652 conn(0x558f5fe0fda0 msgr2=0x558f5fe22e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.555+0000 7f805f7fe700 0 client.25796 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.556+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 shutdown_connections 2023-07-13T22:16:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.556+0000 7f8099cabb80 1 -- 172.21.15.26:0/2945234652 wait complete. 2023-07-13T22:16:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.557+0000 7f8099cabb80 20 client.25796 trim_cache size 0 max 16384 2023-07-13T22:16:35.546 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:16:35.547 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:16:35.547 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:16:35.547 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:16:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 msgr2=0x7f80680a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 0x7f80680a3700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8078009690 tx=0x7f8078009040 comp rx=0 tx=0).stop 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 msgr2=0x7f8080114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f807400b550 tx=0x7f807400b860 comp rx=0 tx=0).stop 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 shutdown_connections 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80680a1250 0x7f80680a3700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f808010bd60 0x7f80801b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8080070ee0 0x7f8080113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 --2- 172.21.15.26:0/1785136843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8080071830 0x7f8080114010 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.558+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 >> 172.21.15.26:0/1785136843 conn(0x7f808006ca90 msgr2=0x7f808006dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.559+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 shutdown_connections 2023-07-13T22:16:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:35.559+0000 7f80863ff700 1 -- 172.21.15.26:0/1785136843 wait complete. 2023-07-13T22:16:35.588 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:16:35.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:16:36.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.155+0000 7fc738bff700 1 -- 172.21.15.26:0/102602178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 msgr2=0x7fc7340712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.155+0000 7fc738bff700 1 --2- 172.21.15.26:0/102602178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc7340712e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098f0 tx=0x7fc728009c00 comp rx=0 tx=0).stop 2023-07-13T22:16:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 -- 172.21.15.26:0/102602178 shutdown_connections 2023-07-13T22:16:36.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 --2- 172.21.15.26:0/102602178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73410bd60 0x7fc73410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 --2- 172.21.15.26:0/102602178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc7340712e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 --2- 172.21.15.26:0/102602178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc734071820 0x7fc73410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 -- 172.21.15.26:0/102602178 >> 172.21.15.26:0/102602178 conn(0x7fc73406ca40 msgr2=0x7fc73406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:36.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 -- 172.21.15.26:0/102602178 shutdown_connections 2023-07-13T22:16:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.156+0000 7fc738bff700 1 -- 172.21.15.26:0/102602178 wait complete. 2023-07-13T22:16:36.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.158+0000 7fc738bff700 1 Processor -- start 2023-07-13T22:16:36.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 -- start start 2023-07-13T22:16:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc734071820 0x7fc734113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73410bd60 0x7fc7341b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7341170b0 con 0x7fc734071820 2023-07-13T22:16:36.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734117220 con 0x7fc73410bd60 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.159+0000 7fc738bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7341b4dc0 con 0x7fc734070ed0 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.161+0000 7fc73259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.161+0000 7fc73259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40036/0 (socket says 172.21.15.26:40036) 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.161+0000 7fc73259c700 1 -- 172.21.15.26:0/1846829864 learned_addr learned my addr 172.21.15.26:0/1846829864 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 -- 172.21.15.26:0/1846829864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73410bd60 msgr2=0x7fc7341b4880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73410bd60 0x7fc7341b4880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 -- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc734071820 msgr2=0x7fc734113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc734071820 0x7fc734113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 -- 172.21.15.26:0/1846829864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728009520 con 0x7fc734070ed0 2023-07-13T22:16:36.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.162+0000 7fc73259c700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b7c0 tx=0x7fc72800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:36.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.166+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728005310 con 0x7fc734070ed0 2023-07-13T22:16:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.166+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc728005470 con 0x7fc734070ed0 2023-07-13T22:16:36.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.166+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728017640 con 0x7fc734070ed0 2023-07-13T22:16:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.166+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7341b4fc0 con 0x7fc734070ed0 2023-07-13T22:16:36.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.166+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7341b5330 con 0x7fc734070ed0 2023-07-13T22:16:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.172+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc714005320 con 0x7fc734070ed0 2023-07-13T22:16:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.176+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc728003680 con 0x7fc734070ed0 2023-07-13T22:16:36.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.181+0000 7fc7237fe700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 0x7fc71c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:36.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.181+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7280cd220 con 0x7fc734070ed0 2023-07-13T22:16:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.182+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc7280cd600 con 0x7fc734070ed0 2023-07-13T22:16:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.184+0000 7fc731d9b700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 0x7fc71c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.185+0000 7fc731d9b700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 0x7fc71c0a3870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc734072530 tx=0x7fc7240096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:36.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.783+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc714005190 con 0x7fc734070ed0 2023-07-13T22:16:36.782 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:16:36.782 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:15:26.683417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:16:36.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.787+0000 7fc7237fe700 1 -- 172.21.15.26:0/1846829864 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10892 (secure 0 0 0) 0x7fc728096ac0 con 0x7fc734070ed0 2023-07-13T22:16:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.812+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 msgr2=0x7fc71c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:36.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.812+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 0x7fc71c0a3870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc734072530 tx=0x7fc7240096c0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.812+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 msgr2=0x7fc734113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.812+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b7c0 tx=0x7fc72800b8a0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 shutdown_connections 2023-07-13T22:16:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc73410bd60 0x7fc7341b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc734070ed0 0x7fc734113a90 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc72800b7c0 tx=0x7fc72800b8a0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc71c0a13c0 0x7fc71c0a3870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 --2- 172.21.15.26:0/1846829864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc734071820 0x7fc734113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.817+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 >> 172.21.15.26:0/1846829864 conn(0x7fc73406ca40 msgr2=0x7fc73406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:36.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.834+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 shutdown_connections 2023-07-13T22:16:36.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:36.834+0000 7fc738bff700 1 -- 172.21.15.26:0/1846829864 wait complete. 2023-07-13T22:16:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2023-07-13T22:16:36.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:16:36.941 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:36 smithi117 ceph-mon[129074]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 84 op/s 2023-07-13T22:16:36.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:16:36.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:16:36.942 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:36 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:36.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:36 smithi026 ceph-mon[95647]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 84 op/s 2023-07-13T22:16:36.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:16:36.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:16:36.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:36 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:36.992 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:36 smithi121 ceph-mon[129187]: pgmap v397: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 84 op/s 2023-07-13T22:16:36.992 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:16:36.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:16:36.993 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:36 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:16:37.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.717+0000 7f22fafff700 1 -- 172.21.15.26:0/2262788728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f410bd60 msgr2=0x7f22f410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.717+0000 7f22fafff700 1 --2- 172.21.15.26:0/2262788728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f410bd60 0x7f22f410e1f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009990 tx=0x7f22e8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.718+0000 7f22fafff700 1 -- 172.21.15.26:0/2262788728 shutdown_connections 2023-07-13T22:16:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.718+0000 7f22fafff700 1 --2- 172.21.15.26:0/2262788728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22f4070ee0 0x7f22f40712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.718+0000 7f22fafff700 1 --2- 172.21.15.26:0/2262788728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f410bd60 0x7f22f410e1f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.718+0000 7f22fafff700 1 --2- 172.21.15.26:0/2262788728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f4071830 0x7f22f410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.718+0000 7f22fafff700 1 -- 172.21.15.26:0/2262788728 >> 172.21.15.26:0/2262788728 conn(0x7f22f406ca90 msgr2=0x7f22f406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.729+0000 7f22fafff700 1 -- 172.21.15.26:0/2262788728 shutdown_connections 2023-07-13T22:16:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.729+0000 7f22fafff700 1 -- 172.21.15.26:0/2262788728 wait complete. 2023-07-13T22:16:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.731+0000 7f22fafff700 1 Processor -- start 2023-07-13T22:16:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.731+0000 7f22fafff700 1 -- start start 2023-07-13T22:16:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22f4071830 0x7f22f41a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 0x7f22f41a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 0x7f22f41ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41ac6d0 con 0x7f22f4071830 2023-07-13T22:16:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f410ec40 con 0x7f22f410bd60 2023-07-13T22:16:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.732+0000 7f22fafff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f410edd0 con 0x7f22f41a9d20 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.733+0000 7f22f959c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 0x7f22f41ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.733+0000 7f22f959c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 0x7f22f41ac100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40052/0 (socket says 172.21.15.26:40052) 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.733+0000 7f22f959c700 1 -- 172.21.15.26:0/1316221585 learned_addr learned my addr 172.21.15.26:0/1316221585 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.733+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 0x7f22f41a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.734+0000 7f22f3fff700 1 -- 172.21.15.26:0/1316221585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 msgr2=0x7f22f41ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.734+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 0x7f22f41ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.734+0000 7f22f3fff700 1 -- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22f4071830 msgr2=0x7f22f41a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.734+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22f4071830 0x7f22f41a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.734+0000 7f22f3fff700 1 -- 172.21.15.26:0/1316221585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80095f0 con 0x7f22f410bd60 2023-07-13T22:16:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 0x7f22f41a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d740 tx=0x7f22e400da50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22f1ffb700 1 -- 172.21.15.26:0/1316221585 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4004220 con 0x7f22f410bd60 2023-07-13T22:16:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22f1ffb700 1 -- 172.21.15.26:0/1316221585 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e4004380 con 0x7f22f410bd60 2023-07-13T22:16:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22f1ffb700 1 -- 172.21.15.26:0/1316221585 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4003b50 con 0x7f22f410bd60 2023-07-13T22:16:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f410f0b0 con 0x7f22f410bd60 2023-07-13T22:16:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.746+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f410f5f0 con 0x7f22f410bd60 2023-07-13T22:16:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.753+0000 7f230e8fcb80 1 Processor -- start 2023-07-13T22:16:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.762+0000 7f230e8fcb80 1 -- start start 2023-07-13T22:16:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 0x55804512a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 0x55804512b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804512c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804502fcb0 con 0x55804512ae20 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558045082a40 con 0x55804512a4d0 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.763+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804507bed0 con 0x55804512bc70 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.774+0000 7f22f959c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 0x55804512a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.774+0000 7f22f959c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 0x55804512a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42048/0 (socket says 172.21.15.26:42048) 2023-07-13T22:16:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.774+0000 7f22f959c700 1 -- 172.21.15.26:0/597032250 learned_addr learned my addr 172.21.15.26:0/597032250 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.776+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 0x55804512b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.777+0000 7f22f8d9b700 1 -- 172.21.15.26:0/597032250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 msgr2=0x55804512c0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.777+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804512c0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.777+0000 7f22f8d9b700 1 -- 172.21.15.26:0/597032250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 msgr2=0x55804512a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.777+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 0x55804512a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.777+0000 7f22f8d9b700 1 -- 172.21.15.26:0/597032250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558044ea9de0 con 0x55804512ae20 2023-07-13T22:16:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.779+0000 7f22f1ffb700 1 -- 172.21.15.26:0/1316221585 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f22e4003cb0 con 0x7f22f410bd60 2023-07-13T22:16:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.782+0000 7f22f1ffb700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 0x7f22dc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.782+0000 7f22f1ffb700 1 -- 172.21.15.26:0/1316221585 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22e40cbf40 con 0x7f22f410bd60 2023-07-13T22:16:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.789+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 0x55804512b6e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f22f41075f0 tx=0x7f22ec00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fdd0be69339efa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.790+0000 7f22f959c700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 0x7f22dc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.790+0000 7f22daffd700 1 -- 172.21.15.26:0/597032250 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00bd70 con 0x55804512ae20 2023-07-13T22:16:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.791+0000 7f22daffd700 1 -- 172.21.15.26:0/597032250 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec0138d0 con 0x55804512ae20 2023-07-13T22:16:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.791+0000 7f22daffd700 1 -- 172.21.15.26:0/597032250 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec012840 con 0x55804512ae20 2023-07-13T22:16:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.792+0000 7f230e8fcb80 1 -- 172.21.15.26:0/597032250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 msgr2=0x55804512b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.792+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 0x55804512b6e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f22f41075f0 tx=0x7f22ec00b410 comp rx=0 tx=0).stop 2023-07-13T22:16:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.815+0000 7f230e8fcb80 1 -- 172.21.15.26:0/597032250 shutdown_connections 2023-07-13T22:16:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.815+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804512c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.815+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512a4d0 0x55804512a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.815+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/597032250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512ae20 0x55804512b6e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.815+0000 7f230e8fcb80 1 -- 172.21.15.26:0/597032250 >> 172.21.15.26:0/597032250 conn(0x55804511fda0 msgr2=0x5580451251d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.826+0000 7f230e8fcb80 1 -- 172.21.15.26:0/597032250 shutdown_connections 2023-07-13T22:16:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.826+0000 7f230e8fcb80 1 -- 172.21.15.26:0/597032250 wait complete. 2023-07-13T22:16:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.828+0000 7f230e8fcb80 1 -- start start 2023-07-13T22:16:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.828+0000 7f230e8fcb80 1 Processor -- start 2023-07-13T22:16:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.829+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045134240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.829+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.829+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512bc70 0x55804513b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.832+0000 7f22f959c700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 0x7f22dc0a3800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005fe0 tx=0x7f22e800b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.832+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40066/0 (socket says 172.21.15.26:40066) 2023-07-13T22:16:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804501bac0 con 0x55804512a4d0 2023-07-13T22:16:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804501b7c0 con 0x55804512bc70 2023-07-13T22:16:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558044eff920 con 0x55804512ae20 2023-07-13T22:16:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f230e8fcb80 10 client.0 _ll_register_callbacks cb 0x558045019760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.833+0000 7f22f3fff700 1 -- 172.21.15.26:0/1764432117 learned_addr learned my addr 172.21.15.26:0/1764432117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.845+0000 7f22f3fff700 1 -- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512bc70 msgr2=0x55804513b7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.845+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512bc70 0x55804513b7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.845+0000 7f22f3fff700 1 -- 172.21.15.26:0/1764432117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 msgr2=0x558045134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.845+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.845+0000 7f22f3fff700 1 -- 172.21.15.26:0/1764432117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec009970 con 0x55804512ae20 2023-07-13T22:16:37.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.850+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22e4099290 tx=0x7f22e4098f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b526d8313e05d61 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.850+0000 7f22d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.850+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e409d170 con 0x55804512ae20 2023-07-13T22:16:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.850+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22e409d330 con 0x55804512ae20 2023-07-13T22:16:37.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.851+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558044fd54f0 con 0x55804512ae20 2023-07-13T22:16:37.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.862+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22e40a1750 con 0x55804512ae20 2023-07-13T22:16:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.873+0000 7f230e8fcb80 10 client.25792 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:37.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.873+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558044e92c60 con 0x55804512ae20 2023-07-13T22:16:37.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.882+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558044fd54f0 con 0x55804512ae20 2023-07-13T22:16:37.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.882+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f22e40950f0 con 0x55804512ae20 2023-07-13T22:16:37.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 10 client.25792 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:37.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 10 client.25792 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:16:37.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 20 client.25792 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:37.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558045140020 0x558045142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 4 client.25792 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:37.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.883+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558045074eb0 con 0x558045140020 2023-07-13T22:16:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.893+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558045140020 0x558045142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.915+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558045140020 0x558045142430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ee22111044a996d server_cookie=ae61a21a5e2f2b39 in_seq=0 out_seq=0 2023-07-13T22:16:37.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.916+0000 7f22d3fff700 10 client.25792 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:16:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.917+0000 7f22d3fff700 1 -- 172.21.15.26:0/1764432117 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558045074eb0 con 0x558045140020 2023-07-13T22:16:37.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.917+0000 7f22d3fff700 10 client.25792 handle_command_reply: tid=0 2023-07-13T22:16:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.917+0000 7f230e8fcb80 1 client.25792 shutdown 2023-07-13T22:16:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 msgr2=0x558045134780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f22e4099290 tx=0x7f22e4098f40 comp rx=0 tx=0).stop 2023-07-13T22:16:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 shutdown_connections 2023-07-13T22:16:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558045140020 0x558045142430 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512bc70 0x55804513b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1764432117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512ae20 0x558045134780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.918+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 >> 172.21.15.26:0/1764432117 conn(0x55804511fda0 msgr2=0x5580451248b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.919+0000 7f22d3fff700 0 client.25792 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:16:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.919+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 shutdown_connections 2023-07-13T22:16:37.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.919+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1764432117 wait complete. 2023-07-13T22:16:37.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.920+0000 7f230e8fcb80 20 client.25792 trim_cache size 0 max 16384 2023-07-13T22:16:37.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.952+0000 7f230e8fcb80 1 Processor -- start 2023-07-13T22:16:37.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.952+0000 7f230e8fcb80 1 -- start start 2023-07-13T22:16:37.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x55804513b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558045133e10 con 0x55804512a4d0 2023-07-13T22:16:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804511f970 con 0x55804512ae20 2023-07-13T22:16:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558045126cc0 con 0x55804512bc70 2023-07-13T22:16:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.953+0000 7f22f959c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.954+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x55804513b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.954+0000 7f22f959c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40082/0 (socket says 172.21.15.26:40082) 2023-07-13T22:16:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.954+0000 7f22f959c700 1 -- 172.21.15.26:0/3652861834 learned_addr learned my addr 172.21.15.26:0/3652861834 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 -- 172.21.15.26:0/3652861834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 msgr2=0x55804513b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x55804513b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 -- 172.21.15.26:0/3652861834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 msgr2=0x558045132ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045132ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 -- 172.21.15.26:0/3652861834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558044ed0250 con 0x55804512bc70 2023-07-13T22:16:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.955+0000 7f22f959c700 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f22e800bc00 tx=0x7f22e801a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73898c6ec8a123d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.956+0000 7f22db7fe700 1 -- 172.21.15.26:0/3652861834 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e801d040 con 0x55804512bc70 2023-07-13T22:16:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.956+0000 7f22db7fe700 1 -- 172.21.15.26:0/3652861834 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e8023430 con 0x55804512bc70 2023-07-13T22:16:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.957+0000 7f230e8fcb80 1 -- 172.21.15.26:0/3652861834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 msgr2=0x558045142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:37.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.957+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f22e800bc00 tx=0x7f22e801a070 comp rx=0 tx=0).stop 2023-07-13T22:16:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.961+0000 7f230e8fcb80 1 -- 172.21.15.26:0/3652861834 shutdown_connections 2023-07-13T22:16:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.961+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045132ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.961+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x558045142030 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.961+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/3652861834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x55804513b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.961+0000 7f230e8fcb80 1 -- 172.21.15.26:0/3652861834 >> 172.21.15.26:0/3652861834 conn(0x55804511fda0 msgr2=0x558045146d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.976+0000 7f230e8fcb80 1 -- 172.21.15.26:0/3652861834 shutdown_connections 2023-07-13T22:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.976+0000 7f230e8fcb80 1 -- 172.21.15.26:0/3652861834 wait complete. 2023-07-13T22:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.978+0000 7f230e8fcb80 1 -- start start 2023-07-13T22:16:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.978+0000 7f230e8fcb80 1 Processor -- start 2023-07-13T22:16:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.979+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045137910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.979+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x558045137e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.980+0000 7f230e8fcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804513a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.980+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804513ada0 con 0x55804512a4d0 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.980+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804513af10 con 0x55804512ae20 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.980+0000 7f230e8fcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804513b0d0 con 0x55804512bc70 2023-07-13T22:16:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.980+0000 7f230e8fcb80 10 client.0 _ll_register_callbacks cb 0x7f22e4099160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:37.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.984+0000 7f22f8d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045137910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.984+0000 7f22f8d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045137910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43474/0 (socket says 172.21.15.26:43474) 2023-07-13T22:16:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.984+0000 7f22f8d9b700 1 -- 172.21.15.26:0/1030530268 learned_addr learned my addr 172.21.15.26:0/1030530268 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.986+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x558045137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.986+0000 7f22f959c700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804513a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.993+0000 7f22f3fff700 1 -- 172.21.15.26:0/1030530268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 msgr2=0x55804513a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.993+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804513a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.993+0000 7f22f3fff700 1 -- 172.21.15.26:0/1030530268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 msgr2=0x558045137910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.993+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045137910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.993+0000 7f22f3fff700 1 -- 172.21.15.26:0/1030530268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f4117640 con 0x55804512ae20 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.994+0000 7f22f3fff700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x558045137e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22e4000c00 tx=0x7f22e40a14f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de396eaba05583f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:37.997+0000 7f22d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.005+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e409d170 con 0x55804512ae20 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.005+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22e40985f0 con 0x55804512ae20 2023-07-13T22:16:37.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.005+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22ec0138d0 con 0x55804512ae20 2023-07-13T22:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.009+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22e409d330 con 0x55804512ae20 2023-07-13T22:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.009+0000 7f230e8fcb80 10 client.25814 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.009+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558044ea82b0 con 0x55804512ae20 2023-07-13T22:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.009+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec0138d0 con 0x55804512ae20 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f22e40c3b30 con 0x55804512ae20 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 10 client.25814 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 20 client.25814 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5580451947b0 0x558045196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 4 client.25814 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.011+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558045037050 con 0x5580451947b0 2023-07-13T22:16:38.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.012+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5580451947b0 0x558045196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.017+0000 7f22f8d9b700 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5580451947b0 0x558045196bc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8c1697b60e2c57d9 server_cookie=413fde808b535c2c in_seq=0 out_seq=0 2023-07-13T22:16:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.017+0000 7f22d3fff700 10 client.25814 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:16:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.017+0000 7f22d3fff700 1 -- 172.21.15.26:0/1030530268 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558045037050 con 0x5580451947b0 2023-07-13T22:16:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.017+0000 7f22d3fff700 10 client.25814 handle_command_reply: tid=0 2023-07-13T22:16:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.017+0000 7f230e8fcb80 1 client.25814 shutdown 2023-07-13T22:16:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 msgr2=0x558045137e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x558045137e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22e4000c00 tx=0x7f22e40a14f0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 shutdown_connections 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5580451947b0 0x558045196bc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55804512a4d0 0x558045137910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55804512bc70 0x55804513a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 --2- 172.21.15.26:0/1030530268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55804512ae20 0x558045137e50 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22e4000c00 tx=0x7f22e40a14f0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 >> 172.21.15.26:0/1030530268 conn(0x55804511fda0 msgr2=0x558045132ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.018+0000 7f22d3fff700 0 client.25814 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.019+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 shutdown_connections 2023-07-13T22:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.019+0000 7f230e8fcb80 1 -- 172.21.15.26:0/1030530268 wait complete. 2023-07-13T22:16:38.009 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.019+0000 7f230e8fcb80 20 client.25814 trim_cache size 0 max 16384 2023-07-13T22:16:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.022+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 msgr2=0x7f22dc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.022+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 0x7f22dc0a3800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005fe0 tx=0x7f22e800b540 comp rx=0 tx=0).stop 2023-07-13T22:16:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.022+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 msgr2=0x7f22f41a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.022+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 0x7f22f41a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f22e400d740 tx=0x7f22e400da50 comp rx=0 tx=0).stop 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 shutdown_connections 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f22dc0a1350 0x7f22dc0a3800 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22f4071830 0x7f22f41a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f22f41a9d20 0x7f22f41ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 --2- 172.21.15.26:0/1316221585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22f410bd60 0x7f22f41a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.023+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 >> 172.21.15.26:0/1316221585 conn(0x7f22f406ca90 msgr2=0x7f22f406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.024+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 shutdown_connections 2023-07-13T22:16:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.024+0000 7f22fafff700 1 -- 172.21.15.26:0/1316221585 wait complete. 2023-07-13T22:16:38.073 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:37 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1846829864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:16:38.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:16:38.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:37 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1846829864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:16:38.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:37 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1846829864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:16:38.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.810+0000 7ff5535ff700 1 -- 172.21.15.26:0/1908260958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c10bd60 msgr2=0x7ff54c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.810+0000 7ff5535ff700 1 --2- 172.21.15.26:0/1908260958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c10bd60 0x7ff54c10e1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff54400b0a0 tx=0x7ff54400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 -- 172.21.15.26:0/1908260958 shutdown_connections 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 --2- 172.21.15.26:0/1908260958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c10bd60 0x7ff54c10e1f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 --2- 172.21.15.26:0/1908260958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 0x7ff54c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 --2- 172.21.15.26:0/1908260958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff54c071830 0x7ff54c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 -- 172.21.15.26:0/1908260958 >> 172.21.15.26:0/1908260958 conn(0x7ff54c06ca90 msgr2=0x7ff54c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 -- 172.21.15.26:0/1908260958 shutdown_connections 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.822+0000 7ff5535ff700 1 -- 172.21.15.26:0/1908260958 wait complete. 2023-07-13T22:16:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.824+0000 7ff5535ff700 1 Processor -- start 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.824+0000 7ff5535ff700 1 -- start start 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.824+0000 7ff5535ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 0x7ff54c113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.824+0000 7ff5535ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff5535ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff54c10bd60 0x7ff54c1b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff5535ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1146c0 con 0x7ff54c10bd60 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff5535ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c114830 con 0x7ff54c070ee0 2023-07-13T22:16:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff5535ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1b4e70 con 0x7ff54c071830 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff550b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff550b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40094/0 (socket says 172.21.15.26:40094) 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff550b9a700 1 -- 172.21.15.26:0/270129439 learned_addr learned my addr 172.21.15.26:0/270129439 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.825+0000 7ff55139b700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 0x7ff54c113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.826+0000 7ff550b9a700 1 -- 172.21.15.26:0/270129439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 msgr2=0x7ff54c113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.826+0000 7ff550b9a700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 0x7ff54c113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.826+0000 7ff550b9a700 1 -- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff54c10bd60 msgr2=0x7ff54c1b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.826+0000 7ff550b9a700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff54c10bd60 0x7ff54c1b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.826+0000 7ff550b9a700 1 -- 172.21.15.26:0/270129439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009d00 con 0x7ff54c071830 2023-07-13T22:16:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.827+0000 7ff550b9a700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b500 tx=0x7ff53c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.833+0000 7ff5427fc700 1 -- 172.21.15.26:0/270129439 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c011830 con 0x7ff54c071830 2023-07-13T22:16:38.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.833+0000 7ff5427fc700 1 -- 172.21.15.26:0/270129439 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff53c011e50 con 0x7ff54c071830 2023-07-13T22:16:38.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.833+0000 7ff5427fc700 1 -- 172.21.15.26:0/270129439 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00f2e0 con 0x7ff54c071830 2023-07-13T22:16:38.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.834+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1b5150 con 0x7ff54c071830 2023-07-13T22:16:38.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.834+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1b5690 con 0x7ff54c071830 2023-07-13T22:16:38.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.844+0000 7ff5427fc700 1 -- 172.21.15.26:0/270129439 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff53c00f440 con 0x7ff54c071830 2023-07-13T22:16:38.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.847+0000 7ff5427fc700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 0x7ff5380a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.847+0000 7ff5427fc700 1 -- 172.21.15.26:0/270129439 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff53c0cd280 con 0x7ff54c071830 2023-07-13T22:16:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.850+0000 7ff55139b700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 0x7ff5380a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.851+0000 7ff566fc6b80 1 Processor -- start 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.851+0000 7ff566fc6b80 1 -- start start 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c1f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c200e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325b23cb0 con 0x559325c1fc70 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325b76a40 con 0x559325c1ee20 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.852+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325b6fed0 con 0x559325c1e4d0 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40108/0 (socket says 172.21.15.26:40108) 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 -- 172.21.15.26:0/858648803 learned_addr learned my addr 172.21.15.26:0/858648803 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff550b9a700 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c1f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 -- 172.21.15.26:0/858648803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 msgr2=0x559325c1f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c1f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 -- 172.21.15.26:0/858648803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 msgr2=0x559325c200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.853+0000 7ff55139b700 1 -- 172.21.15.26:0/858648803 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932599dde0 con 0x559325c1e4d0 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.855+0000 7ff55139b700 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff548014790 tx=0x7ff548014aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e270da21346f93 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.856+0000 7ff55139b700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 0x7ff5380a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff548021000 tx=0x7ff54801a000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.869+0000 7ff5377fe700 1 -- 172.21.15.26:0/858648803 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548019070 con 0x559325c1e4d0 2023-07-13T22:16:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.869+0000 7ff5377fe700 1 -- 172.21.15.26:0/858648803 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff548011b70 con 0x559325c1e4d0 2023-07-13T22:16:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.869+0000 7ff5377fe700 1 -- 172.21.15.26:0/858648803 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548012b80 con 0x559325c1e4d0 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.871+0000 7ff566fc6b80 1 -- 172.21.15.26:0/858648803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 msgr2=0x559325c1e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.871+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff548014790 tx=0x7ff548014aa0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.873+0000 7ff566fc6b80 1 -- 172.21.15.26:0/858648803 shutdown_connections 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.873+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.873+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c1e8e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.873+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/858648803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c1f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.873+0000 7ff566fc6b80 1 -- 172.21.15.26:0/858648803 >> 172.21.15.26:0/858648803 conn(0x559325c13da0 msgr2=0x559325c191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.874+0000 7ff566fc6b80 1 -- 172.21.15.26:0/858648803 shutdown_connections 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.874+0000 7ff566fc6b80 1 -- 172.21.15.26:0/858648803 wait complete. 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.875+0000 7ff566fc6b80 1 -- start start 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.875+0000 7ff566fc6b80 1 Processor -- start 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1ee20 0x559325c286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c2f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325b0fac0 con 0x559325c1ee20 2023-07-13T22:16:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325b0f7c0 con 0x559325c1fc70 2023-07-13T22:16:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593259f3920 con 0x559325c1e4d0 2023-07-13T22:16:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.876+0000 7ff566fc6b80 10 client.0 _ll_register_callbacks cb 0x559325b0d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.883+0000 7ff55139b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.883+0000 7ff55139b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40118/0 (socket says 172.21.15.26:40118) 2023-07-13T22:16:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.883+0000 7ff55139b700 1 -- 172.21.15.26:0/3527771706 learned_addr learned my addr 172.21.15.26:0/3527771706 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.883+0000 7ff551b9c700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.888+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1ee20 0x559325c286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.894+0000 7ff55139b700 1 -- 172.21.15.26:0/3527771706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 msgr2=0x559325c2f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.894+0000 7ff55139b700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c2f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.894+0000 7ff55139b700 1 -- 172.21.15.26:0/3527771706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1ee20 msgr2=0x559325c286c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.894+0000 7ff55139b700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1ee20 0x559325c286c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.894+0000 7ff55139b700 1 -- 172.21.15.26:0/3527771706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff548014470 con 0x559325c1e4d0 2023-07-13T22:16:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.906+0000 7ff55139b700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff54800ad60 tx=0x7ff548011f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b1e045e332704a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.906+0000 7ff534ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.906+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548019040 con 0x559325c1e4d0 2023-07-13T22:16:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.906+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff54802e070 con 0x559325c1e4d0 2023-07-13T22:16:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.907+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559325ac94f0 con 0x559325c1e4d0 2023-07-13T22:16:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.912+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff548012450 con 0x559325c1e4d0 2023-07-13T22:16:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.912+0000 7ff566fc6b80 10 client.25810 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.912+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559325986c60 con 0x559325c1e4d0 2023-07-13T22:16:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.919+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559325ac94f0 con 0x559325c1e4d0 2023-07-13T22:16:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.919+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff548005250 con 0x559325c1e4d0 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.919+0000 7ff566fc6b80 10 client.25810 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.920+0000 7ff566fc6b80 10 client.25810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.920+0000 7ff566fc6b80 20 client.25810 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.920+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c33e30 0x559325c36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.920+0000 7ff566fc6b80 4 client.25810 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:38.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.920+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559325b68eb0 con 0x559325c33e30 2023-07-13T22:16:38.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.924+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c33e30 0x559325c36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.925+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c33e30 0x559325c36240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3f265e41e353c927 server_cookie=c768f35658deb5e4 in_seq=0 out_seq=0 2023-07-13T22:16:38.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.925+0000 7ff534ff9700 10 client.25810 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:16:38.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.928+0000 7ff534ff9700 1 -- 172.21.15.26:0/3527771706 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559325b68eb0 con 0x559325c33e30 2023-07-13T22:16:38.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.928+0000 7ff534ff9700 10 client.25810 handle_command_reply: tid=0 2023-07-13T22:16:38.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.928+0000 7ff566fc6b80 1 client.25810 shutdown 2023-07-13T22:16:38.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 msgr2=0x559325c28180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff54800ad60 tx=0x7ff548011f10 comp rx=0 tx=0).stop 2023-07-13T22:16:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 shutdown_connections 2023-07-13T22:16:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c33e30 0x559325c36240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c28180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3527771706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1ee20 0x559325c286c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.931+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 >> 172.21.15.26:0/3527771706 conn(0x559325c13da0 msgr2=0x559325c188b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.932+0000 7ff534ff9700 0 client.25810 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.932+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 shutdown_connections 2023-07-13T22:16:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.932+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3527771706 wait complete. 2023-07-13T22:16:38.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.933+0000 7ff566fc6b80 20 client.25810 trim_cache size 0 max 16384 2023-07-13T22:16:38.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.962+0000 7ff566fc6b80 1 Processor -- start 2023-07-13T22:16:38.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 -- start start 2023-07-13T22:16:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 0x559325c26ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1ee20 0x559325c2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c35e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c27d50 con 0x559325c1e4d0 2023-07-13T22:16:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c339a0 con 0x559325c1fc70 2023-07-13T22:16:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.963+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c29300 con 0x559325c1ee20 2023-07-13T22:16:38.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.964+0000 7ff551b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c35e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.964+0000 7ff551b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c35e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42112/0 (socket says 172.21.15.26:42112) 2023-07-13T22:16:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.964+0000 7ff551b9c700 1 -- 172.21.15.26:0/2739837100 learned_addr learned my addr 172.21.15.26:0/2739837100 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 0x559325c26ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff550b9a700 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1ee20 0x559325c2f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 -- 172.21.15.26:0/2739837100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1ee20 msgr2=0x559325c2f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1ee20 0x559325c2f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 -- 172.21.15.26:0/2739837100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 msgr2=0x559325c35e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c35e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.965+0000 7ff55139b700 1 -- 172.21.15.26:0/2739837100 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593259c4250 con 0x559325c1e4d0 2023-07-13T22:16:38.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.966+0000 7ff55139b700 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 0x559325c26ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff548005610 tx=0x7ff54800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a8607b820d69d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.967+0000 7ff537fff700 1 -- 172.21.15.26:0/2739837100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548019070 con 0x559325c1e4d0 2023-07-13T22:16:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.967+0000 7ff537fff700 1 -- 172.21.15.26:0/2739837100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54800a860 con 0x559325c1e4d0 2023-07-13T22:16:38.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.967+0000 7ff537fff700 1 -- 172.21.15.26:0/2739837100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54802a8f0 con 0x559325c1e4d0 2023-07-13T22:16:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 -- 172.21.15.26:0/2739837100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 msgr2=0x559325c26ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 0x559325c26ee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff548005610 tx=0x7ff54800a430 comp rx=0 tx=0).stop 2023-07-13T22:16:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 -- 172.21.15.26:0/2739837100 shutdown_connections 2023-07-13T22:16:38.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1e4d0 0x559325c26ee0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1fc70 0x559325c35e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/2739837100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1ee20 0x559325c2f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 -- 172.21.15.26:0/2739837100 >> 172.21.15.26:0/2739837100 conn(0x559325c13da0 msgr2=0x559325c2fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.968+0000 7ff566fc6b80 1 -- 172.21.15.26:0/2739837100 shutdown_connections 2023-07-13T22:16:38.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.969+0000 7ff566fc6b80 1 -- 172.21.15.26:0/2739837100 wait complete. 2023-07-13T22:16:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.969+0000 7ff566fc6b80 1 -- start start 2023-07-13T22:16:38.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.970+0000 7ff566fc6b80 1 Processor -- start 2023-07-13T22:16:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c2ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c2f160 con 0x559325c1fc70 2023-07-13T22:16:38.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c2f2d0 con 0x559325c1ee20 2023-07-13T22:16:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559325c2f490 con 0x559325c1e4d0 2023-07-13T22:16:38.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.971+0000 7ff566fc6b80 10 client.0 _ll_register_callbacks cb 0x7ff53c010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff550b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff550b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42120/0 (socket says 172.21.15.26:42120) 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff550b9a700 1 -- 172.21.15.26:0/3522177287 learned_addr learned my addr 172.21.15.26:0/3522177287 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff551b9c700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c2ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 -- 172.21.15.26:0/3522177287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 msgr2=0x559325c2bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 -- 172.21.15.26:0/3522177287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 msgr2=0x559325c2ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c2ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.972+0000 7ff55139b700 1 -- 172.21.15.26:0/3522177287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559325986c60 con 0x559325c1e4d0 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff55139b700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c2b850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff548005950 tx=0x7ff54802a940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f161915eb3ab95 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:38.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff534ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548019040 con 0x559325c1e4d0 2023-07-13T22:16:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff54802e040 con 0x559325c1e4d0 2023-07-13T22:16:38.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.973+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff548011b70 con 0x559325c1e4d0 2023-07-13T22:16:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.975+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff548012c70 con 0x559325c1e4d0 2023-07-13T22:16:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.975+0000 7ff566fc6b80 10 client.25816 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.975+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55932599c2b0 con 0x559325c1e4d0 2023-07-13T22:16:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff548011b70 con 0x559325c1e4d0 2023-07-13T22:16:38.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff54803b950 con 0x559325c1e4d0 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 10 client.25816 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 20 client.25816 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c889c0 0x559325c8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 4 client.25816 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559325b2b050 con 0x559325c889c0 2023-07-13T22:16:38.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.976+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c889c0 0x559325c8ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:38.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.977+0000 7ff550b9a700 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c889c0 0x559325c8ada0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51613c8c4819853 server_cookie=f3e9d687aa6325ff in_seq=0 out_seq=0 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.977+0000 7ff534ff9700 10 client.25816 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.977+0000 7ff534ff9700 1 -- 172.21.15.26:0/3522177287 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559325b2b050 con 0x559325c889c0 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.977+0000 7ff534ff9700 10 client.25816 handle_command_reply: tid=0 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.977+0000 7ff566fc6b80 1 client.25816 shutdown 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 msgr2=0x559325c2b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c2b850 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff548005950 tx=0x7ff54802a940 comp rx=0 tx=0).stop 2023-07-13T22:16:38.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 shutdown_connections 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x559325c889c0 0x559325c8ada0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559325c1fc70 0x559325c2ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff534ff9700 0 client.25816 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559325c1e4d0 0x559325c2b850 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 --2- 172.21.15.26:0/3522177287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559325c1ee20 0x559325c2bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 >> 172.21.15.26:0/3522177287 conn(0x559325c13da0 msgr2=0x559325c35e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.978+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 shutdown_connections 2023-07-13T22:16:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.979+0000 7ff566fc6b80 1 -- 172.21.15.26:0/3522177287 wait complete. 2023-07-13T22:16:38.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.979+0000 7ff566fc6b80 20 client.25816 trim_cache size 0 max 16384 2023-07-13T22:16:38.970 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 msgr2=0x7ff5380a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 0x7ff5380a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff548021000 tx=0x7ff54801a000 comp rx=0 tx=0).stop 2023-07-13T22:16:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 msgr2=0x7ff54c1140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:38.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b500 tx=0x7ff53c00b810 comp rx=0 tx=0).stop 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 shutdown_connections 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff5380a12a0 0x7ff5380a3750 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff548021000 tx=0x7ff54801a000 comp rx=0 tx=0).stop 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff54c10bd60 0x7ff54c1b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff54c070ee0 0x7ff54c113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 --2- 172.21.15.26:0/270129439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff54c071830 0x7ff54c1140f0 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00b500 tx=0x7ff53c00b810 comp rx=0 tx=0).stop 2023-07-13T22:16:38.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.981+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 >> 172.21.15.26:0/270129439 conn(0x7ff54c06ca90 msgr2=0x7ff54c06dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.986+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 shutdown_connections 2023-07-13T22:16:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:38.987+0000 7ff5535ff700 1 -- 172.21.15.26:0/270129439 wait complete. 2023-07-13T22:16:39.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:16:39.130 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:38 smithi026 ceph-mon[95647]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:16:39.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:38 smithi117 ceph-mon[129074]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:16:39.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:38 smithi121 ceph-mon[129187]: pgmap v398: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:16:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.638+0000 7fe696fff700 1 -- 172.21.15.26:0/1551314787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690070ee0 msgr2=0x7fe6900712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.638+0000 7fe696fff700 1 --2- 172.21.15.26:0/1551314787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690070ee0 0x7fe6900712f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe68801c0d0 tx=0x7fe68801c3e0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 -- 172.21.15.26:0/1551314787 shutdown_connections 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 --2- 172.21.15.26:0/1551314787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe69010bd60 0x7fe69010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 --2- 172.21.15.26:0/1551314787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690070ee0 0x7fe6900712f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 --2- 172.21.15.26:0/1551314787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe690071830 0x7fe69010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 -- 172.21.15.26:0/1551314787 >> 172.21.15.26:0/1551314787 conn(0x7fe69006ca90 msgr2=0x7fe69006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 -- 172.21.15.26:0/1551314787 shutdown_connections 2023-07-13T22:16:39.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.639+0000 7fe696fff700 1 -- 172.21.15.26:0/1551314787 wait complete. 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 Processor -- start 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 -- start start 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe690070ee0 0x7fe690117ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690071830 0x7fe690117ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69011ef60 con 0x7fe690070ee0 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69011f0d0 con 0x7fe690071830 2023-07-13T22:16:39.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.641+0000 7fe696fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69011f240 con 0x7fe69010bd60 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.642+0000 7fe69559c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.642+0000 7fe69559c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40154/0 (socket says 172.21.15.26:40154) 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.642+0000 7fe69559c700 1 -- 172.21.15.26:0/2411569403 learned_addr learned my addr 172.21.15.26:0/2411569403 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.643+0000 7fe69559c700 1 -- 172.21.15.26:0/2411569403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690071830 msgr2=0x7fe690117ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.643+0000 7fe69559c700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690071830 0x7fe690117ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.643+0000 7fe69559c700 1 -- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe690070ee0 msgr2=0x7fe690117ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.643+0000 7fe69559c700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe690070ee0 0x7fe690117ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.643+0000 7fe69559c700 1 -- 172.21.15.26:0/2411569403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68801ad10 con 0x7fe69010bd60 2023-07-13T22:16:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.644+0000 7fe69559c700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b550 tx=0x7fe68000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.644+0000 7fe68dffb700 1 -- 172.21.15.26:0/2411569403 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680011830 con 0x7fe69010bd60 2023-07-13T22:16:39.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.644+0000 7fe68dffb700 1 -- 172.21.15.26:0/2411569403 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe680011e50 con 0x7fe69010bd60 2023-07-13T22:16:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.644+0000 7fe68dffb700 1 -- 172.21.15.26:0/2411569403 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680010e40 con 0x7fe69010bd60 2023-07-13T22:16:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.645+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69011f520 con 0x7fe69010bd60 2023-07-13T22:16:39.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.645+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe69011f9e0 con 0x7fe69010bd60 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.661+0000 7fe68dffb700 1 -- 172.21.15.26:0/2411569403 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe680011990 con 0x7fe69010bd60 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.675+0000 7fe6aa8c5b80 1 Processor -- start 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.675+0000 7fe6aa8c5b80 1 -- start start 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c47336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4733c70 0x5654c47340e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4637cb0 con 0x5654c4733c70 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c468aa40 con 0x5654c47324d0 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.676+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4683ed0 con 0x5654c4732e20 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42144/0 (socket says 172.21.15.26:42144) 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 -- 172.21.15.26:0/1529263748 learned_addr learned my addr 172.21.15.26:0/1529263748 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe694d9b700 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c47336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 -- 172.21.15.26:0/1529263748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 msgr2=0x5654c47336e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c47336e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 -- 172.21.15.26:0/1529263748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4733c70 msgr2=0x5654c47340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4733c70 0x5654c47340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 -- 172.21.15.26:0/1529263748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c44b1de0 con 0x5654c47324d0 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.677+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe684001800 tx=0x7fe684001b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8417f61af1df47d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.680+0000 7fe68dffb700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 0x7fe6780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.680+0000 7fe68dffb700 1 -- 172.21.15.26:0/2411569403 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe6800cc3b0 con 0x7fe69010bd60 2023-07-13T22:16:39.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.686+0000 7fe676ffd700 1 -- 172.21.15.26:0/1529263748 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684004c70 con 0x5654c47324d0 2023-07-13T22:16:39.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.686+0000 7fe676ffd700 1 -- 172.21.15.26:0/1529263748 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe684001d50 con 0x5654c47324d0 2023-07-13T22:16:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.686+0000 7fe676ffd700 1 -- 172.21.15.26:0/1529263748 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400e670 con 0x5654c47324d0 2023-07-13T22:16:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.687+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1529263748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 msgr2=0x5654c47328e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.687+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe684001800 tx=0x7fe684001b10 comp rx=0 tx=0).stop 2023-07-13T22:16:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.688+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1529263748 shutdown_connections 2023-07-13T22:16:39.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.688+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4733c70 0x5654c47340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.688+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c47328e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.688+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1529263748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c47336e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.688+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1529263748 >> 172.21.15.26:0/1529263748 conn(0x5654c4727da0 msgr2=0x5654c472d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.699+0000 7fe694d9b700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 0x7fe6780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.699+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1529263748 shutdown_connections 2023-07-13T22:16:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.699+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1529263748 wait complete. 2023-07-13T22:16:39.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.700+0000 7fe6aa8c5b80 1 -- start start 2023-07-13T22:16:39.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.700+0000 7fe6aa8c5b80 1 Processor -- start 2023-07-13T22:16:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c473c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4733c70 0x5654c47437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4623ac0 con 0x5654c4732e20 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c46237c0 con 0x5654c47324d0 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4507920 con 0x5654c4733c70 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.701+0000 7fe6aa8c5b80 10 client.0 _ll_register_callbacks cb 0x5654c4621760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.706+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.706+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43558/0 (socket says 172.21.15.26:43558) 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.706+0000 7fe68ffff700 1 -- 172.21.15.26:0/3258835344 learned_addr learned my addr 172.21.15.26:0/3258835344 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.715+0000 7fe69559c700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4733c70 0x5654c47437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.715+0000 7fe694d9b700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c473c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.718+0000 7fe68ffff700 1 -- 172.21.15.26:0/3258835344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4733c70 msgr2=0x5654c47437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.718+0000 7fe68ffff700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4733c70 0x5654c47437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.718+0000 7fe68ffff700 1 -- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 msgr2=0x5654c473c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.718+0000 7fe68ffff700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c473c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.718+0000 7fe68ffff700 1 -- 172.21.15.26:0/3258835344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840014b0 con 0x5654c4732e20 2023-07-13T22:16:39.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.726+0000 7fe694d9b700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 0x7fe6780a3750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68801c860 tx=0x7fe68801a140 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.733+0000 7fe68ffff700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe684007760 tx=0x7fe684004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8f0b19f20b4b61 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.733+0000 7fe66ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:16:39.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.744+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684019040 con 0x5654c4732e20 2023-07-13T22:16:39.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.744+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe684008c10 con 0x5654c4732e20 2023-07-13T22:16:39.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.745+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654c45dd4f0 con 0x5654c4732e20 2023-07-13T22:16:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.756+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe684008dd0 con 0x5654c4732e20 2023-07-13T22:16:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.756+0000 7fe6aa8c5b80 10 client.15675 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.756+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5654c449ac60 con 0x5654c4732e20 2023-07-13T22:16:39.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.784+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5654c45dd4f0 con 0x5654c4732e20 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.784+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe68401c030 con 0x5654c4732e20 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.785+0000 7fe6aa8c5b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.785+0000 7fe6aa8c5b80 10 client.15675 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.785+0000 7fe6aa8c5b80 20 client.15675 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.785+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c4747d20 0x5654c474a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.786+0000 7fe6aa8c5b80 4 client.15675 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.786+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654c467ceb0 con 0x5654c4747d20 2023-07-13T22:16:39.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.795+0000 7fe694d9b700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c4747d20 0x5654c474a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.811+0000 7fe694d9b700 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c4747d20 0x5654c474a130 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd5979b5a5287217 server_cookie=3ec4c835575603a6 in_seq=0 out_seq=0 2023-07-13T22:16:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.811+0000 7fe66ffff700 10 client.15675 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:16:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.816+0000 7fe66ffff700 1 -- 172.21.15.26:0/3258835344 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5654c467ceb0 con 0x5654c4747d20 2023-07-13T22:16:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.816+0000 7fe66ffff700 10 client.15675 handle_command_reply: tid=0 2023-07-13T22:16:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.816+0000 7fe6aa8c5b80 1 client.15675 shutdown 2023-07-13T22:16:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 msgr2=0x5654c473c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe684007760 tx=0x7fe684004a70 comp rx=0 tx=0).stop 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 shutdown_connections 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c4747d20 0x5654c474a130 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4733c70 0x5654c47437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c47324d0 0x5654c473c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/3258835344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c473c770 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 >> 172.21.15.26:0/3258835344 conn(0x5654c4727da0 msgr2=0x5654c472c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.817+0000 7fe66ffff700 0 client.15675 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:16:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.818+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 shutdown_connections 2023-07-13T22:16:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.818+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/3258835344 wait complete. 2023-07-13T22:16:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.818+0000 7fe6aa8c5b80 20 client.15675 trim_cache size 0 max 16384 2023-07-13T22:16:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 Processor -- start 2023-07-13T22:16:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 -- start start 2023-07-13T22:16:39.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c47324d0 0x5654c473a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c473c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c472e4c0 con 0x5654c4732e20 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c473b6e0 con 0x5654c4733c70 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.852+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c473bda0 con 0x5654c47324d0 2023-07-13T22:16:39.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43562/0 (socket says 172.21.15.26:43562) 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 -- 172.21.15.26:0/1084924711 learned_addr learned my addr 172.21.15.26:0/1084924711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 -- 172.21.15.26:0/1084924711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c47324d0 msgr2=0x5654c473a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c47324d0 0x5654c473a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 -- 172.21.15.26:0/1084924711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 msgr2=0x5654c473c230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c473c230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.854+0000 7fe68ffff700 1 -- 172.21.15.26:0/1084924711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654c44d8250 con 0x5654c4732e20 2023-07-13T22:16:39.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.855+0000 7fe68ffff700 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe684001b50 tx=0x7fe684001780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06aa230efd46541 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.897+0000 7fe6777fe700 1 -- 172.21.15.26:0/1084924711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684019070 con 0x5654c4732e20 2023-07-13T22:16:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.897+0000 7fe6777fe700 1 -- 172.21.15.26:0/1084924711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe684009620 con 0x5654c4732e20 2023-07-13T22:16:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.897+0000 7fe6777fe700 1 -- 172.21.15.26:0/1084924711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684009860 con 0x5654c4732e20 2023-07-13T22:16:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.899+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1084924711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 msgr2=0x5654c4743770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.899+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe684001b50 tx=0x7fe684001780 comp rx=0 tx=0).stop 2023-07-13T22:16:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.906+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1084924711 shutdown_connections 2023-07-13T22:16:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.906+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c473c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.906+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c47324d0 0x5654c473a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.906+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/1084924711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c4732e20 0x5654c4743770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.906+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1084924711 >> 172.21.15.26:0/1084924711 conn(0x5654c4727da0 msgr2=0x5654c473ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.907+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1084924711 shutdown_connections 2023-07-13T22:16:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.907+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/1084924711 wait complete. 2023-07-13T22:16:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.907+0000 7fe6aa8c5b80 1 -- start start 2023-07-13T22:16:39.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.908+0000 7fe6aa8c5b80 1 Processor -- start 2023-07-13T22:16:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c473fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c4740230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4740910 con 0x5654c47324d0 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c47438d0 con 0x5654c4733c70 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654c4743a60 con 0x5654c4732e20 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.909+0000 7fe6aa8c5b80 10 client.0 _ll_register_callbacks cb 0x7fe6840016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.910+0000 7fe694d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.910+0000 7fe694d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43564/0 (socket says 172.21.15.26:43564) 2023-07-13T22:16:39.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.910+0000 7fe694d9b700 1 -- 172.21.15.26:0/2650067418 learned_addr learned my addr 172.21.15.26:0/2650067418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:39.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.915+0000 7fe69559c700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c4740230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.915+0000 7fe68ffff700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c473fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.916+0000 7fe694d9b700 1 -- 172.21.15.26:0/2650067418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 msgr2=0x5654c473fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.916+0000 7fe694d9b700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c473fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.916+0000 7fe694d9b700 1 -- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 msgr2=0x5654c4740230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:39.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.916+0000 7fe694d9b700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c4740230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.916+0000 7fe694d9b700 1 -- 172.21.15.26:0/2650067418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654c449ac60 con 0x5654c47324d0 2023-07-13T22:16:39.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.933+0000 7fe694d9b700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe68801e010 tx=0x7fe68800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce57c83a90d9764 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:39.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.936+0000 7fe66ffff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:16:39.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.948+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68800e070 con 0x5654c47324d0 2023-07-13T22:16:39.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.948+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe688003cd0 con 0x5654c47324d0 2023-07-13T22:16:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.955+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe684001d50 con 0x5654c47324d0 2023-07-13T22:16:39.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.955+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe68801f040 con 0x5654c47324d0 2023-07-13T22:16:39.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.961+0000 7fe6aa8c5b80 10 client.15681 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:39.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.961+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5654c44b02b0 con 0x5654c47324d0 2023-07-13T22:16:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe684001d50 con 0x5654c47324d0 2023-07-13T22:16:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe6880180b0 con 0x5654c47324d0 2023-07-13T22:16:39.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 10 client.15681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 20 client.15681 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c479d380 0x5654c479f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 4 client.15681 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5654c463f050 con 0x5654c479d380 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.962+0000 7fe68ffff700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c479d380 0x5654c479f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:39.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.963+0000 7fe68ffff700 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c479d380 0x5654c479f790 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc1c6f366d23d766 server_cookie=38ec0cda3ed04459 in_seq=0 out_seq=0 2023-07-13T22:16:39.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.963+0000 7fe66ffff700 10 client.15681 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:16:39.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.963+0000 7fe66ffff700 1 -- 172.21.15.26:0/2650067418 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5654c463f050 con 0x5654c479d380 2023-07-13T22:16:39.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.963+0000 7fe66ffff700 10 client.15681 handle_command_reply: tid=0 2023-07-13T22:16:39.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.963+0000 7fe6aa8c5b80 1 client.15681 shutdown 2023-07-13T22:16:39.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.964+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 msgr2=0x5654c4743250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.964+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe68801e010 tx=0x7fe68800e000 comp rx=0 tx=0).stop 2023-07-13T22:16:39.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 shutdown_connections 2023-07-13T22:16:39.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5654c47324d0 0x5654c4743250 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5654c479d380 0x5654c479f790 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5654c4733c70 0x5654c4740230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 --2- 172.21.15.26:0/2650067418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654c4732e20 0x5654c473fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.967+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 >> 172.21.15.26:0/2650067418 conn(0x5654c4727da0 msgr2=0x5654c473a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.968+0000 7fe66ffff700 0 client.15681 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:16:39.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.968+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 shutdown_connections 2023-07-13T22:16:39.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.968+0000 7fe6aa8c5b80 1 -- 172.21.15.26:0/2650067418 wait complete. 2023-07-13T22:16:39.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.969+0000 7fe6aa8c5b80 20 client.15681 trim_cache size 0 max 16384 2023-07-13T22:16:39.958 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:39.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.970+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 msgr2=0x7fe6780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.970+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 0x7fe6780a3750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68801c860 tx=0x7fe68801a140 comp rx=0 tx=0).stop 2023-07-13T22:16:39.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.970+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 msgr2=0x7fe69011e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.970+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b550 tx=0x7fe68000b910 comp rx=0 tx=0).stop 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 shutdown_connections 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe6780a12a0 0x7fe6780a3750 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe690070ee0 0x7fe690117ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69010bd60 0x7fe69011e960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 --2- 172.21.15.26:0/2411569403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe690071830 0x7fe690117ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 >> 172.21.15.26:0/2411569403 conn(0x7fe69006ca90 msgr2=0x7fe69006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 shutdown_connections 2023-07-13T22:16:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:39.971+0000 7fe696fff700 1 -- 172.21.15.26:0/2411569403 wait complete. 2023-07-13T22:16:40.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:16:40.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.905+0000 7f180fbff700 1 -- 172.21.15.26:0/12199770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1808070ee0 msgr2=0x7f18080712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.905+0000 7f180fbff700 1 --2- 172.21.15.26:0/12199770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1808070ee0 0x7f18080712f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f180401a9a0 tx=0x7f180401acb0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.908+0000 7f180fbff700 1 -- 172.21.15.26:0/12199770 shutdown_connections 2023-07-13T22:16:40.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.908+0000 7f180fbff700 1 --2- 172.21.15.26:0/12199770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180810bd60 0x7f180810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.908+0000 7f180fbff700 1 --2- 172.21.15.26:0/12199770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1808070ee0 0x7f18080712f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.908+0000 7f180fbff700 1 --2- 172.21.15.26:0/12199770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808071830 0x7f180810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.908+0000 7f180fbff700 1 -- 172.21.15.26:0/12199770 >> 172.21.15.26:0/12199770 conn(0x7f180806ca90 msgr2=0x7f180806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:40.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.917+0000 7f180fbff700 1 -- 172.21.15.26:0/12199770 shutdown_connections 2023-07-13T22:16:40.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.917+0000 7f180fbff700 1 -- 172.21.15.26:0/12199770 wait complete. 2023-07-13T22:16:40.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.919+0000 7f180fbff700 1 Processor -- start 2023-07-13T22:16:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.919+0000 7f180fbff700 1 -- start start 2023-07-13T22:16:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.919+0000 7f180fbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180fbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1808071830 0x7f18081c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180fbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f180810bd60 0x7f18081c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180fbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18081c5b00 con 0x7f1808071830 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180fbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18081c5c70 con 0x7f1808070ee0 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180fbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18081c5e30 con 0x7f180810bd60 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180d99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180d99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42184/0 (socket says 172.21.15.26:42184) 2023-07-13T22:16:40.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.920+0000 7f180d99b700 1 -- 172.21.15.26:0/1353654843 learned_addr learned my addr 172.21.15.26:0/1353654843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.921+0000 7f180d99b700 1 -- 172.21.15.26:0/1353654843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f180810bd60 msgr2=0x7f18081c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.921+0000 7f180d99b700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f180810bd60 0x7f18081c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.921+0000 7f180d99b700 1 -- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1808071830 msgr2=0x7f18081c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.921+0000 7f180d99b700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1808071830 0x7f18081c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.921+0000 7f180d99b700 1 -- 172.21.15.26:0/1353654843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180401a600 con 0x7f1808070ee0 2023-07-13T22:16:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.922+0000 7f180d99b700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1804003710 tx=0x7f1804003740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.922+0000 7f17feffd700 1 -- 172.21.15.26:0/1353654843 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804004150 con 0x7f1808070ee0 2023-07-13T22:16:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.922+0000 7f17feffd700 1 -- 172.21.15.26:0/1353654843 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18040042b0 con 0x7f1808070ee0 2023-07-13T22:16:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.922+0000 7f17feffd700 1 -- 172.21.15.26:0/1353654843 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180401f810 con 0x7f1808070ee0 2023-07-13T22:16:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.924+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18081c60b0 con 0x7f1808070ee0 2023-07-13T22:16:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.924+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18081c6500 con 0x7f1808070ee0 2023-07-13T22:16:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.933+0000 7f18235aeb80 1 Processor -- start 2023-07-13T22:16:40.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.934+0000 7f17feffd700 1 -- 172.21.15.26:0/1353654843 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f180400fad0 con 0x7f1808070ee0 2023-07-13T22:16:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 -- start start 2023-07-13T22:16:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 0x5600385d70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600384dacb0 con 0x5600385d5e20 2023-07-13T22:16:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003852da40 con 0x5600385d54d0 2023-07-13T22:16:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.935+0000 7f18235aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560038526ed0 con 0x5600385d6c70 2023-07-13T22:16:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.936+0000 7f180e19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.936+0000 7f180e19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42190/0 (socket says 172.21.15.26:42190) 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.936+0000 7f180e19c700 1 -- 172.21.15.26:0/3708770099 learned_addr learned my addr 172.21.15.26:0/3708770099 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.936+0000 7f180d19a700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 0x5600385d70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d99b700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d19a700 1 -- 172.21.15.26:0/3708770099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 msgr2=0x5600385d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d19a700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d19a700 1 -- 172.21.15.26:0/3708770099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 msgr2=0x5600385d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d19a700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f180d19a700 1 -- 172.21.15.26:0/3708770099 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560038354de0 con 0x5600385d6c70 2023-07-13T22:16:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f17feffd700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 0x7f17f40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.937+0000 7f17feffd700 1 -- 172.21.15.26:0/1353654843 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f18040cdde0 con 0x7f1808070ee0 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f180d19a700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 0x5600385d70e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1808072540 tx=0x7f17f8009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8bed73b0f881a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f180e19c700 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f17f3fff700 1 -- 172.21.15.26:0/3708770099 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8011830 con 0x5600385d6c70 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f17f3fff700 1 -- 172.21.15.26:0/3708770099 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17f8011e50 con 0x5600385d6c70 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f17f3fff700 1 -- 172.21.15.26:0/3708770099 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f800b300 con 0x5600385d6c70 2023-07-13T22:16:40.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.938+0000 7f180e19c700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 0x7f17f40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.939+0000 7f18235aeb80 1 -- 172.21.15.26:0/3708770099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 msgr2=0x5600385d70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.939+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 0x5600385d70e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1808072540 tx=0x7f17f8009a60 comp rx=0 tx=0).stop 2023-07-13T22:16:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 -- 172.21.15.26:0/3708770099 shutdown_connections 2023-07-13T22:16:40.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d6c70 0x5600385d70e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3708770099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 -- 172.21.15.26:0/3708770099 >> 172.21.15.26:0/3708770099 conn(0x5600385cada0 msgr2=0x5600385d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f180e19c700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 0x7f17f40a36d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1800009ed0 tx=0x7f18000094f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 -- 172.21.15.26:0/3708770099 shutdown_connections 2023-07-13T22:16:40.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.940+0000 7f18235aeb80 1 -- 172.21.15.26:0/3708770099 wait complete. 2023-07-13T22:16:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.941+0000 7f18235aeb80 1 -- start start 2023-07-13T22:16:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.941+0000 7f18235aeb80 1 Processor -- start 2023-07-13T22:16:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d6c70 0x5600385e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600384c6ac0 con 0x5600385d6c70 2023-07-13T22:16:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600384c67c0 con 0x5600385d54d0 2023-07-13T22:16:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.942+0000 7f18235aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600383aa920 con 0x5600385d5e20 2023-07-13T22:16:40.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.943+0000 7f18235aeb80 10 client.0 _ll_register_callbacks cb 0x5600384c4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.944+0000 7f180d19a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.944+0000 7f180d19a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40236/0 (socket says 172.21.15.26:40236) 2023-07-13T22:16:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.944+0000 7f180d19a700 1 -- 172.21.15.26:0/3344400203 learned_addr learned my addr 172.21.15.26:0/3344400203 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.944+0000 7f180e19c700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d6c70 0x5600385e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d99b700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 -- 172.21.15.26:0/3344400203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 msgr2=0x5600385df140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385df140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 -- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d6c70 msgr2=0x5600385e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d6c70 0x5600385e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 -- 172.21.15.26:0/3344400203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f80093e0 con 0x5600385d5e20 2023-07-13T22:16:40.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.945+0000 7f180d19a700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17f8010bc0 tx=0x7f17f8010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93535d51bedc525e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.946+0000 7f17f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.946+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f8011830 con 0x5600385d5e20 2023-07-13T22:16:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.946+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17f801e070 con 0x5600385d5e20 2023-07-13T22:16:40.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.947+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600384804f0 con 0x5600385d5e20 2023-07-13T22:16:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.948+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17f8011990 con 0x5600385d5e20 2023-07-13T22:16:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.948+0000 7f18235aeb80 10 client.25837 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.948+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56003833dc60 con 0x5600385d5e20 2023-07-13T22:16:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.948+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600384804f0 con 0x5600385d5e20 2023-07-13T22:16:40.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f17f8022420 con 0x5600385d5e20 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 10 client.25837 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 10 client.25837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 20 client.25837 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5600385ead30 0x5600385ed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 4 client.25837 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.949+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56003851feb0 con 0x5600385ead30 2023-07-13T22:16:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.950+0000 7f180d99b700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5600385ead30 0x5600385ed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.951+0000 7f180d99b700 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5600385ead30 0x5600385ed140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=953f8694a63d490f server_cookie=ae56ee17e0f9daf5 in_seq=0 out_seq=0 2023-07-13T22:16:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.951+0000 7f17f17fa700 10 client.25837 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.952+0000 7f17f17fa700 1 -- 172.21.15.26:0/3344400203 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56003851feb0 con 0x5600385ead30 2023-07-13T22:16:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.952+0000 7f17f17fa700 10 client.25837 handle_command_reply: tid=0 2023-07-13T22:16:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.952+0000 7f18235aeb80 1 client.25837 shutdown 2023-07-13T22:16:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 msgr2=0x5600385df680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f17f8010bc0 tx=0x7f17f8010fd0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 shutdown_connections 2023-07-13T22:16:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5600385ead30 0x5600385ed140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d6c70 0x5600385e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d54d0 0x5600385df140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3344400203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385df680 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f17f17fa700 0 client.25837 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 >> 172.21.15.26:0/3344400203 conn(0x5600385cada0 msgr2=0x5600385cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.953+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 shutdown_connections 2023-07-13T22:16:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.954+0000 7f18235aeb80 1 -- 172.21.15.26:0/3344400203 wait complete. 2023-07-13T22:16:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.954+0000 7f18235aeb80 20 client.25837 trim_cache size 0 max 16384 2023-07-13T22:16:40.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.983+0000 7f18235aeb80 1 Processor -- start 2023-07-13T22:16:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 -- start start 2023-07-13T22:16:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d54d0 0x5600385d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:40.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600385dcd70 con 0x5600385d54d0 2023-07-13T22:16:40.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600385d7400 con 0x5600385d6c70 2023-07-13T22:16:40.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.984+0000 7f18235aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600385d1a00 con 0x5600385d5e20 2023-07-13T22:16:40.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.985+0000 7f180d19a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.985+0000 7f180e19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:40.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.985+0000 7f180e19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42208/0 (socket says 172.21.15.26:42208) 2023-07-13T22:16:40.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.985+0000 7f180e19c700 1 -- 172.21.15.26:0/866117948 learned_addr learned my addr 172.21.15.26:0/866117948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:40.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.991+0000 7f180e19c700 1 -- 172.21.15.26:0/866117948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 msgr2=0x5600385e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.991+0000 7f180e19c700 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.991+0000 7f180e19c700 1 -- 172.21.15.26:0/866117948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d54d0 msgr2=0x5600385d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.991+0000 7f180e19c700 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d54d0 0x5600385d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.991+0000 7f180e19c700 1 -- 172.21.15.26:0/866117948 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56003837b250 con 0x5600385d6c70 2023-07-13T22:16:40.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:40.998+0000 7f180e19c700 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18000165b0 tx=0x7f1800014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6a91f83580735a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:40.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.002+0000 7f17fcbff700 1 -- 172.21.15.26:0/866117948 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180001d040 con 0x5600385d6c70 2023-07-13T22:16:40.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.002+0000 7f17fcbff700 1 -- 172.21.15.26:0/866117948 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1800022b00 con 0x5600385d6c70 2023-07-13T22:16:40.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.002+0000 7f17fcbff700 1 -- 172.21.15.26:0/866117948 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800021ae0 con 0x5600385d6c70 2023-07-13T22:16:40.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.003+0000 7f18235aeb80 1 -- 172.21.15.26:0/866117948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 msgr2=0x5600385da010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:40.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.003+0000 7f18235aeb80 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18000165b0 tx=0x7f1800014000 comp rx=0 tx=0).stop 2023-07-13T22:16:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.007+0000 7f18235aeb80 1 -- 172.21.15.26:0/866117948 shutdown_connections 2023-07-13T22:16:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.007+0000 7f18235aeb80 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d54d0 0x5600385d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.007+0000 7f18235aeb80 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385da010 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.007+0000 7f18235aeb80 1 --2- 172.21.15.26:0/866117948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d5e20 0x5600385e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:40.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.007+0000 7f18235aeb80 1 -- 172.21.15.26:0/866117948 >> 172.21.15.26:0/866117948 conn(0x5600385cada0 msgr2=0x5600385f1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.019+0000 7f18235aeb80 1 -- 172.21.15.26:0/866117948 shutdown_connections 2023-07-13T22:16:41.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.019+0000 7f18235aeb80 1 -- 172.21.15.26:0/866117948 wait complete. 2023-07-13T22:16:41.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.019+0000 7f18235aeb80 1 -- start start 2023-07-13T22:16:41.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.020+0000 7f18235aeb80 1 Processor -- start 2023-07-13T22:16:41.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600385e3b70 con 0x5600385d5e20 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003863ebe0 con 0x5600385d6c70 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56003863ed20 con 0x5600385d54d0 2023-07-13T22:16:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f18235aeb80 10 client.0 _ll_register_callbacks cb 0x7f17f8009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.021+0000 7f180d99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40248/0 (socket says 172.21.15.26:40248) 2023-07-13T22:16:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 -- 172.21.15.26:0/3931516704 learned_addr learned my addr 172.21.15.26:0/3931516704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180e19c700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 -- 172.21.15.26:0/3931516704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 msgr2=0x5600385e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 -- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 msgr2=0x5600385e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.022+0000 7f180d99b700 1 -- 172.21.15.26:0/3931516704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1800007e50 con 0x5600385d54d0 2023-07-13T22:16:41.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.023+0000 7f180d99b700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1804006010 tx=0x7f180401af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0ca08731ce39b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.023+0000 7f17f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.023+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804095ea0 con 0x5600385d54d0 2023-07-13T22:16:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.023+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f180409b290 con 0x5600385d54d0 2023-07-13T22:16:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.023+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17f8011e50 con 0x5600385d54d0 2023-07-13T22:16:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.024+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f180409abf0 con 0x5600385d54d0 2023-07-13T22:16:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.024+0000 7f18235aeb80 10 client.25843 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.024+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5600383532b0 con 0x5600385d54d0 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17f8011e50 con 0x5600385d54d0 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f180409a8a0 con 0x5600385d54d0 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 10 client.25843 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 10 client.25843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 20 client.25843 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560038640350 0x560038642760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 4 client.25843 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.027+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5600384e2050 con 0x560038640350 2023-07-13T22:16:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.028+0000 7f180d19a700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560038640350 0x560038642760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.029+0000 7f180d19a700 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560038640350 0x560038642760 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ded9ef49f08ea1 server_cookie=cc98d3e958a1205a in_seq=0 out_seq=0 2023-07-13T22:16:41.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.029+0000 7f17f17fa700 10 client.25843 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.030+0000 7f17f17fa700 1 -- 172.21.15.26:0/3931516704 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5600384e2050 con 0x560038640350 2023-07-13T22:16:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.030+0000 7f17f17fa700 10 client.25843 handle_command_reply: tid=0 2023-07-13T22:16:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.030+0000 7f18235aeb80 1 client.25843 shutdown 2023-07-13T22:16:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 msgr2=0x5600385e64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1804006010 tx=0x7f180401af80 comp rx=0 tx=0).stop 2023-07-13T22:16:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 shutdown_connections 2023-07-13T22:16:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5600385d6c70 0x5600385e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5600385d54d0 0x5600385e64b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560038640350 0x560038642760 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 --2- 172.21.15.26:0/3931516704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600385d5e20 0x5600385e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.031+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 >> 172.21.15.26:0/3931516704 conn(0x5600385cada0 msgr2=0x5600385d9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.032+0000 7f17f17fa700 0 client.25843 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:16:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.032+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 shutdown_connections 2023-07-13T22:16:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.032+0000 7f18235aeb80 1 -- 172.21.15.26:0/3931516704 wait complete. 2023-07-13T22:16:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.033+0000 7f18235aeb80 20 client.25843 trim_cache size 0 max 16384 2023-07-13T22:16:41.022 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.035+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 msgr2=0x7f17f40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.035+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 0x7f17f40a36d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1800009ed0 tx=0x7f18000094f0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.035+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 msgr2=0x7f18081c07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.035+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1804003710 tx=0x7f1804003740 comp rx=0 tx=0).stop 2023-07-13T22:16:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 shutdown_connections 2023-07-13T22:16:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f180810bd60 0x7f18081c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1808070ee0 0x7f18081c07a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17f40a1220 0x7f17f40a36d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 --2- 172.21.15.26:0/1353654843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1808071830 0x7f18081c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.046+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 >> 172.21.15.26:0/1353654843 conn(0x7f180806ca90 msgr2=0x7f180806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.052+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 shutdown_connections 2023-07-13T22:16:41.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.052+0000 7f180fbff700 1 -- 172.21.15.26:0/1353654843 wait complete. 2023-07-13T22:16:41.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:16:41.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:40 smithi117 ceph-mon[129074]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 281 MiB/s wr, 88 op/s 2023-07-13T22:16:41.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:40 smithi026 ceph-mon[95647]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 281 MiB/s wr, 88 op/s 2023-07-13T22:16:41.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:40 smithi121 ceph-mon[129187]: pgmap v399: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 281 MiB/s wr, 88 op/s 2023-07-13T22:16:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 -- 172.21.15.26:0/1696365610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0106000 msgr2=0x7f1fc0106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 --2- 172.21.15.26:0/1696365610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0106000 0x7f1fc0106410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009990 tx=0x7f1fb4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 -- 172.21.15.26:0/1696365610 shutdown_connections 2023-07-13T22:16:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 --2- 172.21.15.26:0/1696365610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc010e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 --2- 172.21.15.26:0/1696365610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0106000 0x7f1fc0106410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 --2- 172.21.15.26:0/1696365610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0107290 0x7f1fc0107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.741+0000 7f1fc579d700 1 -- 172.21.15.26:0/1696365610 >> 172.21.15.26:0/1696365610 conn(0x7f1fc0075090 msgr2=0x7f1fc00774c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.742+0000 7f1fc579d700 1 -- 172.21.15.26:0/1696365610 shutdown_connections 2023-07-13T22:16:41.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.742+0000 7f1fc579d700 1 -- 172.21.15.26:0/1696365610 wait complete. 2023-07-13T22:16:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.743+0000 7f1fc579d700 1 Processor -- start 2023-07-13T22:16:41.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 -- start start 2023-07-13T22:16:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0106000 0x7f1fc0080ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0107290 0x7f1fc007da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01abfa0 con 0x7f1fc0106000 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01ac110 con 0x7f1fc0107c40 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.744+0000 7f1fc579d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01ac2d0 con 0x7f1fc0107290 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.745+0000 7f1fc4f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.745+0000 7f1fc4f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42228/0 (socket says 172.21.15.26:42228) 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.745+0000 7f1fc4f9c700 1 -- 172.21.15.26:0/4279860904 learned_addr learned my addr 172.21.15.26:0/4279860904 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.745+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0107290 0x7f1fc007da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fbffff700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0106000 0x7f1fc0080ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fc4f9c700 1 -- 172.21.15.26:0/4279860904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0107290 msgr2=0x7f1fc007da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fc4f9c700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0107290 0x7f1fc007da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fc4f9c700 1 -- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0106000 msgr2=0x7f1fc0080ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fc4f9c700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0106000 0x7f1fc0080ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.746+0000 7f1fc4f9c700 1 -- 172.21.15.26:0/4279860904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40095f0 con 0x7f1fc0107c40 2023-07-13T22:16:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.747+0000 7f1fc4f9c700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ef30 tx=0x7f1fb800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.748+0000 7f1fbd7fa700 1 -- 172.21.15.26:0/4279860904 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8018070 con 0x7f1fc0107c40 2023-07-13T22:16:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.748+0000 7f1fbd7fa700 1 -- 172.21.15.26:0/4279860904 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb800f040 con 0x7f1fc0107c40 2023-07-13T22:16:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.748+0000 7f1fbd7fa700 1 -- 172.21.15.26:0/4279860904 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb800cd90 con 0x7f1fc0107c40 2023-07-13T22:16:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.748+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc01ac550 con 0x7f1fc0107c40 2023-07-13T22:16:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.749+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc01aca90 con 0x7f1fc0107c40 2023-07-13T22:16:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.757+0000 7f1fda312b80 1 Processor -- start 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 -- start start 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fbd7fa700 1 -- 172.21.15.26:0/4279860904 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1fb8004ad0 con 0x7f1fc0107c40 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad8720 0x5647adad8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9070 0x5647adad98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ad9ddcb0 con 0x5647adad8720 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ada30a40 con 0x5647adad9070 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.761+0000 7f1fda312b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ada29ed0 con 0x5647adad9e30 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.763+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.763+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40284/0 (socket says 172.21.15.26:40284) 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.763+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3430806237 learned_addr learned my addr 172.21.15.26:0/3430806237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbd7fa700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 0x7f1fa80a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3430806237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9070 msgr2=0x5647adad98a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9070 0x5647adad98a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3430806237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad8720 msgr2=0x5647adad8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad8720 0x5647adad8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbd7fa700 1 -- 172.21.15.26:0/4279860904 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1fb80cc3b0 con 0x7f1fc0107c40 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3430806237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ad857de0 con 0x5647adad9e30 2023-07-13T22:16:41.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.764+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b50 tx=0x7f1fb0009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1918151c93c40b21 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.767+0000 7f1fa67fc700 1 -- 172.21.15.26:0/3430806237 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010a30 con 0x5647adad9e30 2023-07-13T22:16:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.767+0000 7f1fa67fc700 1 -- 172.21.15.26:0/3430806237 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb0010b90 con 0x5647adad9e30 2023-07-13T22:16:41.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.767+0000 7f1fa67fc700 1 -- 172.21.15.26:0/3430806237 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0013630 con 0x5647adad9e30 2023-07-13T22:16:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.770+0000 7f1fda312b80 1 -- 172.21.15.26:0/3430806237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 msgr2=0x5647adada2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.770+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b50 tx=0x7f1fb0009e60 comp rx=0 tx=0).stop 2023-07-13T22:16:41.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 -- 172.21.15.26:0/3430806237 shutdown_connections 2023-07-13T22:16:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad8720 0x5647adad8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9e30 0x5647adada2a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3430806237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9070 0x5647adad98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 -- 172.21.15.26:0/3430806237 >> 172.21.15.26:0/3430806237 conn(0x5647adacdff0 msgr2=0x5647adad3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 -- 172.21.15.26:0/3430806237 shutdown_connections 2023-07-13T22:16:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.773+0000 7f1fda312b80 1 -- 172.21.15.26:0/3430806237 wait complete. 2023-07-13T22:16:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.774+0000 7f1fda312b80 1 -- start start 2023-07-13T22:16:41.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.774+0000 7f1fda312b80 1 Processor -- start 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 0x5647adaf33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ad9c9ac0 con 0x5647adad9070 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ad9c97c0 con 0x5647adad9e30 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ad8ad920 con 0x5647adad8720 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.775+0000 7f1fda312b80 10 client.0 _ll_register_callbacks cb 0x5647ad9c7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:41.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.778+0000 7f1fc4f9c700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 0x7f1fa80a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.778+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.778+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:42236/0 (socket says 172.21.15.26:42236) 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.778+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/1686753892 learned_addr learned my addr 172.21.15.26:0/1686753892 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.778+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 0x5647adaf33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbffff700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 msgr2=0x5647adafa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/1686753892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 msgr2=0x5647adaf38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.779+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/1686753892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb0009800 con 0x5647adad8720 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.780+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 0x5647adaf33b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b20 tx=0x7f1fb0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c0b4bbb033df1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.788+0000 7f1fa3ff7700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.788+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0006df0 con 0x5647adad8720 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.788+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1fb0014990 con 0x5647adad8720 2023-07-13T22:16:41.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.788+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647ad9834f0 con 0x5647adad8720 2023-07-13T22:16:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.795+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb001d660 con 0x5647adad8720 2023-07-13T22:16:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.795+0000 7f1fda312b80 10 client.25855 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.795+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5647ad840c60 con 0x5647adad8720 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647ad9834f0 con 0x5647adad8720 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1fb001f860 con 0x5647adad8720 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 10 client.25855 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 20 client.25855 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adafef80 0x5647adb01390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 4 client.25855 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.806+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647ada22eb0 con 0x5647adafef80 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.807+0000 7f1fbffff700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adafef80 0x5647adb01390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.810+0000 7f1fbffff700 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adafef80 0x5647adb01390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b7f89e15690d67 server_cookie=38c3d1e91d05e786 in_seq=0 out_seq=0 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.810+0000 7f1fa3ff7700 10 client.25855 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.811+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/1686753892 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5647ada22eb0 con 0x5647adafef80 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.811+0000 7f1fa3ff7700 10 client.25855 handle_command_reply: tid=0 2023-07-13T22:16:41.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.811+0000 7f1fda312b80 1 client.25855 shutdown 2023-07-13T22:16:41.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 msgr2=0x5647adaf33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 0x5647adaf33b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b20 tx=0x7f1fb0004860 comp rx=0 tx=0).stop 2023-07-13T22:16:41.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 shutdown_connections 2023-07-13T22:16:41.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adafef80 0x5647adb01390 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad8720 0x5647adaf33b0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b20 tx=0x7f1fb0004860 comp rx=0 tx=0).stop 2023-07-13T22:16:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 --2- 172.21.15.26:0/1686753892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.814+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 >> 172.21.15.26:0/1686753892 conn(0x5647adacdff0 msgr2=0x5647adad2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.822+0000 7f1fa3ff7700 0 client.25855 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.828+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 shutdown_connections 2023-07-13T22:16:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.839+0000 7f1fda312b80 1 -- 172.21.15.26:0/1686753892 wait complete. 2023-07-13T22:16:41.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.841+0000 7f1fc4f9c700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 0x7f1fa80a38e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009c10 tx=0x7f1fb8004e00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.852+0000 7f1fda312b80 20 client.25855 trim_cache size 0 max 16384 2023-07-13T22:16:41.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.888+0000 7f1fda312b80 1 Processor -- start 2023-07-13T22:16:41.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.891+0000 7f1fda312b80 1 -- start start 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adb014d0 0x5647adae01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ada22eb0 con 0x5647adb014d0 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647ad9834f0 con 0x5647adad9e30 2023-07-13T22:16:41.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fda312b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647adacde00 con 0x5647adad9070 2023-07-13T22:16:41.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40308/0 (socket says 172.21.15.26:40308) 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.892+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3638495389 learned_addr learned my addr 172.21.15.26:0/3638495389 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3638495389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 msgr2=0x5647adafa8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3638495389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adb014d0 msgr2=0x5647adae01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adb014d0 0x5647adae01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3638495389 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647ad87e250 con 0x5647adad9070 2023-07-13T22:16:41.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.893+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b20 tx=0x7f1fb00101e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8053865bd60f760 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.906+0000 7f1fa6ffd700 1 -- 172.21.15.26:0/3638495389 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0009420 con 0x5647adad9070 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.906+0000 7f1fa6ffd700 1 -- 172.21.15.26:0/3638495389 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb001fe90 con 0x5647adad9070 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.906+0000 7f1fa6ffd700 1 -- 172.21.15.26:0/3638495389 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0012590 con 0x5647adad9070 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 -- 172.21.15.26:0/3638495389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 msgr2=0x5647adb00f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b20 tx=0x7f1fb00101e0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 -- 172.21.15.26:0/3638495389 shutdown_connections 2023-07-13T22:16:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adb014d0 0x5647adae01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adad9070 0x5647adb00f90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3638495389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adafa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.912+0000 7f1fda312b80 1 -- 172.21.15.26:0/3638495389 >> 172.21.15.26:0/3638495389 conn(0x5647adacdff0 msgr2=0x5647adaf20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.913+0000 7f1fda312b80 1 -- 172.21.15.26:0/3638495389 shutdown_connections 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.913+0000 7f1fda312b80 1 -- 172.21.15.26:0/3638495389 wait complete. 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.913+0000 7f1fda312b80 1 -- start start 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.914+0000 7f1fda312b80 1 Processor -- start 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adaf6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adb014d0 0x5647adaf9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647adafa190 con 0x5647adad9070 2023-07-13T22:16:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647adafa300 con 0x5647adad9e30 2023-07-13T22:16:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647adafa4c0 con 0x5647adb014d0 2023-07-13T22:16:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.915+0000 7f1fda312b80 10 client.0 _ll_register_callbacks cb 0x7f1fb0014780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.917+0000 7f1fbffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adaf6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.917+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.917+0000 7f1fbf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43668/0 (socket says 172.21.15.26:43668) 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.917+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3100261562 learned_addr learned my addr 172.21.15.26:0/3100261562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.917+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adb014d0 0x5647adaf9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.918+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3100261562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adb014d0 msgr2=0x5647adaf9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.918+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adb014d0 0x5647adaf9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.918+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 msgr2=0x5647adaf6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.918+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adaf6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.918+0000 7f1fbf7fe700 1 -- 172.21.15.26:0/3100261562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647ad840c60 con 0x5647adad9070 2023-07-13T22:16:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.919+0000 7f1fbf7fe700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b50 tx=0x7f1fb0010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d995687c59b5dcc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.919+0000 7f1fa3ff7700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:16:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.924+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0006d90 con 0x5647adad9070 2023-07-13T22:16:41.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.925+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1fb0000e40 con 0x5647adad9070 2023-07-13T22:16:41.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.926+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fb0010b90 con 0x5647adad9070 2023-07-13T22:16:41.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.927+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb0012540 con 0x5647adad9070 2023-07-13T22:16:41.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.927+0000 7f1fda312b80 10 client.15696 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:41.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.927+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5647ad8562b0 con 0x5647adad9070 2023-07-13T22:16:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.927+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb0010b90 con 0x5647adad9070 2023-07-13T22:16:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.927+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1fb001d7c0 con 0x5647adad9070 2023-07-13T22:16:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 20 client.15696 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adb53a30 0x5647adb55e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 4 client.15696 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:16:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.928+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5647ad9e5050 con 0x5647adb53a30 2023-07-13T22:16:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.930+0000 7f1fbffff700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adb53a30 0x5647adb55e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.930+0000 7f1fbffff700 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adb53a30 0x5647adb55e40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3c0e4eb8af52f9 server_cookie=a49fce6cdcd5e6ea in_seq=0 out_seq=0 2023-07-13T22:16:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.930+0000 7f1fa3ff7700 10 client.15696 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fa3ff7700 1 -- 172.21.15.26:0/3100261562 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5647ad9e5050 con 0x5647adb53a30 2023-07-13T22:16:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fa3ff7700 10 client.15696 handle_command_reply: tid=0 2023-07-13T22:16:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 client.15696 shutdown 2023-07-13T22:16:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 msgr2=0x5647adaf66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009b50 tx=0x7f1fb0010af0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 shutdown_connections 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5647adad9070 0x5647adaf66f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5647adb53a30 0x5647adb55e40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647adb014d0 0x5647adaf9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 --2- 172.21.15.26:0/3100261562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5647adad9e30 0x5647adaf6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 >> 172.21.15.26:0/3100261562 conn(0x5647adacdff0 msgr2=0x5647adae01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.931+0000 7f1fa3ff7700 0 client.15696 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.932+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 shutdown_connections 2023-07-13T22:16:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.932+0000 7f1fda312b80 1 -- 172.21.15.26:0/3100261562 wait complete. 2023-07-13T22:16:41.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.932+0000 7f1fda312b80 20 client.15696 trim_cache size 0 max 16384 2023-07-13T22:16:41.922 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 msgr2=0x7f1fa80a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 0x7f1fa80a38e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009c10 tx=0x7f1fb8004e00 comp rx=0 tx=0).stop 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 msgr2=0x7f1fc007e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800ef30 tx=0x7f1fb800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 shutdown_connections 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1fa80a1430 0x7f1fa80a38e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fc0106000 0x7f1fc0080ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1fc0107c40 0x7f1fc007e0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 --2- 172.21.15.26:0/4279860904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fc0107290 0x7f1fc007da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.933+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 >> 172.21.15.26:0/4279860904 conn(0x7f1fc0075090 msgr2=0x7f1fc0077170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.934+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 shutdown_connections 2023-07-13T22:16:41.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:41.934+0000 7f1fc579d700 1 -- 172.21.15.26:0/4279860904 wait complete. 2023-07-13T22:16:42.959 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:16:42.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:16:43.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:42 smithi117 ceph-mon[129074]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 253 MiB/s wr, 79 op/s 2023-07-13T22:16:43.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:42 smithi026 ceph-mon[95647]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 253 MiB/s wr, 79 op/s 2023-07-13T22:16:43.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:42 smithi121 ceph-mon[129187]: pgmap v400: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 253 MiB/s wr, 79 op/s 2023-07-13T22:16:43.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2460474335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 msgr2=0x7f1ae40fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2460474335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae40fedf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009990 tx=0x7f1ad4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2460474335 shutdown_connections 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2460474335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae4106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2460474335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae40fedf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2460474335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2460474335 >> 172.21.15.26:0/2460474335 conn(0x7f1ae40fa000 msgr2=0x7f1ae40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2460474335 shutdown_connections 2023-07-13T22:16:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.378+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2460474335 wait complete. 2023-07-13T22:16:43.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.380+0000 7f1aeb9ff700 1 Processor -- start 2023-07-13T22:16:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.380+0000 7f1aeb9ff700 1 -- start start 2023-07-13T22:16:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae4102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae41014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae4101c90 con 0x7f1ae4100560 2023-07-13T22:16:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae4101e00 con 0x7f1ae40fe9e0 2023-07-13T22:16:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1aeb9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae419f8f0 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1ae979b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae4102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae41014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1ae8f9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae41014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34224/0 (socket says 172.21.15.26:34224) 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.381+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/2555474681 learned_addr learned my addr 172.21.15.26:0/2555474681 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/2555474681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 msgr2=0x7f1ae4102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae4102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 msgr2=0x7f1ae41014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae41014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/2555474681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40095f0 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.382+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000e9f0 tx=0x7f1ae000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.383+0000 7f1ada7fc700 1 -- 172.21.15.26:0/2555474681 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000c9c0 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.383+0000 7f1ada7fc700 1 -- 172.21.15.26:0/2555474681 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ae0010430 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.383+0000 7f1ada7fc700 1 -- 172.21.15.26:0/2555474681 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0018680 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.383+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae419fbd0 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.383+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae41a0090 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.388+0000 7f1aff36cb80 1 Processor -- start 2023-07-13T22:16:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 -- start start 2023-07-13T22:16:43.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1ada7fc700 1 -- 172.21.15.26:0/2555474681 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ae0010860 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276ab500 0x5612276ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612275a6d30 con 0x5612276a9d60 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612275f9ac0 con 0x5612276aa6b0 2023-07-13T22:16:43.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.389+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612275f2f50 con 0x5612276ab500 2023-07-13T22:16:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60464/0 (socket says 172.21.15.26:60464) 2023-07-13T22:16:43.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 -- 172.21.15.26:0/2939127860 learned_addr learned my addr 172.21.15.26:0/2939127860 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 -- 172.21.15.26:0/2939127860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276ab500 msgr2=0x5612276ab970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276ab500 0x5612276ab970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 -- 172.21.15.26:0/2939127860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 msgr2=0x5612276aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.390+0000 7f1ae979b700 1 -- 172.21.15.26:0/2939127860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561227420e60 con 0x5612276aa6b0 2023-07-13T22:16:43.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.391+0000 7f1ae979b700 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40ffa30 tx=0x7f1ad4005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55159e3b28394bed server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.391+0000 7f1acf7fe700 1 -- 172.21.15.26:0/2939127860 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad401d070 con 0x5612276aa6b0 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.392+0000 7f1acf7fe700 1 -- 172.21.15.26:0/2939127860 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad401e040 con 0x5612276aa6b0 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.392+0000 7f1acf7fe700 1 -- 172.21.15.26:0/2939127860 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4004920 con 0x5612276aa6b0 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.392+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2939127860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 msgr2=0x5612276aaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.392+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40ffa30 tx=0x7f1ad4005ea0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2939127860 shutdown_connections 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276ab500 0x5612276ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2939127860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276aa6b0 0x5612276aaf70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2939127860 >> 172.21.15.26:0/2939127860 conn(0x561227696e20 msgr2=0x56122769c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1ada7fc700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 0x7f1ad00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1ada7fc700 1 -- 172.21.15.26:0/2555474681 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ae0014070 con 0x7f1ae40ffbd0 2023-07-13T22:16:43.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 0x7f1ad00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.393+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2939127860 shutdown_connections 2023-07-13T22:16:43.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.394+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2939127860 wait complete. 2023-07-13T22:16:43.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.394+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 0x7f1ad00a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009fd0 tx=0x7f1adc00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.395+0000 7f1aff36cb80 1 -- start start 2023-07-13T22:16:43.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.395+0000 7f1aff36cb80 1 Processor -- start 2023-07-13T22:16:43.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561227592b40 con 0x5612276aa6b0 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561227592840 con 0x5612276ab500 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612274769a0 con 0x5612276a9d60 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1aff36cb80 10 client.0 _ll_register_callbacks cb 0x5612275907e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60478/0 (socket says 172.21.15.26:60478) 2023-07-13T22:16:43.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.396+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/2874323662 learned_addr learned my addr 172.21.15.26:0/2874323662 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/2874323662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 msgr2=0x5612276b3a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276b3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/2874323662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 msgr2=0x5612276b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/2874323662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae4107750 con 0x5612276ab500 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1adc008ac0 tx=0x7f1adc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2971c243314b1e2d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1accff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.397+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00e040 con 0x5612276ab500 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1adc016910 con 0x5612276ab500 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56122754c570 con 0x5612276ab500 2023-07-13T22:16:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1adc016ad0 con 0x5612276ab500 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1aff36cb80 10 client.25871 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561227409ce0 con 0x5612276ab500 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56122754c570 con 0x5612276ab500 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1adc02b3b0 con 0x5612276ab500 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.398+0000 7f1aff36cb80 10 client.25871 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1aff36cb80 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1aff36cb80 20 client.25871 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276d0850 0x5612276d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1aff36cb80 4 client.25871 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612275ebf30 con 0x5612276d0850 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.399+0000 7f1ae979b700 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276d0850 0x5612276d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.400+0000 7f1ae979b700 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276d0850 0x5612276d2c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3896c2b98163c03 server_cookie=a8f928f3d7fd2b66 in_seq=0 out_seq=0 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.400+0000 7f1accff9700 10 client.25871 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:43.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.400+0000 7f1accff9700 1 -- 172.21.15.26:0/2874323662 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612275ebf30 con 0x5612276d0850 2023-07-13T22:16:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.400+0000 7f1accff9700 10 client.25871 handle_command_reply: tid=0 2023-07-13T22:16:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.400+0000 7f1aff36cb80 1 client.25871 shutdown 2023-07-13T22:16:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 msgr2=0x5612276baf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1adc008ac0 tx=0x7f1adc004b40 comp rx=0 tx=0).stop 2023-07-13T22:16:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 shutdown_connections 2023-07-13T22:16:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276d0850 0x5612276d2c60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276baf90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1accff9700 0 client.25871 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/2874323662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 >> 172.21.15.26:0/2874323662 conn(0x561227696e20 msgr2=0x56122769b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 shutdown_connections 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.401+0000 7f1aff36cb80 1 -- 172.21.15.26:0/2874323662 wait complete. 2023-07-13T22:16:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.402+0000 7f1aff36cb80 20 client.25871 trim_cache size 0 max 16384 2023-07-13T22:16:43.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 Processor -- start 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 -- start start 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276b21a0 con 0x5612276aa6b0 2023-07-13T22:16:43.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276969f0 con 0x5612276ab500 2023-07-13T22:16:43.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.420+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276aad50 con 0x5612276a9d60 2023-07-13T22:16:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.421+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.421+0000 7f1ae9f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60486/0 (socket says 172.21.15.26:60486) 2023-07-13T22:16:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.421+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/1646336481 learned_addr learned my addr 172.21.15.26:0/1646336481 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:43.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.421+0000 7f1ae979b700 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.421+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/1646336481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 msgr2=0x5612276ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/1646336481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 msgr2=0x5612276baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 -- 172.21.15.26:0/1646336481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612274472d0 con 0x5612276ab500 2023-07-13T22:16:43.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e5a0 tx=0x7f1adc00e650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748d5c31870bba9b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.422+0000 7f1acffff700 1 -- 172.21.15.26:0/1646336481 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc02ba00 con 0x5612276ab500 2023-07-13T22:16:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.423+0000 7f1acffff700 1 -- 172.21.15.26:0/1646336481 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1adc00e030 con 0x5612276ab500 2023-07-13T22:16:43.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.423+0000 7f1acffff700 1 -- 172.21.15.26:0/1646336481 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc016ce0 con 0x5612276ab500 2023-07-13T22:16:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1646336481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 msgr2=0x5612276ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e5a0 tx=0x7f1adc00e650 comp rx=0 tx=0).stop 2023-07-13T22:16:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1646336481 shutdown_connections 2023-07-13T22:16:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276ae980 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276a9d60 0x5612276ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1646336481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276aa6b0 0x5612276baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1646336481 >> 172.21.15.26:0/1646336481 conn(0x561227696e20 msgr2=0x5612276cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1646336481 shutdown_connections 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.424+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1646336481 wait complete. 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.425+0000 7f1aff36cb80 1 -- start start 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.425+0000 7f1aff36cb80 1 Processor -- start 2023-07-13T22:16:43.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276a07b0 con 0x5612276a9d60 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276a0920 con 0x5612276ab500 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612276a0ae0 con 0x5612276aa6b0 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1aff36cb80 10 client.0 _ll_register_callbacks cb 0x7f1adc009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1ae8f9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1ae8f9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54450/0 (socket says 172.21.15.26:54450) 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.426+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/1367171415 learned_addr learned my addr 172.21.15.26:0/1367171415 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae979b700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae9f9c700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 msgr2=0x5612276a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/1367171415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 msgr2=0x5612276b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:43.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.427+0000 7f1ae8f9a700 1 -- 172.21.15.26:0/1367171415 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561227409ce0 con 0x5612276aa6b0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1ae8f9a700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0099e10 tx=0x7f1ae009a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60d892bdd1b30edc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1accff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae009deb0 con 0x5612276aa6b0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ae00a22d0 con 0x5612276aa6b0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ad401e040 con 0x5612276aa6b0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae00a1e40 con 0x5612276aa6b0 2023-07-13T22:16:43.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1aff36cb80 10 client.25870 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:43.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.428+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56122741f330 con 0x5612276aa6b0 2023-07-13T22:16:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ad401e040 con 0x5612276aa6b0 2023-07-13T22:16:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ae0095c70 con 0x5612276aa6b0 2023-07-13T22:16:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 10 client.25870 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:43.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 20 client.25870 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276a2570 0x5612276a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 4 client.25870 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.429+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5612275ae0d0 con 0x5612276a2570 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.430+0000 7f1ae979b700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276a2570 0x5612276a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.430+0000 7f1ae979b700 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276a2570 0x5612276a4980 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0190e3af98ed50 server_cookie=7b53dae88f440774 in_seq=0 out_seq=0 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.430+0000 7f1accff9700 10 client.25870 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.432+0000 7f1accff9700 1 -- 172.21.15.26:0/1367171415 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612275ae0d0 con 0x5612276a2570 2023-07-13T22:16:43.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.432+0000 7f1accff9700 10 client.25870 handle_command_reply: tid=0 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.432+0000 7f1aff36cb80 1 client.25870 shutdown 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.432+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 msgr2=0x5612276b64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.432+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0099e10 tx=0x7f1ae009a3c0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 shutdown_connections 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612276a9d60 0x5612276b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5612276a2570 0x5612276a4980 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612276ab500 0x5612276a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 --2- 172.21.15.26:0/1367171415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612276aa6b0 0x5612276b64a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 >> 172.21.15.26:0/1367171415 conn(0x561227696e20 msgr2=0x5612276ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1accff9700 0 client.25870 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:43.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 shutdown_connections 2023-07-13T22:16:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.433+0000 7f1aff36cb80 1 -- 172.21.15.26:0/1367171415 wait complete. 2023-07-13T22:16:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.434+0000 7f1aff36cb80 20 client.25870 trim_cache size 0 max 16384 2023-07-13T22:16:43.425 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:16:43.425 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:16:43.425 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "a8026cd6-9ece-44fb-9b0e-819810d70cd3", 2023-07-13T22:16:43.426 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:16:43.426 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:16:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 msgr2=0x7f1ad00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 0x7f1ad00a3680 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009fd0 tx=0x7f1adc00b040 comp rx=0 tx=0).stop 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 msgr2=0x7f1ae4100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100fb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000e9f0 tx=0x7f1ae000ed00 comp rx=0 tx=0).stop 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 shutdown_connections 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ad00a11d0 0x7f1ad00a3680 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ae4100560 0x7f1ae41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ae40fe9e0 0x7f1ae4102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 --2- 172.21.15.26:0/2555474681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ae40ffbd0 0x7f1ae4100fb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 >> 172.21.15.26:0/2555474681 conn(0x7f1ae40fa000 msgr2=0x7f1ae40fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 shutdown_connections 2023-07-13T22:16:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:43.435+0000 7f1aeb9ff700 1 -- 172.21.15.26:0/2555474681 wait complete. 2023-07-13T22:16:43.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:16:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.118+0000 7fbc8adff700 1 -- 172.21.15.26:0/2091322128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc84071830 msgr2=0x7fbc8410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.118+0000 7fbc8adff700 1 --2- 172.21.15.26:0/2091322128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc84071830 0x7fbc8410b7d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009990 tx=0x7fbc74009ca0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 -- 172.21.15.26:0/2091322128 shutdown_connections 2023-07-13T22:16:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 --2- 172.21.15.26:0/2091322128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8410bd60 0x7fbc8410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 --2- 172.21.15.26:0/2091322128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84070ee0 0x7fbc840712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 --2- 172.21.15.26:0/2091322128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc84071830 0x7fbc8410b7d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 -- 172.21.15.26:0/2091322128 >> 172.21.15.26:0/2091322128 conn(0x7fbc8406ca90 msgr2=0x7fbc8406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.119+0000 7fbc8adff700 1 -- 172.21.15.26:0/2091322128 shutdown_connections 2023-07-13T22:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.120+0000 7fbc8adff700 1 -- 172.21.15.26:0/2091322128 wait complete. 2023-07-13T22:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.121+0000 7fbc8adff700 1 Processor -- start 2023-07-13T22:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 -- start start 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc84070ee0 0x7fbc84113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84071830 0x7fbc84113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84117050 con 0x7fbc84070ee0 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841171c0 con 0x7fbc8410bd60 2023-07-13T22:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.122+0000 7fbc8adff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841b4e50 con 0x7fbc84071830 2023-07-13T22:16:44.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.125+0000 7fbc8939c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.125+0000 7fbc8939c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60512/0 (socket says 172.21.15.26:60512) 2023-07-13T22:16:44.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.125+0000 7fbc8939c700 1 -- 172.21.15.26:0/1024125347 learned_addr learned my addr 172.21.15.26:0/1024125347 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:44.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.126+0000 7fbc83fff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84071830 0x7fbc84113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc84070ee0 0x7fbc84113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 -- 172.21.15.26:0/1024125347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84071830 msgr2=0x7fbc84113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84071830 0x7fbc84113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 -- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc84070ee0 msgr2=0x7fbc84113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc84070ee0 0x7fbc84113a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 -- 172.21.15.26:0/1024125347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740095f0 con 0x7fbc8410bd60 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.127+0000 7fbc8939c700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00b970 tx=0x7fbc7c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.128+0000 7fbc81ffb700 1 -- 172.21.15.26:0/1024125347 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00d8e0 con 0x7fbc8410bd60 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.128+0000 7fbc81ffb700 1 -- 172.21.15.26:0/1024125347 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7c015420 con 0x7fbc8410bd60 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.128+0000 7fbc81ffb700 1 -- 172.21.15.26:0/1024125347 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00c660 con 0x7fbc8410bd60 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.128+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc841b4ff0 con 0x7fbc8410bd60 2023-07-13T22:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.128+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc841b5480 con 0x7fbc8410bd60 2023-07-13T22:16:44.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.136+0000 7fbc9e7aeb80 1 Processor -- start 2023-07-13T22:16:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 -- start start 2023-07-13T22:16:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6dc70 0x55d3aae6e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aad71cb0 con 0x55d3aae6c4d0 2023-07-13T22:16:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aadc4a40 con 0x55d3aae6ce20 2023-07-13T22:16:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.137+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aadbded0 con 0x55d3aae6dc70 2023-07-13T22:16:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60520/0 (socket says 172.21.15.26:60520) 2023-07-13T22:16:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 -- 172.21.15.26:0/2722097954 learned_addr learned my addr 172.21.15.26:0/2722097954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 -- 172.21.15.26:0/2722097954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6dc70 msgr2=0x55d3aae6e0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6dc70 0x55d3aae6e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 -- 172.21.15.26:0/2722097954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 msgr2=0x55d3aae6c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:44.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.139+0000 7fbc88b9b700 1 -- 172.21.15.26:0/2722097954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aabebde0 con 0x55d3aae6ce20 2023-07-13T22:16:44.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.140+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d80 tx=0x7fbc78011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c3491ab8dc4ac8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.140+0000 7fbc6affd700 1 -- 172.21.15.26:0/2722097954 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78004230 con 0x55d3aae6ce20 2023-07-13T22:16:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.141+0000 7fbc6affd700 1 -- 172.21.15.26:0/2722097954 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc78004390 con 0x55d3aae6ce20 2023-07-13T22:16:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.141+0000 7fbc6affd700 1 -- 172.21.15.26:0/2722097954 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780197c0 con 0x55d3aae6ce20 2023-07-13T22:16:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.142+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/2722097954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 msgr2=0x55d3aae6d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.142+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d80 tx=0x7fbc78011720 comp rx=0 tx=0).stop 2023-07-13T22:16:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc81ffb700 1 -- 172.21.15.26:0/1024125347 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc7c00da40 con 0x7fbc8410bd60 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/2722097954 shutdown_connections 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae6c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6dc70 0x55d3aae6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/2722097954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae6d6e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/2722097954 >> 172.21.15.26:0/2722097954 conn(0x55d3aae61da0 msgr2=0x55d3aae671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.144+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/2722097954 shutdown_connections 2023-07-13T22:16:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.147+0000 7fbc81ffb700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 0x7fbc6c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.147+0000 7fbc81ffb700 1 -- 172.21.15.26:0/1024125347 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc7c011070 con 0x7fbc8410bd60 2023-07-13T22:16:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.147+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/2722097954 wait complete. 2023-07-13T22:16:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.148+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 0x7fbc6c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.148+0000 7fbc9e7aeb80 1 -- start start 2023-07-13T22:16:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.148+0000 7fbc9e7aeb80 1 Processor -- start 2023-07-13T22:16:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6c4d0 0x55d3aae76200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aad5dac0 con 0x55d3aae6dc70 2023-07-13T22:16:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aad5d7c0 con 0x55d3aae6c4d0 2023-07-13T22:16:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.149+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aac41920 con 0x55d3aae6ce20 2023-07-13T22:16:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.150+0000 7fbc9e7aeb80 10 client.0 _ll_register_callbacks cb 0x55d3aad5b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.151+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.151+0000 7fbc83fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae76740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54484/0 (socket says 172.21.15.26:54484) 2023-07-13T22:16:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.151+0000 7fbc83fff700 1 -- 172.21.15.26:0/669574185 learned_addr learned my addr 172.21.15.26:0/669574185 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.151+0000 7fbc8939c700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.152+0000 7fbc8939c700 1 -- 172.21.15.26:0/669574185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 msgr2=0x55d3aae76740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.152+0000 7fbc8939c700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae76740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.152+0000 7fbc8939c700 1 -- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6c4d0 msgr2=0x55d3aae76200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:16:44.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.152+0000 7fbc8939c700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6c4d0 0x55d3aae76200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.152+0000 7fbc8939c700 1 -- 172.21.15.26:0/669574185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc780099e0 con 0x55d3aae6dc70 2023-07-13T22:16:44.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.153+0000 7fbc8939c700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0c9ce0 tx=0x7fbc7c0996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18ae7c0e6c28ab0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.159+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 0x7fbc6c0a3690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d80 tx=0x7fbc78009ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.160+0000 7fbc63fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:16:44.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.163+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c09d350 con 0x55d3aae6dc70 2023-07-13T22:16:44.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.163+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc7c006e40 con 0x55d3aae6dc70 2023-07-13T22:16:44.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.163+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3aad174f0 con 0x55d3aae6dc70 2023-07-13T22:16:44.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.169+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc7c0a19f0 con 0x55d3aae6dc70 2023-07-13T22:16:44.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.169+0000 7fbc9e7aeb80 10 client.15717 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:44.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.169+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d3aabd4c60 con 0x55d3aae6dc70 2023-07-13T22:16:44.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.171+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3aad174f0 con 0x55d3aae6dc70 2023-07-13T22:16:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.171+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbc7c0a1560 con 0x55d3aae6dc70 2023-07-13T22:16:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:44.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 20 client.15717 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aae82010 0x55d3aae84420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 4 client.15717 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:16:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.173+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3aadb6eb0 con 0x55d3aae82010 2023-07-13T22:16:44.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.174+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aae82010 0x55d3aae84420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.175+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aae82010 0x55d3aae84420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac1a3b7b2c18cd2 server_cookie=9c5751cbe01b74e1 in_seq=0 out_seq=0 2023-07-13T22:16:44.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.175+0000 7fbc63fff700 10 client.15717 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.176+0000 7fbc63fff700 1 -- 172.21.15.26:0/669574185 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3aadb6eb0 con 0x55d3aae82010 2023-07-13T22:16:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.176+0000 7fbc63fff700 10 client.15717 handle_command_reply: tid=0 2023-07-13T22:16:44.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.176+0000 7fbc9e7aeb80 1 client.15717 shutdown 2023-07-13T22:16:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 msgr2=0x55d3aae7d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0c9ce0 tx=0x7fbc7c0996e0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 shutdown_connections 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7d780 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aae82010 0x55d3aae84420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6c4d0 0x55d3aae76200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/669574185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 >> 172.21.15.26:0/669574185 conn(0x55d3aae61da0 msgr2=0x55d3aae668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.177+0000 7fbc63fff700 0 client.15717 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.178+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 shutdown_connections 2023-07-13T22:16:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.178+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/669574185 wait complete. 2023-07-13T22:16:44.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.178+0000 7fbc9e7aeb80 20 client.15717 trim_cache size 0 max 16384 2023-07-13T22:16:44.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:16:44.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:16:44.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:16:44.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:16:44.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:16:44.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:16:44.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.217+0000 7fbc9e7aeb80 1 Processor -- start 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 -- start start 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae7d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae66fc0 con 0x55d3aae6c4d0 2023-07-13T22:16:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae61970 con 0x55d3aae6dc70 2023-07-13T22:16:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.218+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae73dd0 con 0x55d3aae6ce20 2023-07-13T22:16:44.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60522/0 (socket says 172.21.15.26:60522) 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 -- 172.21.15.26:0/979943938 learned_addr learned my addr 172.21.15.26:0/979943938 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 -- 172.21.15.26:0/979943938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 msgr2=0x55d3aae7d740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae7d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 -- 172.21.15.26:0/979943938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 msgr2=0x55d3aae758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.219+0000 7fbc8939c700 1 -- 172.21.15.26:0/979943938 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aae63600 con 0x55d3aae6dc70 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.220+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.220+0000 7fbc8939c700 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c096010 tx=0x7fbc7c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176e6e23c8489c18 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.220+0000 7fbc6b7fe700 1 -- 172.21.15.26:0/979943938 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0a19f0 con 0x55d3aae6dc70 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.220+0000 7fbc6b7fe700 1 -- 172.21.15.26:0/979943938 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7c0a1560 con 0x55d3aae6dc70 2023-07-13T22:16:44.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.220+0000 7fbc6b7fe700 1 -- 172.21.15.26:0/979943938 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c09d150 con 0x55d3aae6dc70 2023-07-13T22:16:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.221+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/979943938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 msgr2=0x55d3aae84020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.221+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c096010 tx=0x7fbc7c004ad0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.221+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/979943938 shutdown_connections 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6c4d0 0x55d3aae758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6dc70 0x55d3aae84020 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/979943938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6ce20 0x55d3aae7d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/979943938 >> 172.21.15.26:0/979943938 conn(0x55d3aae61da0 msgr2=0x55d3aae668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/979943938 shutdown_connections 2023-07-13T22:16:44.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/979943938 wait complete. 2023-07-13T22:16:44.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.222+0000 7fbc9e7aeb80 1 -- start start 2023-07-13T22:16:44.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.223+0000 7fbc9e7aeb80 1 Processor -- start 2023-07-13T22:16:44.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae79eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae7d0e0 con 0x55d3aae6dc70 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae7d220 con 0x55d3aae6ce20 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aae7d3e0 con 0x55d3aae6c4d0 2023-07-13T22:16:44.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc9e7aeb80 10 client.0 _ll_register_callbacks cb 0x7fbc7c0991d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:16:44.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc88b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc88b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54510/0 (socket says 172.21.15.26:54510) 2023-07-13T22:16:44.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.224+0000 7fbc88b9b700 1 -- 172.21.15.26:0/965307736 learned_addr learned my addr 172.21.15.26:0/965307736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:16:44.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.225+0000 7fbc8939c700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.225+0000 7fbc88b9b700 1 -- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 msgr2=0x55d3aae79eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae79eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc88b9b700 1 -- 172.21.15.26:0/965307736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 msgr2=0x55d3aae7cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc88b9b700 1 -- 172.21.15.26:0/965307736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3aabd4c60 con 0x55d3aae6c4d0 2023-07-13T22:16:44.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc88b9b700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc7801b040 tx=0x7fbc78009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339155d146edbb41 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:16:44.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.226+0000 7fbc63fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:16:44.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.227+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78003a90 con 0x55d3aae6c4d0 2023-07-13T22:16:44.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.227+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc7802a040 con 0x55d3aae6c4d0 2023-07-13T22:16:44.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.228+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc78004390 con 0x55d3aae6c4d0 2023-07-13T22:16:44.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.229+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc78003bf0 con 0x55d3aae6c4d0 2023-07-13T22:16:44.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.229+0000 7fbc9e7aeb80 10 client.25882 fetch_fsmap learned FSMap version 28 2023-07-13T22:16:44.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.229+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d3aac12250 con 0x55d3aae6c4d0 2023-07-13T22:16:44.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc78004390 con 0x55d3aae6c4d0 2023-07-13T22:16:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fbc78044070 con 0x55d3aae6c4d0 2023-07-13T22:16:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc9e7aeb80 10 client.25882 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:16:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc9e7aeb80 10 client.25882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:16:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc9e7aeb80 20 client.25882 populate_metadata read hostname 'smithi026' 2023-07-13T22:16:44.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.230+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aaed6ef0 0x55d3aaed9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:16:44.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.231+0000 7fbc9e7aeb80 4 client.25882 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:16:44.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.231+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3aad79050 con 0x55d3aaed6ef0 2023-07-13T22:16:44.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.231+0000 7fbc83fff700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aaed6ef0 0x55d3aaed9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:16:44.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.234+0000 7fbc83fff700 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aaed6ef0 0x55d3aaed9320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d396d9f25b258120 server_cookie=f272182df75e521f in_seq=0 out_seq=0 2023-07-13T22:16:44.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.235+0000 7fbc63fff700 10 client.25882 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:44.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.235+0000 7fbc63fff700 1 -- 172.21.15.26:0/965307736 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d3aad79050 con 0x55d3aaed6ef0 2023-07-13T22:16:44.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.235+0000 7fbc63fff700 10 client.25882 handle_command_reply: tid=0 2023-07-13T22:16:44.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.235+0000 7fbc9e7aeb80 1 client.25882 shutdown 2023-07-13T22:16:44.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 msgr2=0x55d3aae79970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc7801b040 tx=0x7fbc78009260 comp rx=0 tx=0).stop 2023-07-13T22:16:44.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 shutdown_connections 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d3aae6dc70 0x55d3aae7cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d3aaed6ef0 0x55d3aaed9320 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3aae6c4d0 0x55d3aae79970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 --2- 172.21.15.26:0/965307736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d3aae6ce20 0x55d3aae79eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 >> 172.21.15.26:0/965307736 conn(0x55d3aae61da0 msgr2=0x55d3aae84020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.236+0000 7fbc63fff700 0 client.25882 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:16:44.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.237+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 shutdown_connections 2023-07-13T22:16:44.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.237+0000 7fbc9e7aeb80 1 -- 172.21.15.26:0/965307736 wait complete. 2023-07-13T22:16:44.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.237+0000 7fbc9e7aeb80 20 client.25882 trim_cache size 0 max 16384 2023-07-13T22:16:44.227 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:16:44.227 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:16:44.227 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: "a8026cd6-9ece-44fb-9b0e-819810d70cd3": { 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "a8026cd6-9ece-44fb-9b0e-819810d70cd3", 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:16:44.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:16:44.229 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:16:44.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.242+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 msgr2=0x7fbc6c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.242+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 0x7fbc6c0a3690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009d80 tx=0x7fbc78009ed0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.242+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 msgr2=0x7fbc841b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:16:44.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.242+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00b970 tx=0x7fbc7c00bc80 comp rx=0 tx=0).stop 2023-07-13T22:16:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 shutdown_connections 2023-07-13T22:16:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc6c0a11e0 0x7fbc6c0a3690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc84070ee0 0x7fbc84113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8410bd60 0x7fbc841b4910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 --2- 172.21.15.26:0/1024125347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc84071830 0x7fbc84113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:16:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 >> 172.21.15.26:0/1024125347 conn(0x7fbc8406ca90 msgr2=0x7fbc8406dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:16:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 shutdown_connections 2023-07-13T22:16:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:16:44.243+0000 7fbc8adff700 1 -- 172.21.15.26:0/1024125347 wait complete. 2023-07-13T22:16:44.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:16:44.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:16:44.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:16:44.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:16:44.345 INFO:tasks.cephfs.filesystem:scrub status for tag:a8026cd6-9ece-44fb-9b0e-819810d70cd3 - {'path': '/', 'tag': 'a8026cd6-9ece-44fb-9b0e-819810d70cd3', 'options': 'recursive,force'} 2023-07-13T22:16:45.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:44 smithi121 ceph-mon[129187]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:16:45.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:44 smithi117 ceph-mon[129074]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:16:45.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:44 smithi026 ceph-mon[95647]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:16:47.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:46 smithi121 ceph-mon[129187]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:16:47.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:46 smithi117 ceph-mon[129074]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:16:47.464 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:46 smithi026 ceph-mon[95647]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:16:49.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:49 smithi117 ceph-mon[129074]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:16:49.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:49 smithi026 ceph-mon[95647]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:16:49.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:49 smithi121 ceph-mon[129187]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:16:51.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:51 smithi117 ceph-mon[129074]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:16:51.464 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:51 smithi026 ceph-mon[95647]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:16:51.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:51 smithi121 ceph-mon[129187]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:16:52.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:52 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:16:52.465 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:52 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:16:52.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:52 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:16:53.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:53 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:16:53.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:53 smithi117 ceph-mon[129074]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:16:53.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:53 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:16:53.459 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:53 smithi026 ceph-mon[95647]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:16:53.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:53 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:16:53.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:53 smithi121 ceph-mon[129187]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:16:54.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:54 smithi117 ceph-mon[129074]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 105 op/s 2023-07-13T22:16:54.462 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:54 smithi026 ceph-mon[95647]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 105 op/s 2023-07-13T22:16:54.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:54 smithi121 ceph-mon[129187]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 105 op/s 2023-07-13T22:16:57.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:56 smithi117 ceph-mon[129074]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:16:57.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:56 smithi026 ceph-mon[95647]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:16:57.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:56 smithi121 ceph-mon[129187]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:16:59.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:16:58 smithi117 ceph-mon[129074]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 91 op/s 2023-07-13T22:16:59.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:16:58 smithi026 ceph-mon[95647]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 91 op/s 2023-07-13T22:16:59.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:16:58 smithi121 ceph-mon[129187]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 91 op/s 2023-07-13T22:17:01.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:00 smithi117 ceph-mon[129074]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 313 MiB/s wr, 96 op/s 2023-07-13T22:17:01.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:00 smithi026 ceph-mon[95647]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 313 MiB/s wr, 96 op/s 2023-07-13T22:17:01.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:00 smithi121 ceph-mon[129187]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 313 MiB/s wr, 96 op/s 2023-07-13T22:17:03.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:02 smithi117 ceph-mon[129074]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 285 MiB/s wr, 88 op/s 2023-07-13T22:17:03.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:02 smithi026 ceph-mon[95647]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 285 MiB/s wr, 88 op/s 2023-07-13T22:17:03.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:02 smithi121 ceph-mon[129187]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 285 MiB/s wr, 88 op/s 2023-07-13T22:17:05.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:04 smithi121 ceph-mon[129187]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 MiB/s wr, 112 op/s 2023-07-13T22:17:05.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:04 smithi117 ceph-mon[129074]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 MiB/s wr, 112 op/s 2023-07-13T22:17:05.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:04 smithi026 ceph-mon[95647]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 353 MiB/s wr, 112 op/s 2023-07-13T22:17:07.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:06 smithi121 ceph-mon[129187]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:17:07.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:06 smithi117 ceph-mon[129074]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:17:07.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:06 smithi026 ceph-mon[95647]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:17:09.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:09 smithi117 ceph-mon[129074]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 303 MiB/s wr, 100 op/s 2023-07-13T22:17:09.458 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:09 smithi026 ceph-mon[95647]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 303 MiB/s wr, 100 op/s 2023-07-13T22:17:09.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:09 smithi121 ceph-mon[129187]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 303 MiB/s wr, 100 op/s 2023-07-13T22:17:11.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:11 smithi117 ceph-mon[129074]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 101 op/s 2023-07-13T22:17:11.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:11 smithi026 ceph-mon[95647]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 101 op/s 2023-07-13T22:17:11.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:11 smithi121 ceph-mon[129187]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 101 op/s 2023-07-13T22:17:13.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:13 smithi117 ceph-mon[129074]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:17:13.470 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:13 smithi026 ceph-mon[95647]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:17:13.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:13 smithi121 ceph-mon[129187]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:17:14.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:17:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.983+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1292006032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4070ee0 msgr2=0x7f4fa40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:14.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.983+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1292006032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4070ee0 0x7f4fa40712f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009990 tx=0x7f4fa0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1292006032 shutdown_connections 2023-07-13T22:17:14.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1292006032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4070ee0 0x7f4fa40712f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1292006032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 0x7f4fa410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1292006032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4071830 0x7f4fa410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1292006032 >> 172.21.15.26:0/1292006032 conn(0x7f4fa406ca90 msgr2=0x7f4fa406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1292006032 shutdown_connections 2023-07-13T22:17:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.984+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1292006032 wait complete. 2023-07-13T22:17:14.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.987+0000 7f4fab5ff700 1 Processor -- start 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 -- start start 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4070ee0 0x7f4fa41a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 0x7f4fa41abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41ac350 con 0x7f4fa4071830 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41ac490 con 0x7f4fa4070ee0 2023-07-13T22:17:14.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.988+0000 7f4fab5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41ac620 con 0x7f4fa410bd60 2023-07-13T22:17:14.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.990+0000 7f4fa939b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4070ee0 0x7f4fa41a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.990+0000 7f4fa8b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.990+0000 7f4fa8b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39898/0 (socket says 172.21.15.26:39898) 2023-07-13T22:17:14.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.990+0000 7f4fa8b9a700 1 -- 172.21.15.26:0/1092665239 learned_addr learned my addr 172.21.15.26:0/1092665239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:14.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.990+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 0x7f4fa41abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 -- 172.21.15.26:0/1092665239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 msgr2=0x7f4fa41abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:14.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 0x7f4fa41abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 -- 172.21.15.26:0/1092665239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4070ee0 msgr2=0x7f4fa41a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:14.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4070ee0 0x7f4fa41a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 -- 172.21.15.26:0/1092665239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00095f0 con 0x7f4fa4071830 2023-07-13T22:17:14.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.991+0000 7f4fa8b9a700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b500 tx=0x7f4f9400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:14.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.992+0000 7f4f9a7fc700 1 -- 172.21.15.26:0/1092665239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94011830 con 0x7f4fa4071830 2023-07-13T22:17:14.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.992+0000 7f4f9a7fc700 1 -- 172.21.15.26:0/1092665239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f94011e50 con 0x7f4fa4071830 2023-07-13T22:17:14.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.992+0000 7f4f9a7fc700 1 -- 172.21.15.26:0/1092665239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400f2e0 con 0x7f4fa4071830 2023-07-13T22:17:14.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.992+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa41ac900 con 0x7f4fa4071830 2023-07-13T22:17:14.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:14.993+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa41ace40 con 0x7f4fa4071830 2023-07-13T22:17:14.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.000+0000 7f4fbee23b80 1 Processor -- start 2023-07-13T22:17:14.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.000+0000 7f4fbee23b80 1 -- start start 2023-07-13T22:17:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e698c70 0x555f7e6990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e59ccb0 con 0x555f7e697e20 2023-07-13T22:17:14.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e5efa40 con 0x555f7e6974d0 2023-07-13T22:17:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e5e8ed0 con 0x555f7e698c70 2023-07-13T22:17:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.001+0000 7f4f9a7fc700 1 -- 172.21.15.26:0/1092665239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f94010580 con 0x7f4fa4071830 2023-07-13T22:17:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.002+0000 7f4fa8b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e698c70 0x555f7e6990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.002+0000 7f4fa939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.002+0000 7f4fa939b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39914/0 (socket says 172.21.15.26:39914) 2023-07-13T22:17:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.002+0000 7f4fa939b700 1 -- 172.21.15.26:0/2957543021 learned_addr learned my addr 172.21.15.26:0/2957543021 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa939b700 1 -- 172.21.15.26:0/2957543021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e698c70 msgr2=0x555f7e6990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e698c70 0x555f7e6990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa939b700 1 -- 172.21.15.26:0/2957543021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 msgr2=0x555f7e6978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:14.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.003+0000 7f4fa939b700 1 -- 172.21.15.26:0/2957543021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f7e416de0 con 0x555f7e697e20 2023-07-13T22:17:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.004+0000 7f4f9a7fc700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 0x7f4f900a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.004+0000 7f4f9a7fc700 1 -- 172.21.15.26:0/1092665239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f940cc280 con 0x7f4fa4071830 2023-07-13T22:17:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.004+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00059d0 tx=0x7f4fa0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f5b043d23980e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:14.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.005+0000 7f4f8f7fe700 1 -- 172.21.15.26:0/2957543021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000b9d0 con 0x555f7e697e20 2023-07-13T22:17:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.005+0000 7f4f8f7fe700 1 -- 172.21.15.26:0/2957543021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fa001e040 con 0x555f7e697e20 2023-07-13T22:17:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.005+0000 7f4f8f7fe700 1 -- 172.21.15.26:0/2957543021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000f460 con 0x555f7e697e20 2023-07-13T22:17:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.006+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2957543021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 msgr2=0x555f7e6986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:14.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.006+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4fa00059d0 tx=0x7f4fa0009ee0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.007+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2957543021 shutdown_connections 2023-07-13T22:17:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.007+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e698c70 0x555f7e6990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.007+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.007+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2957543021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6986e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.008+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2957543021 >> 172.21.15.26:0/2957543021 conn(0x555f7e68cda0 msgr2=0x555f7e6921d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:14.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.008+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2957543021 shutdown_connections 2023-07-13T22:17:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.008+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2957543021 wait complete. 2023-07-13T22:17:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.009+0000 7f4fbee23b80 1 -- start start 2023-07-13T22:17:14.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.009+0000 7f4fbee23b80 1 Processor -- start 2023-07-13T22:17:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6a1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e588ac0 con 0x555f7e698c70 2023-07-13T22:17:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e5887c0 con 0x555f7e6974d0 2023-07-13T22:17:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e46c920 con 0x555f7e697e20 2023-07-13T22:17:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.010+0000 7f4fbee23b80 10 client.0 _ll_register_callbacks cb 0x555f7e586760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:14.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.011+0000 7f4fa9b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.011+0000 7f4fa9b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39930/0 (socket says 172.21.15.26:39930) 2023-07-13T22:17:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.011+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2622320329 learned_addr learned my addr 172.21.15.26:0/2622320329 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.012+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2622320329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 msgr2=0x555f7e6a1680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.012+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a1680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.012+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 msgr2=0x555f7e6a1140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.012+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6a1140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.012+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2622320329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa4117640 con 0x555f7e698c70 2023-07-13T22:17:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.013+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 0x7f4f900a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.016+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00ad20 tx=0x7f4f9c011430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0a9645b1f80e42 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.016+0000 7f4f8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:15.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.021+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00f070 con 0x555f7e698c70 2023-07-13T22:17:15.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.021+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f9c01f020 con 0x555f7e698c70 2023-07-13T22:17:15.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.021+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f7e5424f0 con 0x555f7e698c70 2023-07-13T22:17:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.025+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f9c019b40 con 0x555f7e698c70 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.025+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 0x7f4f900a37d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c020000 tx=0x7f4f9c01d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.025+0000 7f4fbee23b80 10 client.15735 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.025+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f7e3ffc60 con 0x555f7e698c70 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.025+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f7e5424f0 con 0x555f7e698c70 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4f9c023b90 con 0x555f7e698c70 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 20 client.15735 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e6ad040 0x555f7e6af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 4 client.15735 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.026+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f7e5e1eb0 con 0x555f7e6ad040 2023-07-13T22:17:15.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.027+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e6ad040 0x555f7e6af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.028+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e6ad040 0x555f7e6af450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b38d28372268216 server_cookie=fefb164ab100aa5d in_seq=0 out_seq=0 2023-07-13T22:17:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.028+0000 7f4f8cff9700 10 client.15735 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2622320329 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f7e5e1eb0 con 0x555f7e6ad040 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4f8cff9700 10 client.15735 handle_command_reply: tid=0 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 client.15735 shutdown 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 msgr2=0x555f7e6a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00ad20 tx=0x7f4f9c011430 comp rx=0 tx=0).stop 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 shutdown_connections 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e698c70 0x555f7e6a86c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e6ad040 0x555f7e6af450 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e6974d0 0x555f7e6a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2622320329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 >> 172.21.15.26:0/2622320329 conn(0x555f7e68cda0 msgr2=0x555f7e6918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.029+0000 7f4f8cff9700 0 client.15735 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.030+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 shutdown_connections 2023-07-13T22:17:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.030+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2622320329 wait complete. 2023-07-13T22:17:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.030+0000 7f4fbee23b80 20 client.15735 trim_cache size 0 max 16384 2023-07-13T22:17:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 Processor -- start 2023-07-13T22:17:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 -- start start 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 0x555f7e69f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e69bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e693330 con 0x555f7e697e20 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.052+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e692ed0 con 0x555f7e698c70 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.053+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e69ee70 con 0x555f7e6974d0 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.053+0000 7f4fa939b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 0x555f7e69f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.053+0000 7f4fa939b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 0x555f7e69f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57354/0 (socket says 172.21.15.26:57354) 2023-07-13T22:17:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.053+0000 7f4fa939b700 1 -- 172.21.15.26:0/1677124593 learned_addr learned my addr 172.21.15.26:0/1677124593 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.053+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e69bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/1677124593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 msgr2=0x555f7e69f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 0x555f7e69f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/1677124593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 msgr2=0x555f7e6a85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/1677124593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f7e43d250 con 0x555f7e698c70 2023-07-13T22:17:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.054+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e69bc10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c018410 tx=0x7f4f9c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b8b5270e1a0660 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.063+0000 7f4f8ffff700 1 -- 172.21.15.26:0/1677124593 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00f040 con 0x555f7e698c70 2023-07-13T22:17:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.063+0000 7f4f8ffff700 1 -- 172.21.15.26:0/1677124593 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f9c007000 con 0x555f7e698c70 2023-07-13T22:17:15.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.063+0000 7f4f8ffff700 1 -- 172.21.15.26:0/1677124593 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0236f0 con 0x555f7e698c70 2023-07-13T22:17:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 -- 172.21.15.26:0/1677124593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 msgr2=0x555f7e69bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e69bc10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c018410 tx=0x7f4f9c009f60 comp rx=0 tx=0).stop 2023-07-13T22:17:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 -- 172.21.15.26:0/1677124593 shutdown_connections 2023-07-13T22:17:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e69bc10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e6974d0 0x555f7e69f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/1677124593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e697e20 0x555f7e6a85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.075+0000 7f4fbee23b80 1 -- 172.21.15.26:0/1677124593 >> 172.21.15.26:0/1677124593 conn(0x555f7e68cda0 msgr2=0x555f7e69fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.082+0000 7f4fbee23b80 1 -- 172.21.15.26:0/1677124593 shutdown_connections 2023-07-13T22:17:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.083+0000 7f4fbee23b80 1 -- 172.21.15.26:0/1677124593 wait complete. 2023-07-13T22:17:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.086+0000 7f4fbee23b80 1 -- start start 2023-07-13T22:17:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.088+0000 7f4fbee23b80 1 Processor -- start 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e6974d0 0x555f7e6a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e6a4fa0 con 0x555f7e6974d0 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e6a5110 con 0x555f7e698c70 2023-07-13T22:17:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f7e6a52d0 con 0x555f7e697e20 2023-07-13T22:17:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.090+0000 7f4fbee23b80 10 client.0 _ll_register_callbacks cb 0x7f4f9c00a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.091+0000 7f4fa9b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.091+0000 7f4fa9b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40718/0 (socket says 172.21.15.26:40718) 2023-07-13T22:17:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.091+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2407112572 learned_addr learned my addr 172.21.15.26:0/2407112572 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.093+0000 7f4fa8b9a700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.094+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2407112572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 msgr2=0x555f7e6a83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.094+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.094+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2407112572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e6974d0 msgr2=0x555f7e6a7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.094+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e6974d0 0x555f7e6a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.094+0000 7f4fa9b9c700 1 -- 172.21.15.26:0/2407112572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f7e3ffc60 con 0x555f7e698c70 2023-07-13T22:17:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.095+0000 7f4fa9b9c700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0183e0 tx=0x7f4f9c009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813745fea2cdc727 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.114+0000 7f4f8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.114+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c019c40 con 0x555f7e698c70 2023-07-13T22:17:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.115+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f9c019da0 con 0x555f7e698c70 2023-07-13T22:17:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.116+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fa001e040 con 0x555f7e698c70 2023-07-13T22:17:15.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.128+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f9c003cb0 con 0x555f7e698c70 2023-07-13T22:17:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.128+0000 7f4fbee23b80 10 client.25898 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.128+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f7e4152b0 con 0x555f7e698c70 2023-07-13T22:17:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fa001e040 con 0x555f7e698c70 2023-07-13T22:17:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4f9c035070 con 0x555f7e698c70 2023-07-13T22:17:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 20 client.25898 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e7130c0 0x555f7e7154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 4 client.25898 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:17:15.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.137+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f7e5a4050 con 0x555f7e7130c0 2023-07-13T22:17:15.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.148+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e7130c0 0x555f7e7154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.197+0000 7f4fa939b700 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e7130c0 0x555f7e7154d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc566629a744db1d server_cookie=67b5cde602ebe91c in_seq=0 out_seq=0 2023-07-13T22:17:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.200+0000 7f4f8cff9700 10 client.25898 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.211+0000 7f4f8cff9700 1 -- 172.21.15.26:0/2407112572 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555f7e5a4050 con 0x555f7e7130c0 2023-07-13T22:17:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.211+0000 7f4f8cff9700 10 client.25898 handle_command_reply: tid=0 2023-07-13T22:17:15.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.211+0000 7f4fbee23b80 1 client.25898 shutdown 2023-07-13T22:17:15.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.215+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 msgr2=0x555f7e6a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.215+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c0183e0 tx=0x7f4f9c009cf0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 shutdown_connections 2023-07-13T22:17:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555f7e6974d0 0x555f7e6a7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555f7e7130c0 0x555f7e7154d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555f7e698c70 0x555f7e6a4950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 --2- 172.21.15.26:0/2407112572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f7e697e20 0x555f7e6a83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 >> 172.21.15.26:0/2407112572 conn(0x555f7e68cda0 msgr2=0x555f7e69f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:15.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.216+0000 7f4f8cff9700 0 client.25898 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:15.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.230+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 shutdown_connections 2023-07-13T22:17:15.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.230+0000 7f4fbee23b80 1 -- 172.21.15.26:0/2407112572 wait complete. 2023-07-13T22:17:15.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.232+0000 7f4fbee23b80 20 client.25898 trim_cache size 0 max 16384 2023-07-13T22:17:15.220 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:17:15.220 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:17:15.221 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:17:15.221 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:17:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.233+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 msgr2=0x7f4f900a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.233+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 0x7f4f900a37d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c020000 tx=0x7f4f9c01d000 comp rx=0 tx=0).stop 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.233+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 msgr2=0x7f4fa41a4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.233+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b500 tx=0x7f4f9400b810 comp rx=0 tx=0).stop 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 shutdown_connections 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4fa410bd60 0x7f4fa41abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fa4070ee0 0x7f4fa41a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f900a1320 0x7f4f900a37d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 --2- 172.21.15.26:0/1092665239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa4071830 0x7f4fa41a4d10 secure :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400b500 tx=0x7f4f9400b810 comp rx=0 tx=0).stop 2023-07-13T22:17:15.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.234+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 >> 172.21.15.26:0/1092665239 conn(0x7f4fa406ca90 msgr2=0x7f4fa406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.243+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 shutdown_connections 2023-07-13T22:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.243+0000 7f4fab5ff700 1 -- 172.21.15.26:0/1092665239 wait complete. 2023-07-13T22:17:15.275 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:17:15.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:17:15.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:15 smithi026 ceph-mon[95647]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:17:15.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:15 smithi121 ceph-mon[129187]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:17:15.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:15 smithi117 ceph-mon[129074]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:17:15.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 -- 172.21.15.26:0/1388316073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734071820 msgr2=0x7ff73410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1388316073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734071820 0x7ff73410b7d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00b0a0 tx=0x7ff72c00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 -- 172.21.15.26:0/1388316073 shutdown_connections 2023-07-13T22:17:15.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1388316073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff734070ed0 0x7ff7340712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1388316073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff73410bd60 0x7ff73410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1388316073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734071820 0x7ff73410b7d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.900+0000 7ff7393ff700 1 -- 172.21.15.26:0/1388316073 >> 172.21.15.26:0/1388316073 conn(0x7ff73406ca40 msgr2=0x7ff73406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.909+0000 7ff7393ff700 1 -- 172.21.15.26:0/1388316073 shutdown_connections 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.911+0000 7ff7393ff700 1 -- 172.21.15.26:0/1388316073 wait complete. 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.914+0000 7ff7393ff700 1 Processor -- start 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.914+0000 7ff7393ff700 1 -- start start 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734070ed0 0x7ff7341a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73410bd60 0x7ff7341abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341ac210 con 0x7ff73410bd60 2023-07-13T22:17:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341ac350 con 0x7ff734071820 2023-07-13T22:17:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.915+0000 7ff7393ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341ac4e0 con 0x7ff734070ed0 2023-07-13T22:17:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40740/0 (socket says 172.21.15.26:40740) 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 -- 172.21.15.26:0/1439926515 learned_addr learned my addr 172.21.15.26:0/1439926515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff732ffd700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734070ed0 0x7ff7341a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 -- 172.21.15.26:0/1439926515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734070ed0 msgr2=0x7ff7341a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734070ed0 0x7ff7341a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 -- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73410bd60 msgr2=0x7ff7341abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73410bd60 0x7ff7341abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.916+0000 7ff7327fc700 1 -- 172.21.15.26:0/1439926515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c009d00 con 0x7ff734071820 2023-07-13T22:17:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.917+0000 7ff7327fc700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007b30 tx=0x7ff72c00bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.917+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c021070 con 0x7ff734071820 2023-07-13T22:17:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.917+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff72c004c60 con 0x7ff734071820 2023-07-13T22:17:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.917+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00e040 con 0x7ff734071820 2023-07-13T22:17:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.918+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7341ac760 con 0x7ff734071820 2023-07-13T22:17:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.918+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7341acbb0 con 0x7ff734071820 2023-07-13T22:17:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.921+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73411a1d0 con 0x7ff734071820 2023-07-13T22:17:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.924+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff72c004740 con 0x7ff734071820 2023-07-13T22:17:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.927+0000 7ff71bfff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 0x7ff71c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.928+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff72c0cd760 con 0x7ff734071820 2023-07-13T22:17:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.929+0000 7ff732ffd700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 0x7ff71c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.930+0000 7ff732ffd700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 0x7ff71c0a37f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff728006fd0 tx=0x7ff728008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:15.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:15.931+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff72c096fe0 con 0x7ff734071820 2023-07-13T22:17:16.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.618+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7341ad2b0 con 0x7ff734071820 2023-07-13T22:17:16.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.619+0000 7ff71bfff700 1 -- 172.21.15.26:0/1439926515 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10892 (secure 0 0 0) 0x7ff72c09a600 con 0x7ff734071820 2023-07-13T22:17:16.611 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:17:16.611 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:15:26.683417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:17:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 msgr2=0x7ff71c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 0x7ff71c0a37f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff728006fd0 tx=0x7ff728008040 comp rx=0 tx=0).stop 2023-07-13T22:17:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 msgr2=0x7ff7341a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007b30 tx=0x7ff72c00bd90 comp rx=0 tx=0).stop 2023-07-13T22:17:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 shutdown_connections 2023-07-13T22:17:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff71c0a1340 0x7ff71c0a37f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff73410bd60 0x7ff7341abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff734070ed0 0x7ff7341a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 --2- 172.21.15.26:0/1439926515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff734071820 0x7ff7341a4bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 >> 172.21.15.26:0/1439926515 conn(0x7ff73406ca40 msgr2=0x7ff73406dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 shutdown_connections 2023-07-13T22:17:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:16.627+0000 7ff7393ff700 1 -- 172.21.15.26:0/1439926515 wait complete. 2023-07-13T22:17:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2023-07-13T22:17:16.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:17:16.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:16 smithi026 ceph-mon[95647]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 266 MiB/s wr, 84 op/s 2023-07-13T22:17:16.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:16 smithi117 ceph-mon[129074]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 266 MiB/s wr, 84 op/s 2023-07-13T22:17:16.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:16 smithi121 ceph-mon[129187]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 266 MiB/s wr, 84 op/s 2023-07-13T22:17:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.335+0000 7fb8743ff700 1 -- 172.21.15.26:0/2224119188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 msgr2=0x7fb86c100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.335+0000 7fb8743ff700 1 --2- 172.21.15.26:0/2224119188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 0x7fb86c100020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8680098f0 tx=0x7fb868009c00 comp rx=0 tx=0).stop 2023-07-13T22:17:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.336+0000 7fb8743ff700 1 -- 172.21.15.26:0/2224119188 shutdown_connections 2023-07-13T22:17:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.336+0000 7fb8743ff700 1 --2- 172.21.15.26:0/2224119188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb86c100560 0x7fb86c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.336+0000 7fb8743ff700 1 --2- 172.21.15.26:0/2224119188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.336+0000 7fb8743ff700 1 --2- 172.21.15.26:0/2224119188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 0x7fb86c100020 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.336+0000 7fb8743ff700 1 -- 172.21.15.26:0/2224119188 >> 172.21.15.26:0/2224119188 conn(0x7fb86c0fa000 msgr2=0x7fb86c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.337+0000 7fb8743ff700 1 -- 172.21.15.26:0/2224119188 shutdown_connections 2023-07-13T22:17:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.337+0000 7fb8743ff700 1 -- 172.21.15.26:0/2224119188 wait complete. 2023-07-13T22:17:17.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.339+0000 7fb8743ff700 1 Processor -- start 2023-07-13T22:17:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.339+0000 7fb8743ff700 1 -- start start 2023-07-13T22:17:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 0x7fb86c06a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb86c100560 0x7fb86c071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c071b50 con 0x7fb86c100560 2023-07-13T22:17:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c071c90 con 0x7fb86c0ffbd0 2023-07-13T22:17:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.340+0000 7fb8743ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c071e20 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.347+0000 7fb87219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.347+0000 7fb87219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57410/0 (socket says 172.21.15.26:57410) 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.347+0000 7fb87219b700 1 -- 172.21.15.26:0/247527019 learned_addr learned my addr 172.21.15.26:0/247527019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.348+0000 7fb87219b700 1 -- 172.21.15.26:0/247527019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 msgr2=0x7fb86c06a690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.348+0000 7fb87219b700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 0x7fb86c06a690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.348+0000 7fb87219b700 1 -- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb86c100560 msgr2=0x7fb86c071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.348+0000 7fb87219b700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb86c100560 0x7fb86c071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.348+0000 7fb87219b700 1 -- 172.21.15.26:0/247527019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb868009520 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.351+0000 7fb87219b700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d950 tx=0x7fb85c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.351+0000 7fb8637fe700 1 -- 172.21.15.26:0/247527019 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c009b40 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.351+0000 7fb8637fe700 1 -- 172.21.15.26:0/247527019 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb85c00de30 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.351+0000 7fb8637fe700 1 -- 172.21.15.26:0/247527019 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00f620 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.352+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c072100 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.353+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c072590 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.360+0000 7fb887e22b80 1 Processor -- start 2023-07-13T22:17:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.361+0000 7fb887e22b80 1 -- start start 2023-07-13T22:17:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d3fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d3fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d301cb0 con 0x55659d3fce20 2023-07-13T22:17:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d354a40 con 0x55659d3fc4d0 2023-07-13T22:17:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb887e22b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d34ded0 con 0x55659d3fdc70 2023-07-13T22:17:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40770/0 (socket says 172.21.15.26:40770) 2023-07-13T22:17:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.362+0000 7fb87299c700 1 -- 172.21.15.26:0/1439354186 learned_addr learned my addr 172.21.15.26:0/1439354186 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 -- 172.21.15.26:0/1439354186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 msgr2=0x55659d3fe0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:17:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d3fe0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 -- 172.21.15.26:0/1439354186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 msgr2=0x55659d3fd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d3fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 -- 172.21.15.26:0/1439354186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659d17bde0 con 0x55659d3fc4d0 2023-07-13T22:17:17.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.367+0000 7fb87299c700 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb864009c60 tx=0x7fb864009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e20c13df88d39a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.368+0000 7fb8637fe700 1 -- 172.21.15.26:0/247527019 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb85c010460 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.370+0000 7fb8637fe700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 0x7fb8580a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.372+0000 7fb8637fe700 1 -- 172.21.15.26:0/247527019 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb85c0ce150 con 0x7fb86c0fe9e0 2023-07-13T22:17:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.374+0000 7fb87219b700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 0x7fb8580a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.375+0000 7fb87219b700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 0x7fb8580a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb85c012040 tx=0x7fb85c009f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.375+0000 7fb860bfe700 1 -- 172.21.15.26:0/1439354186 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864017070 con 0x55659d3fc4d0 2023-07-13T22:17:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.375+0000 7fb860bfe700 1 -- 172.21.15.26:0/1439354186 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86400f040 con 0x55659d3fc4d0 2023-07-13T22:17:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.375+0000 7fb860bfe700 1 -- 172.21.15.26:0/1439354186 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864005980 con 0x55659d3fc4d0 2023-07-13T22:17:17.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.376+0000 7fb887e22b80 1 -- 172.21.15.26:0/1439354186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 msgr2=0x55659d3fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.376+0000 7fb887e22b80 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb864009c60 tx=0x7fb864009f70 comp rx=0 tx=0).stop 2023-07-13T22:17:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 -- 172.21.15.26:0/1439354186 shutdown_connections 2023-07-13T22:17:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d3fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d3fc8e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 --2- 172.21.15.26:0/1439354186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d3fd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 -- 172.21.15.26:0/1439354186 >> 172.21.15.26:0/1439354186 conn(0x55659d3f1da0 msgr2=0x55659d3f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 -- 172.21.15.26:0/1439354186 shutdown_connections 2023-07-13T22:17:17.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.378+0000 7fb887e22b80 1 -- 172.21.15.26:0/1439354186 wait complete. 2023-07-13T22:17:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.379+0000 7fb887e22b80 1 -- start start 2023-07-13T22:17:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.380+0000 7fb887e22b80 1 Processor -- start 2023-07-13T22:17:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.380+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d4061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.380+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fce20 0x55659d406730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.381+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.381+0000 7fb887e22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d2edac0 con 0x55659d3fc4d0 2023-07-13T22:17:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.381+0000 7fb887e22b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d2ed7c0 con 0x55659d3fdc70 2023-07-13T22:17:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.381+0000 7fb887e22b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d1d1920 con 0x55659d3fce20 2023-07-13T22:17:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.381+0000 7fb887e22b80 10 client.0 _ll_register_callbacks cb 0x55659d2eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40772/0 (socket says 172.21.15.26:40772) 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 -- 172.21.15.26:0/3623391558 learned_addr learned my addr 172.21.15.26:0/3623391558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d4061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 -- 172.21.15.26:0/3623391558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fce20 msgr2=0x55659d406730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fce20 0x55659d406730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 -- 172.21.15.26:0/3623391558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 msgr2=0x55659d4061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d4061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.382+0000 7fb87299c700 1 -- 172.21.15.26:0/3623391558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8640098c0 con 0x55659d3fdc70 2023-07-13T22:17:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.383+0000 7fb87299c700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8640054b0 tx=0x7fb864006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf552c59e48b159 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.383+0000 7fb855ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.383+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864017040 con 0x55659d3fdc70 2023-07-13T22:17:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.383+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb864005a10 con 0x55659d3fdc70 2023-07-13T22:17:17.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.383+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55659d2a74f0 con 0x55659d3fdc70 2023-07-13T22:17:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.386+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb864003c00 con 0x55659d3fdc70 2023-07-13T22:17:17.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.388+0000 7fb887e22b80 10 client.25913 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.388+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55659d164c60 con 0x55659d3fdc70 2023-07-13T22:17:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55659d2a74f0 con 0x55659d3fdc70 2023-07-13T22:17:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb864012070 con 0x55659d3fdc70 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 10 client.25913 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 20 client.25913 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d411e10 0x55659d414220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 4 client.25913 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.405+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55659d346eb0 con 0x55659d411e10 2023-07-13T22:17:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.406+0000 7fb87199a700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d411e10 0x55659d414220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.417+0000 7fb87199a700 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d411e10 0x55659d414220 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6ab3cf665b1d1cd3 server_cookie=693a52ea434b69d2 in_seq=0 out_seq=0 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.417+0000 7fb855ffb700 10 client.25913 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.422+0000 7fb855ffb700 1 -- 172.21.15.26:0/3623391558 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55659d346eb0 con 0x55659d411e10 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.422+0000 7fb855ffb700 10 client.25913 handle_command_reply: tid=0 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.422+0000 7fb887e22b80 1 client.25913 shutdown 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.423+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 msgr2=0x55659d40d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.423+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8640054b0 tx=0x7fb864006000 comp rx=0 tx=0).stop 2023-07-13T22:17:17.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 shutdown_connections 2023-07-13T22:17:17.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d411e10 0x55659d414220 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d4061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fdc70 0x55659d40d770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 --2- 172.21.15.26:0/3623391558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fce20 0x55659d406730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 >> 172.21.15.26:0/3623391558 conn(0x55659d3f1da0 msgr2=0x55659d3f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.426+0000 7fb855ffb700 0 client.25913 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:17.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.427+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 shutdown_connections 2023-07-13T22:17:17.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.427+0000 7fb887e22b80 1 -- 172.21.15.26:0/3623391558 wait complete. 2023-07-13T22:17:17.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.437+0000 7fb887e22b80 20 client.25913 trim_cache size 0 max 16384 2023-07-13T22:17:17.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.524+0000 7fb887e22b80 1 Processor -- start 2023-07-13T22:17:17.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 -- start start 2023-07-13T22:17:17.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d40d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d404e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d4056a0 con 0x55659d3fce20 2023-07-13T22:17:17.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d3f8d00 con 0x55659d3fc4d0 2023-07-13T22:17:17.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.537+0000 7fb887e22b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d406e70 con 0x55659d3fdc70 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40780/0 (socket says 172.21.15.26:40780) 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 -- 172.21.15.26:0/665064697 learned_addr learned my addr 172.21.15.26:0/665064697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d404e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 -- 172.21.15.26:0/665064697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 msgr2=0x55659d404e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d404e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 -- 172.21.15.26:0/665064697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 msgr2=0x55659d40d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.539+0000 7fb87299c700 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d40d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.540+0000 7fb87299c700 1 -- 172.21.15.26:0/665064697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55659d1a2250 con 0x55659d3fc4d0 2023-07-13T22:17:17.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.540+0000 7fb87299c700 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8640092d0 tx=0x7fb864005330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf9e6e9696666075 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.564+0000 7fb8613ff700 1 -- 172.21.15.26:0/665064697 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8640054e0 con 0x55659d3fc4d0 2023-07-13T22:17:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.564+0000 7fb8613ff700 1 -- 172.21.15.26:0/665064697 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb864004920 con 0x55659d3fc4d0 2023-07-13T22:17:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.564+0000 7fb8613ff700 1 -- 172.21.15.26:0/665064697 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86401db70 con 0x55659d3fc4d0 2023-07-13T22:17:17.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.571+0000 7fb887e22b80 1 -- 172.21.15.26:0/665064697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 msgr2=0x55659d4058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.571+0000 7fb887e22b80 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8640092d0 tx=0x7fb864005330 comp rx=0 tx=0).stop 2023-07-13T22:17:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.583+0000 7fb887e22b80 1 -- 172.21.15.26:0/665064697 shutdown_connections 2023-07-13T22:17:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.583+0000 7fb887e22b80 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d404e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.583+0000 7fb887e22b80 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fc4d0 0x55659d4058a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.583+0000 7fb887e22b80 1 --2- 172.21.15.26:0/665064697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fce20 0x55659d40d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.583+0000 7fb887e22b80 1 -- 172.21.15.26:0/665064697 >> 172.21.15.26:0/665064697 conn(0x55659d3f1da0 msgr2=0x55659d418b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.587+0000 7fb887e22b80 1 -- 172.21.15.26:0/665064697 shutdown_connections 2023-07-13T22:17:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.590+0000 7fb887e22b80 1 -- 172.21.15.26:0/665064697 wait complete. 2023-07-13T22:17:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.591+0000 7fb887e22b80 1 -- start start 2023-07-13T22:17:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.591+0000 7fb887e22b80 1 Processor -- start 2023-07-13T22:17:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d409650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d40cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d40d0f0 con 0x55659d3fc4d0 2023-07-13T22:17:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d40d260 con 0x55659d3fce20 2023-07-13T22:17:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55659d40d420 con 0x55659d3fdc70 2023-07-13T22:17:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.592+0000 7fb887e22b80 10 client.0 _ll_register_callbacks cb 0x7fb864009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:17.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.606+0000 7fb87199a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.606+0000 7fb87199a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40796/0 (socket says 172.21.15.26:40796) 2023-07-13T22:17:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.606+0000 7fb87199a700 1 -- 172.21.15.26:0/4035552427 learned_addr learned my addr 172.21.15.26:0/4035552427 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.606+0000 7fb87299c700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d40cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.606+0000 7fb87299c700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d409650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 -- 172.21.15.26:0/4035552427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 msgr2=0x55659d40cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d40cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 -- 172.21.15.26:0/4035552427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 msgr2=0x55659d409650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:17.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d409650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 -- 172.21.15.26:0/4035552427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55659d164c60 con 0x55659d3fce20 2023-07-13T22:17:17.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb87199a700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb868009230 tx=0x7fb868005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aec80346df89c01 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:17.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb855ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:17.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.607+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86801c040 con 0x55659d3fce20 2023-07-13T22:17:17.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.608+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb86400f040 con 0x55659d3fce20 2023-07-13T22:17:17.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.608+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8680235c0 con 0x55659d3fce20 2023-07-13T22:17:17.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.635+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868010e80 con 0x55659d3fce20 2023-07-13T22:17:17.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.657+0000 7fb887e22b80 10 client.25919 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:17.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.657+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55659d17a2b0 con 0x55659d3fce20 2023-07-13T22:17:17.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.668+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb86400f040 con 0x55659d3fce20 2023-07-13T22:17:17.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.668+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fb868015020 con 0x55659d3fce20 2023-07-13T22:17:17.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 10 client.25919 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:17.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 10 client.25919 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:17:17.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 20 client.25919 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:17.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d466db0 0x55659d4691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:17.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 4 client.25919 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:17.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.669+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55659d309050 con 0x55659d466db0 2023-07-13T22:17:17.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.677+0000 7fb87299c700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d466db0 0x55659d4691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:17.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.700+0000 7fb87299c700 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d466db0 0x55659d4691c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65119af6b9649eab server_cookie=e8b96e1ea8f7312b in_seq=0 out_seq=0 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb855ffb700 10 client.25919 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb855ffb700 1 -- 172.21.15.26:0/4035552427 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55659d309050 con 0x55659d466db0 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb855ffb700 10 client.25919 handle_command_reply: tid=0 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb887e22b80 1 client.25919 shutdown 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 msgr2=0x55659d409b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.701+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb868009230 tx=0x7fb868005730 comp rx=0 tx=0).stop 2023-07-13T22:17:17.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 shutdown_connections 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55659d466db0 0x55659d4691c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55659d3fc4d0 0x55659d409650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55659d3fdc70 0x55659d40cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 --2- 172.21.15.26:0/4035552427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55659d3fce20 0x55659d409b90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 >> 172.21.15.26:0/4035552427 conn(0x55659d3f1da0 msgr2=0x55659d404e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.702+0000 7fb855ffb700 0 client.25919 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:17.692 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:17 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1439926515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:17.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.703+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 shutdown_connections 2023-07-13T22:17:17.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.703+0000 7fb887e22b80 1 -- 172.21.15.26:0/4035552427 wait complete. 2023-07-13T22:17:17.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.704+0000 7fb887e22b80 20 client.25919 trim_cache size 0 max 16384 2023-07-13T22:17:17.693 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:17.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 msgr2=0x7fb8580a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 0x7fb8580a3800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb85c012040 tx=0x7fb85c009f60 comp rx=0 tx=0).stop 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 msgr2=0x7fb86c06a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00d950 tx=0x7fb85c00dd10 comp rx=0 tx=0).stop 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 shutdown_connections 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb8580a1350 0x7fb8580a3800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb86c100560 0x7fb86c071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb86c0fe9e0 0x7fb86c06a150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 --2- 172.21.15.26:0/247527019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb86c0ffbd0 0x7fb86c06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 >> 172.21.15.26:0/247527019 conn(0x7fb86c0fa000 msgr2=0x7fb86c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.705+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 shutdown_connections 2023-07-13T22:17:17.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:17.706+0000 7fb8743ff700 1 -- 172.21.15.26:0/247527019 wait complete. 2023-07-13T22:17:17.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:17 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1439926515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:17.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:17 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1439926515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:17.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:17:18.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 -- 172.21.15.26:0/2511914363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 msgr2=0x7f8474103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2511914363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f8474103130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8464009990 tx=0x7f8464009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 -- 172.21.15.26:0/2511914363 shutdown_connections 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2511914363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84740fe350 0x7f8474100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2511914363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f8474103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2511914363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f8474103130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.442+0000 7f847c3ff700 1 -- 172.21.15.26:0/2511914363 >> 172.21.15.26:0/2511914363 conn(0x7f84740fa000 msgr2=0x7f84740fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.443+0000 7f847c3ff700 1 -- 172.21.15.26:0/2511914363 shutdown_connections 2023-07-13T22:17:18.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.443+0000 7f847c3ff700 1 -- 172.21.15.26:0/2511914363 wait complete. 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.444+0000 7f847c3ff700 1 Processor -- start 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.444+0000 7f847c3ff700 1 -- start start 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84740fe350 0x7f847406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f84741ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84740723d0 con 0x7f84740fe350 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474072540 con 0x7f8474100cb0 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847c3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474104970 con 0x7f8474103670 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40824/0 (socket says 172.21.15.26:40824) 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.445+0000 7f847999a700 1 -- 172.21.15.26:0/2725775467 learned_addr learned my addr 172.21.15.26:0/2725775467 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847a99c700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f84741ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 -- 172.21.15.26:0/2725775467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 msgr2=0x7f84741ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f84741ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 -- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84740fe350 msgr2=0x7f847406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84740fe350 0x7f847406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 -- 172.21.15.26:0/2725775467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640095f0 con 0x7f8474100cb0 2023-07-13T22:17:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847a99c700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f84741ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.446+0000 7f847999a700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8464005910 tx=0x7f846400f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.447+0000 7f846b7fe700 1 -- 172.21.15.26:0/2725775467 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846401d070 con 0x7f8474100cb0 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.447+0000 7f846b7fe700 1 -- 172.21.15.26:0/2725775467 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846400fc30 con 0x7f8474100cb0 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.447+0000 7f846b7fe700 1 -- 172.21.15.26:0/2725775467 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464017650 con 0x7f8474100cb0 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.447+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8474104b90 con 0x7f8474100cb0 2023-07-13T22:17:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.447+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8474104ff0 con 0x7f8474100cb0 2023-07-13T22:17:18.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.453+0000 7f846b7fe700 1 -- 172.21.15.26:0/2725775467 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8464022470 con 0x7f8474100cb0 2023-07-13T22:17:18.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.453+0000 7f848fd69b80 1 Processor -- start 2023-07-13T22:17:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 -- start start 2023-07-13T22:17:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af984f4cb0 con 0x55af985f0c70 2023-07-13T22:17:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af98547a40 con 0x55af985ef4d0 2023-07-13T22:17:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.454+0000 7f848fd69b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af98540ed0 con 0x55af985efe20 2023-07-13T22:17:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57478/0 (socket says 172.21.15.26:57478) 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a19b700 1 -- 172.21.15.26:0/2558073311 learned_addr learned my addr 172.21.15.26:0/2558073311 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 -- 172.21.15.26:0/2558073311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 msgr2=0x55af985f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 -- 172.21.15.26:0/2558073311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 msgr2=0x55af985f10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.455+0000 7f847a99c700 1 -- 172.21.15.26:0/2558073311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af9836ede0 con 0x55af985ef4d0 2023-07-13T22:17:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.456+0000 7f847a99c700 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985ef8e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f846c009e10 tx=0x7f846c0077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d82c250716754f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.456+0000 7f8468bfe700 1 -- 172.21.15.26:0/2558073311 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c014610 con 0x55af985ef4d0 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.456+0000 7f8468bfe700 1 -- 172.21.15.26:0/2558073311 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846c00f050 con 0x55af985ef4d0 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.456+0000 7f8468bfe700 1 -- 172.21.15.26:0/2558073311 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c013a20 con 0x55af985ef4d0 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.456+0000 7f846b7fe700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 0x7f84600a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f846b7fe700 1 -- 172.21.15.26:0/2725775467 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84640cd830 con 0x7f8474100cb0 2023-07-13T22:17:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f847a19b700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 0x7f84600a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 -- 172.21.15.26:0/2558073311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 msgr2=0x55af985ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985ef8e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f846c009e10 tx=0x7f846c0077e0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 -- 172.21.15.26:0/2558073311 shutdown_connections 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985ef8e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2558073311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.457+0000 7f848fd69b80 1 -- 172.21.15.26:0/2558073311 >> 172.21.15.26:0/2558073311 conn(0x55af985e4da0 msgr2=0x55af985ea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.458+0000 7f848fd69b80 1 -- 172.21.15.26:0/2558073311 shutdown_connections 2023-07-13T22:17:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.458+0000 7f848fd69b80 1 -- 172.21.15.26:0/2558073311 wait complete. 2023-07-13T22:17:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.458+0000 7f847a19b700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 0x7f84600a3700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8470009b40 tx=0x7f8470009240 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.459+0000 7f848fd69b80 1 -- start start 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.459+0000 7f848fd69b80 1 Processor -- start 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 0x55af98600770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af984e0ac0 con 0x55af985ef4d0 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af984e07c0 con 0x55af985f0c70 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af983c4920 con 0x55af985efe20 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f848fd69b80 10 client.0 _ll_register_callbacks cb 0x55af984de760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f847999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f847999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57486/0 (socket says 172.21.15.26:57486) 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.460+0000 7f847999a700 1 -- 172.21.15.26:0/2244145953 learned_addr learned my addr 172.21.15.26:0/2244145953 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 0x55af98600770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a19b700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 -- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 msgr2=0x55af985f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 -- 172.21.15.26:0/2244145953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 msgr2=0x55af985f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a99c700 1 -- 172.21.15.26:0/2244145953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c009a40 con 0x55af985f0c70 2023-07-13T22:17:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.461+0000 7f847a19b700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:18.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:18 smithi026 ceph-mon[95647]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 92 op/s 2023-07-13T22:17:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.462+0000 7f847a99c700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 0x55af98600770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f846c009de0 tx=0x7f846c013f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d71b9f9150b8303 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.462+0000 7f845dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.462+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c014820 con 0x55af985f0c70 2023-07-13T22:17:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.462+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f846c013370 con 0x55af985f0c70 2023-07-13T22:17:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.462+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af9849a4f0 con 0x55af985f0c70 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.463+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f846c014980 con 0x55af985f0c70 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.463+0000 7f848fd69b80 10 client.25931 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.463+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55af98357c60 con 0x55af985f0c70 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.463+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af9849a4f0 con 0x55af985f0c70 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f846c025bf0 con 0x55af985f0c70 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 10 client.25931 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:17:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 20 client.25931 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98604f40 0x55af98607350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 4 client.25931 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af98539eb0 con 0x55af98604f40 2023-07-13T22:17:18.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.464+0000 7f847a19b700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98604f40 0x55af98607350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.465+0000 7f847a19b700 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98604f40 0x55af98607350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15a9253cd0f35281 server_cookie=9475fc1bca1f97ec in_seq=0 out_seq=0 2023-07-13T22:17:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.465+0000 7f845dffb700 10 client.25931 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.466+0000 7f845dffb700 1 -- 172.21.15.26:0/2244145953 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af98539eb0 con 0x55af98604f40 2023-07-13T22:17:18.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.466+0000 7f845dffb700 10 client.25931 handle_command_reply: tid=0 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.466+0000 7f848fd69b80 1 client.25931 shutdown 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 msgr2=0x55af98600770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 0x55af98600770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f846c009de0 tx=0x7f846c013f20 comp rx=0 tx=0).stop 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 shutdown_connections 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98604f40 0x55af98607350 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985f0c70 0x55af98600770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 --2- 172.21.15.26:0/2244145953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 >> 172.21.15.26:0/2244145953 conn(0x55af985e4da0 msgr2=0x55af985e98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f845dffb700 0 client.25931 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:18.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.467+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 shutdown_connections 2023-07-13T22:17:18.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.468+0000 7f848fd69b80 1 -- 172.21.15.26:0/2244145953 wait complete. 2023-07-13T22:17:18.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.483+0000 7f848fd69b80 20 client.25931 trim_cache size 0 max 16384 2023-07-13T22:17:18.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.543+0000 7f848fd69b80 1 Processor -- start 2023-07-13T22:17:18.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 -- start start 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985f0c70 0x55af985f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af985e4970 con 0x55af985ef4d0 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af986079b0 con 0x55af985efe20 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.544+0000 7f848fd69b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af985f6e50 con 0x55af985f0c70 2023-07-13T22:17:18.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40850/0 (socket says 172.21.15.26:40850) 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 -- 172.21.15.26:0/3291891276 learned_addr learned my addr 172.21.15.26:0/3291891276 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847a99c700 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985f0c70 0x55af985f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 -- 172.21.15.26:0/3291891276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985f0c70 msgr2=0x55af985f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985f0c70 0x55af985f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 -- 172.21.15.26:0/3291891276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 msgr2=0x55af985f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.545+0000 7f847999a700 1 -- 172.21.15.26:0/3291891276 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af98395250 con 0x55af985efe20 2023-07-13T22:17:18.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.546+0000 7f847999a700 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8464009550 tx=0x7f8464005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f418ae9d3080de8c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.562+0000 7f84693ff700 1 -- 172.21.15.26:0/3291891276 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846409ac80 con 0x55af985efe20 2023-07-13T22:17:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.562+0000 7f84693ff700 1 -- 172.21.15.26:0/3291891276 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f846409ecb0 con 0x55af985efe20 2023-07-13T22:17:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.562+0000 7f84693ff700 1 -- 172.21.15.26:0/3291891276 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846409a170 con 0x55af985efe20 2023-07-13T22:17:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.563+0000 7f848fd69b80 1 -- 172.21.15.26:0/3291891276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 msgr2=0x55af98600730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.563+0000 7f848fd69b80 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8464009550 tx=0x7f8464005790 comp rx=0 tx=0).stop 2023-07-13T22:17:18.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.563+0000 7f848fd69b80 1 -- 172.21.15.26:0/3291891276 shutdown_connections 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985ef4d0 0x55af985f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985f0c70 0x55af985f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 --2- 172.21.15.26:0/3291891276 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985efe20 0x55af98600730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 -- 172.21.15.26:0/3291891276 >> 172.21.15.26:0/3291891276 conn(0x55af985e4da0 msgr2=0x55af9860bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 -- 172.21.15.26:0/3291891276 shutdown_connections 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 -- 172.21.15.26:0/3291891276 wait complete. 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.564+0000 7f848fd69b80 1 -- start start 2023-07-13T22:17:18.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.565+0000 7f848fd69b80 1 Processor -- start 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.565+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.565+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985fca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.566+0000 7f848fd69b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.566+0000 7f848fd69b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af985fff90 con 0x55af985f0c70 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.566+0000 7f848fd69b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af98600100 con 0x55af985ef4d0 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.566+0000 7f848fd69b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af986002c0 con 0x55af985efe20 2023-07-13T22:17:18.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.566+0000 7f848fd69b80 10 client.0 _ll_register_callbacks cb 0x7f84700090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:18.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.569+0000 7f847a99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.569+0000 7f847a99c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40072/0 (socket says 172.21.15.26:40072) 2023-07-13T22:17:18.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.569+0000 7f847a99c700 1 -- 172.21.15.26:0/4126047326 learned_addr learned my addr 172.21.15.26:0/4126047326 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:18.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.573+0000 7f847999a700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.573+0000 7f847a19b700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.574+0000 7f847999a700 1 -- 172.21.15.26:0/4126047326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 msgr2=0x55af985fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.574+0000 7f847999a700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.574+0000 7f847999a700 1 -- 172.21.15.26:0/4126047326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 msgr2=0x55af985ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.574+0000 7f847999a700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.574+0000 7f847999a700 1 -- 172.21.15.26:0/4126047326 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8474104ff0 con 0x55af985efe20 2023-07-13T22:17:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.575+0000 7f847999a700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985fca30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84640095c0 tx=0x7f8464009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5943b5e8f3aa3283 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.575+0000 7f845dffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.578+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846409a370 con 0x55af985efe20 2023-07-13T22:17:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.578+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f846409ac80 con 0x55af985efe20 2023-07-13T22:17:18.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.579+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f846c00f050 con 0x55af985efe20 2023-07-13T22:17:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.599+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f846409b160 con 0x55af985efe20 2023-07-13T22:17:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.600+0000 7f848fd69b80 10 client.25927 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.600+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55af9836d2b0 con 0x55af985efe20 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84640c9080 con 0x55af985efe20 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8464099e50 con 0x55af985efe20 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 10 client.25927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 20 client.25927 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:18.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98659830 0x55af9865bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:18.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 4 client.25927 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:18.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.611+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af984fc050 con 0x55af98659830 2023-07-13T22:17:18.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.623+0000 7f847a19b700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98659830 0x55af9865bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:18.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.624+0000 7f847a19b700 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98659830 0x55af9865bc10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d40aa01e3b054e4f server_cookie=1819e6f3b8abf8e8 in_seq=0 out_seq=0 2023-07-13T22:17:18.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.633+0000 7f845dffb700 10 client.25927 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:18.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.633+0000 7f845dffb700 1 -- 172.21.15.26:0/4126047326 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af984fc050 con 0x55af98659830 2023-07-13T22:17:18.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.633+0000 7f845dffb700 10 client.25927 handle_command_reply: tid=0 2023-07-13T22:17:18.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.633+0000 7f848fd69b80 1 client.25927 shutdown 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 msgr2=0x55af985fca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985fca30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84640095c0 tx=0x7f8464009310 comp rx=0 tx=0).stop 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 shutdown_connections 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55af98659830 0x55af9865bc10 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af985f0c70 0x55af985ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af985ef4d0 0x55af985fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 --2- 172.21.15.26:0/4126047326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af985efe20 0x55af985fca30 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f84640095c0 tx=0x7f8464009310 comp rx=0 tx=0).stop 2023-07-13T22:17:18.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.644+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 >> 172.21.15.26:0/4126047326 conn(0x55af985e4da0 msgr2=0x55af985f7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.645+0000 7f845dffb700 0 client.25927 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:18.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.645+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 shutdown_connections 2023-07-13T22:17:18.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.645+0000 7f848fd69b80 1 -- 172.21.15.26:0/4126047326 wait complete. 2023-07-13T22:17:18.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.656+0000 7f848fd69b80 20 client.25927 trim_cache size 0 max 16384 2023-07-13T22:17:18.644 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.656+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 msgr2=0x7f84600a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.656+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 0x7f84600a3700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8470009b40 tx=0x7f8470009240 comp rx=0 tx=0).stop 2023-07-13T22:17:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.656+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 msgr2=0x7f847406f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:18.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8464005910 tx=0x7f846400f740 comp rx=0 tx=0).stop 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 shutdown_connections 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84600a1250 0x7f84600a3700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84740fe350 0x7f847406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8474103670 0x7f84741ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 --2- 172.21.15.26:0/2725775467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8474100cb0 0x7f847406f2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 >> 172.21.15.26:0/2725775467 conn(0x7f84740fa000 msgr2=0x7f84740fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:18.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 shutdown_connections 2023-07-13T22:17:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:18.657+0000 7f847c3ff700 1 -- 172.21.15.26:0/2725775467 wait complete. 2023-07-13T22:17:18.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:17:18.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:18 smithi117 ceph-mon[129074]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 92 op/s 2023-07-13T22:17:18.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:18 smithi121 ceph-mon[129187]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 92 op/s 2023-07-13T22:17:19.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.331+0000 7f1288fff700 1 -- 172.21.15.26:0/453426666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12841016d0 msgr2=0x7f1284107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.331+0000 7f1288fff700 1 --2- 172.21.15.26:0/453426666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12841016d0 0x7f1284107f90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f127c01c0d0 tx=0x7f127c01c3e0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 -- 172.21.15.26:0/453426666 shutdown_connections 2023-07-13T22:17:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 --2- 172.21.15.26:0/453426666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12841016d0 0x7f1284107f90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 --2- 172.21.15.26:0/453426666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12840ffb50 0x7f12840fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 --2- 172.21.15.26:0/453426666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1284100d40 0x7f1284101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 -- 172.21.15.26:0/453426666 >> 172.21.15.26:0/453426666 conn(0x7f12840fb170 msgr2=0x7f12840fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 -- 172.21.15.26:0/453426666 shutdown_connections 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.335+0000 7f1288fff700 1 -- 172.21.15.26:0/453426666 wait complete. 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 Processor -- start 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 -- start start 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 0x7f1284070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 0x7f1284071160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12841016d0 0x7f12840721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1284072770 con 0x7f12840ffb50 2023-07-13T22:17:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840728e0 con 0x7f1284100d40 2023-07-13T22:17:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.339+0000 7f1288fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1284072aa0 con 0x7f12841016d0 2023-07-13T22:17:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.340+0000 7f1282d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12841016d0 0x7f12840721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.340+0000 7f128259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 0x7f1284070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.340+0000 7f128259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 0x7f1284070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40086/0 (socket says 172.21.15.26:40086) 2023-07-13T22:17:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.340+0000 7f128259c700 1 -- 172.21.15.26:0/642407711 learned_addr learned my addr 172.21.15.26:0/642407711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.340+0000 7f1281d9b700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 0x7f1284071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.342+0000 7f1281d9b700 1 -- 172.21.15.26:0/642407711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12841016d0 msgr2=0x7f12840721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.342+0000 7f1281d9b700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12841016d0 0x7f12840721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.342+0000 7f1281d9b700 1 -- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 msgr2=0x7f1284070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.342+0000 7f1281d9b700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 0x7f1284070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.342+0000 7f1281d9b700 1 -- 172.21.15.26:0/642407711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c01ad10 con 0x7f1284100d40 2023-07-13T22:17:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.343+0000 7f1281d9b700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 0x7f1284071160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f127400b450 tx=0x7f127400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.345+0000 7f12737fe700 1 -- 172.21.15.26:0/642407711 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274011830 con 0x7f1284100d40 2023-07-13T22:17:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.345+0000 7f12737fe700 1 -- 172.21.15.26:0/642407711 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1274011e50 con 0x7f1284100d40 2023-07-13T22:17:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.345+0000 7f12737fe700 1 -- 172.21.15.26:0/642407711 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127400f2e0 con 0x7f1284100d40 2023-07-13T22:17:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.346+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1284072d20 con 0x7f1284100d40 2023-07-13T22:17:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.346+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840731e0 con 0x7f1284100d40 2023-07-13T22:17:19.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.360+0000 7f129c9b7b80 1 Processor -- start 2023-07-13T22:17:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.361+0000 7f129c9b7b80 1 -- start start 2023-07-13T22:17:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.361+0000 7f12737fe700 1 -- 172.21.15.26:0/642407711 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f12740105a0 con 0x7f1284100d40 2023-07-13T22:17:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de6fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de6fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de5ffcb0 con 0x55b4de6fa4d0 2023-07-13T22:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de652a40 con 0x55b4de6fae20 2023-07-13T22:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.362+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de64bed0 con 0x55b4de6fbc70 2023-07-13T22:17:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.364+0000 7f12737fe700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 0x7f126c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.365+0000 7f12737fe700 1 -- 172.21.15.26:0/642407711 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12740cc2f0 con 0x7f1284100d40 2023-07-13T22:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.369+0000 7f128259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.369+0000 7f128259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40882/0 (socket says 172.21.15.26:40882) 2023-07-13T22:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.369+0000 7f128259c700 1 -- 172.21.15.26:0/3651249719 learned_addr learned my addr 172.21.15.26:0/3651249719 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.371+0000 7f1281d9b700 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de6fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.375+0000 7f128259c700 1 -- 172.21.15.26:0/3651249719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 msgr2=0x55b4de6fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.375+0000 7f128259c700 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de6fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.375+0000 7f128259c700 1 -- 172.21.15.26:0/3651249719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 msgr2=0x55b4de6fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.375+0000 7f128259c700 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de6fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.375+0000 7f128259c700 1 -- 172.21.15.26:0/3651249719 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4de479de0 con 0x55b4de6fae20 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.377+0000 7f128259c700 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f127800be20 tx=0x7f127800f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90750629a436523 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.378+0000 7f1270ff9700 1 -- 172.21.15.26:0/3651249719 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800fe50 con 0x55b4de6fae20 2023-07-13T22:17:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.378+0000 7f1270ff9700 1 -- 172.21.15.26:0/3651249719 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127801a920 con 0x55b4de6fae20 2023-07-13T22:17:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.378+0000 7f1270ff9700 1 -- 172.21.15.26:0/3651249719 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278011810 con 0x55b4de6fae20 2023-07-13T22:17:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.379+0000 7f129c9b7b80 1 -- 172.21.15.26:0/3651249719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 msgr2=0x55b4de6fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.379+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f127800be20 tx=0x7f127800f640 comp rx=0 tx=0).stop 2023-07-13T22:17:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f1282d9d700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 0x7f126c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f129c9b7b80 1 -- 172.21.15.26:0/3651249719 shutdown_connections 2023-07-13T22:17:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de6fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de6fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/3651249719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de6fb6e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.382+0000 7f129c9b7b80 1 -- 172.21.15.26:0/3651249719 >> 172.21.15.26:0/3651249719 conn(0x55b4de6efda0 msgr2=0x55b4de6f51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.383+0000 7f1282d9d700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 0x7f126c0a3800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f127c0062a0 tx=0x7f127c0061f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.384+0000 7f129c9b7b80 1 -- 172.21.15.26:0/3651249719 shutdown_connections 2023-07-13T22:17:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.384+0000 7f129c9b7b80 1 -- 172.21.15.26:0/3651249719 wait complete. 2023-07-13T22:17:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.385+0000 7f129c9b7b80 1 -- start start 2023-07-13T22:17:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.385+0000 7f129c9b7b80 1 Processor -- start 2023-07-13T22:17:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de704710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de70b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de5ebac0 con 0x55b4de6fa4d0 2023-07-13T22:17:19.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de5eb7c0 con 0x55b4de6fae20 2023-07-13T22:17:19.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de4cf920 con 0x55b4de6fbc70 2023-07-13T22:17:19.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.386+0000 7f129c9b7b80 10 client.0 _ll_register_callbacks cb 0x55b4de5e9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f1281d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de704710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40100/0 (socket says 172.21.15.26:40100) 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 -- 172.21.15.26:0/156129735 learned_addr learned my addr 172.21.15.26:0/156129735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f1282d9d700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de70b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 -- 172.21.15.26:0/156129735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 msgr2=0x55b4de70b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de70b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.387+0000 7f128259c700 1 -- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 msgr2=0x55b4de704710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.388+0000 7f128259c700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de704710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.388+0000 7f128259c700 1 -- 172.21.15.26:0/156129735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127800ba50 con 0x55b4de6fa4d0 2023-07-13T22:17:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.388+0000 7f1281d9b700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de704710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:17:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.388+0000 7f128259c700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f127800ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da57577fcf225d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.390+0000 7f1269ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.390+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127801a660 con 0x55b4de6fa4d0 2023-07-13T22:17:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.390+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1278011590 con 0x55b4de6fa4d0 2023-07-13T22:17:19.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.391+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4de5a54f0 con 0x55b4de6fa4d0 2023-07-13T22:17:19.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.391+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127801a7c0 con 0x55b4de6fa4d0 2023-07-13T22:17:19.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.391+0000 7f129c9b7b80 10 client.15756 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.391+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b4de462c60 con 0x55b4de6fa4d0 2023-07-13T22:17:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.392+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4de5a54f0 con 0x55b4de6fa4d0 2023-07-13T22:17:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.392+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1278022070 con 0x55b4de6fa4d0 2023-07-13T22:17:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 10 client.15756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:17:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 20 client.15756 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de70fd40 0x55b4de712150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 4 client.15756 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.393+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4de644eb0 con 0x55b4de70fd40 2023-07-13T22:17:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.395+0000 7f1281d9b700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de70fd40 0x55b4de712150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.397+0000 7f1269ffb700 10 client.15756 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.397+0000 7f1281d9b700 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de70fd40 0x55b4de712150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecf67e84a2022500 server_cookie=44a5980a14ac5de7 in_seq=0 out_seq=0 2023-07-13T22:17:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.397+0000 7f1269ffb700 1 -- 172.21.15.26:0/156129735 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b4de644eb0 con 0x55b4de70fd40 2023-07-13T22:17:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f1269ffb700 10 client.15756 handle_command_reply: tid=0 2023-07-13T22:17:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 client.15756 shutdown 2023-07-13T22:17:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 msgr2=0x55b4de7041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1278000c00 tx=0x7f127800ffc0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 shutdown_connections 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fa4d0 0x55b4de7041d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de70fd40 0x55b4de712150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de70b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/156129735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de704710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 >> 172.21.15.26:0/156129735 conn(0x55b4de6efda0 msgr2=0x55b4de6f48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.398+0000 7f1269ffb700 0 client.15756 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.399+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 shutdown_connections 2023-07-13T22:17:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.399+0000 7f129c9b7b80 1 -- 172.21.15.26:0/156129735 wait complete. 2023-07-13T22:17:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.400+0000 7f129c9b7b80 20 client.15756 trim_cache size 0 max 16384 2023-07-13T22:17:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.427+0000 7f129c9b7b80 1 Processor -- start 2023-07-13T22:17:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 -- start start 2023-07-13T22:17:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fae20 0x55b4de7020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de702600 0x55b4de6febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de703510 con 0x55b4de6fae20 2023-07-13T22:17:19.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de6fb220 con 0x55b4de6fbc70 2023-07-13T22:17:19.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.428+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de6f6d50 con 0x55b4de702600 2023-07-13T22:17:19.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.429+0000 7f1281d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.429+0000 7f1281d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40898/0 (socket says 172.21.15.26:40898) 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.429+0000 7f1281d9b700 1 -- 172.21.15.26:0/1434239873 learned_addr learned my addr 172.21.15.26:0/1434239873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.429+0000 7f1282d9d700 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de702600 0x55b4de6febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 -- 172.21.15.26:0/1434239873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de702600 msgr2=0x55b4de6febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de702600 0x55b4de6febd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 -- 172.21.15.26:0/1434239873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fae20 msgr2=0x55b4de7020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fae20 0x55b4de7020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 -- 172.21.15.26:0/1434239873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4de4a0250 con 0x55b4de6fbc70 2023-07-13T22:17:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1274095430 tx=0x7f127409c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55407686a186b7d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f12717fa700 1 -- 172.21.15.26:0/1434239873 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740b4070 con 0x55b4de6fbc70 2023-07-13T22:17:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f12717fa700 1 -- 172.21.15.26:0/1434239873 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12740996b0 con 0x55b4de6fbc70 2023-07-13T22:17:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.430+0000 7f12717fa700 1 -- 172.21.15.26:0/1434239873 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274097a50 con 0x55b4de6fbc70 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.431+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1434239873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 msgr2=0x55b4de70b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.431+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1274095430 tx=0x7f127409c040 comp rx=0 tx=0).stop 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1434239873 shutdown_connections 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de6fae20 0x55b4de7020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de702600 0x55b4de6febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1434239873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fbc70 0x55b4de70b710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1434239873 >> 172.21.15.26:0/1434239873 conn(0x55b4de6efda0 msgr2=0x55b4de70c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1434239873 shutdown_connections 2023-07-13T22:17:19.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.432+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1434239873 wait complete. 2023-07-13T22:17:19.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.433+0000 7f129c9b7b80 1 -- start start 2023-07-13T22:17:19.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.433+0000 7f129c9b7b80 1 Processor -- start 2023-07-13T22:17:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de70b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de707e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de702600 0x55b4de7083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de708910 con 0x55b4de702600 2023-07-13T22:17:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de708a80 con 0x55b4de6fae20 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4de763bf0 con 0x55b4de6fbc70 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.434+0000 7f129c9b7b80 10 client.0 _ll_register_callbacks cb 0x7f127800bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.435+0000 7f1281d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de707e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.435+0000 7f128259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de70b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.435+0000 7f128259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de70b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40912/0 (socket says 172.21.15.26:40912) 2023-07-13T22:17:19.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.435+0000 7f128259c700 1 -- 172.21.15.26:0/1117248457 learned_addr learned my addr 172.21.15.26:0/1117248457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.435+0000 7f1282d9d700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de702600 0x55b4de7083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.436+0000 7f1281d9b700 1 -- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 msgr2=0x55b4de70b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.436+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de70b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.436+0000 7f1281d9b700 1 -- 172.21.15.26:0/1117248457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de702600 msgr2=0x55b4de7083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.436+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de702600 0x55b4de7083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.436+0000 7f1281d9b700 1 -- 172.21.15.26:0/1117248457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840731e0 con 0x55b4de6fbc70 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.437+0000 7f1281d9b700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de707e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12740993b0 tx=0x7f1274004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d93ad9a1e88ab57 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.437+0000 7f1269ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.437+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740b4070 con 0x55b4de6fbc70 2023-07-13T22:17:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.438+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12740a1b00 con 0x55b4de6fbc70 2023-07-13T22:17:19.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.439+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127801a920 con 0x55b4de6fbc70 2023-07-13T22:17:19.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.443+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127409d6e0 con 0x55b4de6fbc70 2023-07-13T22:17:19.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.444+0000 7f129c9b7b80 10 client.25942 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:19.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.444+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b4de4782b0 con 0x55b4de6fbc70 2023-07-13T22:17:19.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.445+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127801a920 con 0x55b4de6fbc70 2023-07-13T22:17:19.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.445+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f12740b7360 con 0x55b4de6fbc70 2023-07-13T22:17:19.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 10 client.25942 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:19.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 10 client.25942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:17:19.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 20 client.25942 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:19.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de7652a0 0x55b4de7676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 4 client.25942 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4de607050 con 0x55b4de7652a0 2023-07-13T22:17:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.446+0000 7f128259c700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de7652a0 0x55b4de7676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.447+0000 7f128259c700 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de7652a0 0x55b4de7676b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91b004e62f657d77 server_cookie=16f8a5a6e0a4b07d in_seq=0 out_seq=0 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.447+0000 7f1269ffb700 10 client.25942 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.447+0000 7f1269ffb700 1 -- 172.21.15.26:0/1117248457 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4de607050 con 0x55b4de7652a0 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.447+0000 7f1269ffb700 10 client.25942 handle_command_reply: tid=0 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.447+0000 7f129c9b7b80 1 client.25942 shutdown 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 msgr2=0x55b4de707e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de707e90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12740993b0 tx=0x7f1274004750 comp rx=0 tx=0).stop 2023-07-13T22:17:19.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 shutdown_connections 2023-07-13T22:17:19.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b4de702600 0x55b4de7083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55b4de7652a0 0x55b4de7676b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b4de6fae20 0x55b4de70b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 --2- 172.21.15.26:0/1117248457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b4de6fbc70 0x55b4de707e90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 >> 172.21.15.26:0/1117248457 conn(0x55b4de6efda0 msgr2=0x55b4de6febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f1269ffb700 0 client.25942 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 shutdown_connections 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.448+0000 7f129c9b7b80 1 -- 172.21.15.26:0/1117248457 wait complete. 2023-07-13T22:17:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.449+0000 7f129c9b7b80 20 client.25942 trim_cache size 0 max 16384 2023-07-13T22:17:19.438 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:19.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 msgr2=0x7f126c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 0x7f126c0a3800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f127c0062a0 tx=0x7f127c0061f0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 msgr2=0x7f1284071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 0x7f1284071160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f127400b450 tx=0x7f127400b810 comp rx=0 tx=0).stop 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 shutdown_connections 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f126c0a1350 0x7f126c0a3800 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840ffb50 0x7f1284070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12841016d0 0x7f12840721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 --2- 172.21.15.26:0/642407711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1284100d40 0x7f1284071160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.450+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 >> 172.21.15.26:0/642407711 conn(0x7f12840fb170 msgr2=0x7f12840fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.451+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 shutdown_connections 2023-07-13T22:17:19.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.451+0000 7f1288fff700 1 -- 172.21.15.26:0/642407711 wait complete. 2023-07-13T22:17:19.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:17:19.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.907+0000 7f463739d700 1 -- 172.21.15.26:0/3552561828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630100ba0 msgr2=0x7f4630103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.907+0000 7f463739d700 1 --2- 172.21.15.26:0/3552561828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630100ba0 0x7f4630103020 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4620009990 tx=0x7f4620009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.909+0000 7f463739d700 1 -- 172.21.15.26:0/3552561828 shutdown_connections 2023-07-13T22:17:19.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.909+0000 7f463739d700 1 --2- 172.21.15.26:0/3552561828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630103560 0x7f46301039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.909+0000 7f463739d700 1 --2- 172.21.15.26:0/3552561828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f4630100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.909+0000 7f463739d700 1 --2- 172.21.15.26:0/3552561828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630100ba0 0x7f4630103020 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.909+0000 7f463739d700 1 -- 172.21.15.26:0/3552561828 >> 172.21.15.26:0/3552561828 conn(0x7f46300f9ed0 msgr2=0x7f46300fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.931+0000 7f463739d700 1 -- 172.21.15.26:0/3552561828 shutdown_connections 2023-07-13T22:17:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.931+0000 7f463739d700 1 -- 172.21.15.26:0/3552561828 wait complete. 2023-07-13T22:17:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.933+0000 7f463739d700 1 Processor -- start 2023-07-13T22:17:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.933+0000 7f463739d700 1 -- start start 2023-07-13T22:17:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100ba0 0x7f4630070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630103560 0x7f4630071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630071620 con 0x7f4630100ba0 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630071790 con 0x7f46300fe240 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463739d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630071950 con 0x7f4630103560 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40920/0 (socket says 172.21.15.26:40920) 2023-07-13T22:17:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.934+0000 7f463639b700 1 -- 172.21.15.26:0/852336801 learned_addr learned my addr 172.21.15.26:0/852336801 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 -- 172.21.15.26:0/852336801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630103560 msgr2=0x7f4630071050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:17:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630103560 0x7f4630071050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 -- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100ba0 msgr2=0x7f4630070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100ba0 0x7f4630070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 -- 172.21.15.26:0/852336801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200095f0 con 0x7f46300fe240 2023-07-13T22:17:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.935+0000 7f463639b700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e9a0 tx=0x7f462c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.952+0000 7f46277fe700 1 -- 172.21.15.26:0/852336801 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00cbd0 con 0x7f46300fe240 2023-07-13T22:17:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.952+0000 7f46277fe700 1 -- 172.21.15.26:0/852336801 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f462c00cd30 con 0x7f46300fe240 2023-07-13T22:17:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.952+0000 7f46277fe700 1 -- 172.21.15.26:0/852336801 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c018710 con 0x7f46300fe240 2023-07-13T22:17:19.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.957+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4630071bd0 con 0x7f46300fe240 2023-07-13T22:17:19.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.957+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4630072060 con 0x7f46300fe240 2023-07-13T22:17:19.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.964+0000 7f464bebab80 1 Processor -- start 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.968+0000 7f46277fe700 1 -- 172.21.15.26:0/852336801 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f462c010980 con 0x7f46300fe240 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.970+0000 7f46277fe700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 0x7f461c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.970+0000 7f46277fe700 1 -- 172.21.15.26:0/852336801 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f462c014070 con 0x7f46300fe240 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.971+0000 7f464bebab80 1 -- start start 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 0x55af0cca1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0cca28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cba6cb0 con 0x55af0cca1720 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cbf9a40 con 0x55af0cca2070 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.972+0000 7f464bebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cbf2ed0 con 0x55af0cca2e30 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.973+0000 7f463639b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 0x55af0cca1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.973+0000 7f463639b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 0x55af0cca1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40146/0 (socket says 172.21.15.26:40146) 2023-07-13T22:17:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.973+0000 7f463639b700 1 -- 172.21.15.26:0/3277823299 learned_addr learned my addr 172.21.15.26:0/3277823299 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.975+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0cca28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.975+0000 7f4636b9c700 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4636b9c700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 0x7f461c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4635b9a700 1 -- 172.21.15.26:0/3277823299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 msgr2=0x55af0cca32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4635b9a700 1 -- 172.21.15.26:0/3277823299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 msgr2=0x55af0cca1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 0x55af0cca1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.976+0000 7f4635b9a700 1 -- 172.21.15.26:0/3277823299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0ca20de0 con 0x55af0cca2070 2023-07-13T22:17:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.977+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0cca28a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4620009990 tx=0x7f4620005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e767d6c206ed1994 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.977+0000 7f4624bfe700 1 -- 172.21.15.26:0/3277823299 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462001d070 con 0x55af0cca2070 2023-07-13T22:17:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.977+0000 7f4624bfe700 1 -- 172.21.15.26:0/3277823299 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4620004000 con 0x55af0cca2070 2023-07-13T22:17:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.977+0000 7f4624bfe700 1 -- 172.21.15.26:0/3277823299 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000be00 con 0x55af0cca2070 2023-07-13T22:17:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.978+0000 7f464bebab80 1 -- 172.21.15.26:0/3277823299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 msgr2=0x55af0cca28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.978+0000 7f464bebab80 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0cca28a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4620009990 tx=0x7f4620005c20 comp rx=0 tx=0).stop 2023-07-13T22:17:19.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.980+0000 7f4636b9c700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 0x7f461c0a3890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4628009d90 tx=0x7f4628006d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.983+0000 7f464bebab80 1 -- 172.21.15.26:0/3277823299 shutdown_connections 2023-07-13T22:17:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.983+0000 7f464bebab80 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca1720 0x55af0cca1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.983+0000 7f464bebab80 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.983+0000 7f464bebab80 1 --2- 172.21.15.26:0/3277823299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0cca28a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.983+0000 7f464bebab80 1 -- 172.21.15.26:0/3277823299 >> 172.21.15.26:0/3277823299 conn(0x55af0cc96ff0 msgr2=0x55af0cc9c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.984+0000 7f464bebab80 1 -- 172.21.15.26:0/3277823299 shutdown_connections 2023-07-13T22:17:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.984+0000 7f464bebab80 1 -- 172.21.15.26:0/3277823299 wait complete. 2023-07-13T22:17:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.985+0000 7f464bebab80 1 -- start start 2023-07-13T22:17:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.985+0000 7f464bebab80 1 Processor -- start 2023-07-13T22:17:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca1720 0x55af0ccab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cb92ac0 con 0x55af0cca2e30 2023-07-13T22:17:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cb927c0 con 0x55af0cca2070 2023-07-13T22:17:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ca76920 con 0x55af0cca1720 2023-07-13T22:17:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.986+0000 7f464bebab80 10 client.0 _ll_register_callbacks cb 0x55af0cb90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:19.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.987+0000 7f4635b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.987+0000 7f4635b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40932/0 (socket says 172.21.15.26:40932) 2023-07-13T22:17:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.987+0000 7f4635b9a700 1 -- 172.21.15.26:0/3591592897 learned_addr learned my addr 172.21.15.26:0/3591592897 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.987+0000 7f463639b700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca1720 0x55af0ccab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:19.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.988+0000 7f4635b9a700 1 -- 172.21.15.26:0/3591592897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca1720 msgr2=0x55af0ccab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.988+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca1720 0x55af0ccab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.988+0000 7f4635b9a700 1 -- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 msgr2=0x55af0ccb2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.988+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.988+0000 7f4635b9a700 1 -- 172.21.15.26:0/3591592897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4630104500 con 0x55af0cca2070 2023-07-13T22:17:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.989+0000 7f4635b9a700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4620005cd0 tx=0x7f46200092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=435dca39f7996e77 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.989+0000 7f4619ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.990+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462001d040 con 0x55af0cca2070 2023-07-13T22:17:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.990+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4620003b90 con 0x55af0cca2070 2023-07-13T22:17:19.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.990+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af0cb4c4f0 con 0x55af0cca2070 2023-07-13T22:17:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.992+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4620003d50 con 0x55af0cca2070 2023-07-13T22:17:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.994+0000 7f464bebab80 10 client.25967 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.994+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55af0ca09c60 con 0x55af0cca2070 2023-07-13T22:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.995+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af0cb4c4f0 con 0x55af0cca2070 2023-07-13T22:17:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.995+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4620029030 con 0x55af0cca2070 2023-07-13T22:17:19.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 10 client.25967 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:19.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 10 client.25967 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:17:19.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 20 client.25967 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:19.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0ccb6b00 0x55af0ccb8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:19.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 4 client.25967 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:19.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:19.996+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af0cbebeb0 con 0x55af0ccb6b00 2023-07-13T22:17:19.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.011+0000 7f463639b700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0ccb6b00 0x55af0ccb8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.017+0000 7f463639b700 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0ccb6b00 0x55af0ccb8f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7889c595d7f0a33f server_cookie=e60598dedadbc9e5 in_seq=0 out_seq=0 2023-07-13T22:17:20.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.017+0000 7f4619ffb700 10 client.25967 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:20.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.020+0000 7f4619ffb700 1 -- 172.21.15.26:0/3591592897 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af0cbebeb0 con 0x55af0ccb6b00 2023-07-13T22:17:20.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.020+0000 7f4619ffb700 10 client.25967 handle_command_reply: tid=0 2023-07-13T22:17:20.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.020+0000 7f464bebab80 1 client.25967 shutdown 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 msgr2=0x55af0ccab7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4620005cd0 tx=0x7f46200092f0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 shutdown_connections 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0ccb6b00 0x55af0ccb8f10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca1720 0x55af0ccab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 --2- 172.21.15.26:0/3591592897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca2070 0x55af0ccab7e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 >> 172.21.15.26:0/3591592897 conn(0x55af0cc96ff0 msgr2=0x55af0cc9bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f4619ffb700 0 client.25967 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:20.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.021+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 shutdown_connections 2023-07-13T22:17:20.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.022+0000 7f464bebab80 1 -- 172.21.15.26:0/3591592897 wait complete. 2023-07-13T22:17:20.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.024+0000 7f464bebab80 20 client.25967 trim_cache size 0 max 16384 2023-07-13T22:17:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.054+0000 7f464bebab80 1 Processor -- start 2023-07-13T22:17:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.054+0000 7f464bebab80 1 -- start start 2023-07-13T22:17:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccb3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2070 0x55af0ccb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cca2440 con 0x55af0cca2070 2023-07-13T22:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0cca9a50 con 0x55af0cca1720 2023-07-13T22:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.055+0000 7f464bebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ccabf20 con 0x55af0cca2e30 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f463639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccb3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f463639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccb3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40946/0 (socket says 172.21.15.26:40946) 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f463639b700 1 -- 172.21.15.26:0/1759689427 learned_addr learned my addr 172.21.15.26:0/1759689427 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 -- 172.21.15.26:0/1759689427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 msgr2=0x55af0ccb3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccb3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 -- 172.21.15.26:0/1759689427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2070 msgr2=0x55af0ccb27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2070 0x55af0ccb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.056+0000 7f4636b9c700 1 -- 172.21.15.26:0/1759689427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0ca47250 con 0x55af0cca2e30 2023-07-13T22:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.057+0000 7f4636b9c700 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca5cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f462800e770 tx=0x7f4628004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a49a1cb47478db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.057+0000 7f46253ff700 1 -- 172.21.15.26:0/1759689427 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462801d070 con 0x55af0cca2e30 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.057+0000 7f46253ff700 1 -- 172.21.15.26:0/1759689427 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4628009e70 con 0x55af0cca2e30 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.057+0000 7f46253ff700 1 -- 172.21.15.26:0/1759689427 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628021ca0 con 0x55af0cca2e30 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 -- 172.21.15.26:0/1759689427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 msgr2=0x55af0cca5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca5cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f462800e770 tx=0x7f4628004ad0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 -- 172.21.15.26:0/1759689427 shutdown_connections 2023-07-13T22:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2e30 0x55af0cca5cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccb3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 --2- 172.21.15.26:0/1759689427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2070 0x55af0ccb27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.058+0000 7f464bebab80 1 -- 172.21.15.26:0/1759689427 >> 172.21.15.26:0/1759689427 conn(0x55af0cc96ff0 msgr2=0x55af0cca9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.059+0000 7f464bebab80 1 -- 172.21.15.26:0/1759689427 shutdown_connections 2023-07-13T22:17:20.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.059+0000 7f464bebab80 1 -- 172.21.15.26:0/1759689427 wait complete. 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.059+0000 7f464bebab80 1 -- start start 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.060+0000 7f464bebab80 1 Processor -- start 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.060+0000 7f464bebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.060+0000 7f464bebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 0x55af0ccae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.061+0000 7f464bebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.061+0000 7f464bebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ccb1e40 con 0x55af0cca2e30 2023-07-13T22:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.061+0000 7f464bebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ccb1fb0 con 0x55af0cca1720 2023-07-13T22:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.061+0000 7f464bebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0ccb2170 con 0x55af0cca2070 2023-07-13T22:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.061+0000 7f464bebab80 10 client.0 _ll_register_callbacks cb 0x7f462c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.064+0000 7f4635b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 0x55af0ccae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.064+0000 7f4635b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 0x55af0ccae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:57640/0 (socket says 172.21.15.26:57640) 2023-07-13T22:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.064+0000 7f4635b9a700 1 -- 172.21.15.26:0/4166800669 learned_addr learned my addr 172.21.15.26:0/4166800669 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.064+0000 7f463639b700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.064+0000 7f4636b9c700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.065+0000 7f4636b9c700 1 -- 172.21.15.26:0/4166800669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 msgr2=0x55af0ccae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.066+0000 7f4636b9c700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 0x55af0ccae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.066+0000 7f4636b9c700 1 -- 172.21.15.26:0/4166800669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 msgr2=0x55af0ccae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.066+0000 7f4636b9c700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.066+0000 7f4636b9c700 1 -- 172.21.15.26:0/4166800669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462800e920 con 0x55af0cca2e30 2023-07-13T22:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.067+0000 7f4636b9c700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb1900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46280218f0 tx=0x7f4628021c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6508519276273bba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.067+0000 7f4619ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.067+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462801d050 con 0x55af0cca2e30 2023-07-13T22:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.068+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4628012040 con 0x55af0cca2e30 2023-07-13T22:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.068+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4620004000 con 0x55af0cca2e30 2023-07-13T22:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.068+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4628006a60 con 0x55af0cca2e30 2023-07-13T22:17:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.068+0000 7f464bebab80 10 client.15765 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.069+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55af0ca1f2b0 con 0x55af0cca2e30 2023-07-13T22:17:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.069+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4620004000 con 0x55af0cca2e30 2023-07-13T22:17:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.069+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f462802a5c0 con 0x55af0cca2e30 2023-07-13T22:17:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 10 client.15765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:17:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 20 client.15765 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0cd0b660 0x55af0cd0da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 4 client.15765 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.071+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55af0cbae050 con 0x55af0cd0b660 2023-07-13T22:17:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.072+0000 7f463639b700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0cd0b660 0x55af0cd0da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.073+0000 7f463639b700 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0cd0b660 0x55af0cd0da70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c82ab796b84e72d0 server_cookie=dd52f0e56b42865a in_seq=0 out_seq=0 2023-07-13T22:17:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.073+0000 7f4619ffb700 10 client.15765 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.074+0000 7f4619ffb700 1 -- 172.21.15.26:0/4166800669 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55af0cbae050 con 0x55af0cd0b660 2023-07-13T22:17:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.074+0000 7f4619ffb700 10 client.15765 handle_command_reply: tid=0 2023-07-13T22:17:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.074+0000 7f464bebab80 1 client.15765 shutdown 2023-07-13T22:17:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 msgr2=0x55af0ccb1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb1900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46280218f0 tx=0x7f4628021c30 comp rx=0 tx=0).stop 2023-07-13T22:17:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 shutdown_connections 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55af0cd0b660 0x55af0cd0da70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55af0cca2e30 0x55af0ccb1900 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55af0cca1720 0x55af0ccae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 --2- 172.21.15.26:0/4166800669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0cca2070 0x55af0ccae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 >> 172.21.15.26:0/4166800669 conn(0x55af0cc96ff0 msgr2=0x55af0ccb3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.075+0000 7f4619ffb700 0 client.15765 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.076+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 shutdown_connections 2023-07-13T22:17:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.076+0000 7f464bebab80 1 -- 172.21.15.26:0/4166800669 wait complete. 2023-07-13T22:17:20.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.077+0000 7f464bebab80 20 client.15765 trim_cache size 0 max 16384 2023-07-13T22:17:20.064 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 msgr2=0x7f461c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 0x7f461c0a3890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4628009d90 tx=0x7f4628006d00 comp rx=0 tx=0).stop 2023-07-13T22:17:20.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 msgr2=0x7f463006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e9a0 tx=0x7f462c00ed60 comp rx=0 tx=0).stop 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 shutdown_connections 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4630103560 0x7f4630071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46300fe240 0x7f463006fad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f461c0a13e0 0x7f461c0a3890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 --2- 172.21.15.26:0/852336801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4630100ba0 0x7f4630070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.080+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 >> 172.21.15.26:0/852336801 conn(0x7f46300f9ed0 msgr2=0x7f46300fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.081+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 shutdown_connections 2023-07-13T22:17:20.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.081+0000 7f463739d700 1 -- 172.21.15.26:0/852336801 wait complete. 2023-07-13T22:17:20.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:17:20.794 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:20 smithi026 ceph-mon[95647]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 95 op/s 2023-07-13T22:17:20.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.880+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/2167577372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 msgr2=0x7f2f38106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.880+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/2167577372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f38106e80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000b0a0 tx=0x7f2f3000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.889+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/2167577372 shutdown_connections 2023-07-13T22:17:20.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.889+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/2167577372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f38106e80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.890+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/2167577372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f380fea40 0x7f2f380fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.890+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/2167577372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380ffc30 0x7f2f38100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.890+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/2167577372 >> 172.21.15.26:0/2167577372 conn(0x7f2f380fa060 msgr2=0x7f2f380fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.911+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/2167577372 shutdown_connections 2023-07-13T22:17:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.923+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/2167577372 wait complete. 2023-07-13T22:17:20.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.925+0000 7f2f3e5ff700 1 Processor -- start 2023-07-13T22:17:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.925+0000 7f2f3e5ff700 1 -- start start 2023-07-13T22:17:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380fea40 0x7f2f3819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f380ffc30 0x7f2f3819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381a3ab0 con 0x7f2f380fea40 2023-07-13T22:17:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f38069f60 con 0x7f2f381005c0 2023-07-13T22:17:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.926+0000 7f2f3e5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3806a120 con 0x7f2f380ffc30 2023-07-13T22:17:20.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.928+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.929+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40972/0 (socket says 172.21.15.26:40972) 2023-07-13T22:17:20.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.929+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/786110615 learned_addr learned my addr 172.21.15.26:0/786110615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f37fff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380fea40 0x7f2f3819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/786110615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f380ffc30 msgr2=0x7f2f3819c5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:17:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f380ffc30 0x7f2f3819c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380fea40 msgr2=0x7f2f3819c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380fea40 0x7f2f3819c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.934+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/786110615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30009d00 con 0x7f2f381005c0 2023-07-13T22:17:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.939+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000b070 tx=0x7f2f30004d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.940+0000 7f2f35ffb700 1 -- 172.21.15.26:0/786110615 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000e040 con 0x7f2f381005c0 2023-07-13T22:17:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.941+0000 7f2f35ffb700 1 -- 172.21.15.26:0/786110615 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f3001bd20 con 0x7f2f381005c0 2023-07-13T22:17:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.941+0000 7f2f35ffb700 1 -- 172.21.15.26:0/786110615 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300059b0 con 0x7f2f381005c0 2023-07-13T22:17:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.941+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3806a3a0 con 0x7f2f381005c0 2023-07-13T22:17:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.941+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3806a8e0 con 0x7f2f381005c0 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.951+0000 7f2f51fecb80 1 Processor -- start 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.954+0000 7f2f51fecb80 1 -- start start 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb6c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb6d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cba70cb0 con 0x5639cbb6cc70 2023-07-13T22:17:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbac3a40 con 0x5639cbb6b4d0 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbabced0 con 0x5639cbb6be20 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.955+0000 7f2f2f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.956+0000 7f2f2f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40976/0 (socket says 172.21.15.26:40976) 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.956+0000 7f2f2f7fe700 1 -- 172.21.15.26:0/156952701 learned_addr learned my addr 172.21.15.26:0/156952701 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.959+0000 7f2f37fff700 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb6c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.961+0000 7f2f2f7fe700 1 -- 172.21.15.26:0/156952701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 msgr2=0x5639cbb6c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.961+0000 7f2f2f7fe700 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb6c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.961+0000 7f2f2f7fe700 1 -- 172.21.15.26:0/156952701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 msgr2=0x5639cbb6d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.961+0000 7f2f2f7fe700 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb6d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.961+0000 7f2f2f7fe700 1 -- 172.21.15.26:0/156952701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639cb8eade0 con 0x5639cbb6b4d0 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.962+0000 7f2f2f7fe700 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f200017c0 tx=0x7f2f20001ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eafc704b1a82c1c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.964+0000 7f2f35ffb700 1 -- 172.21.15.26:0/786110615 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2f30005040 con 0x7f2f381005c0 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.965+0000 7f2f2e7fc700 1 -- 172.21.15.26:0/156952701 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20004c70 con 0x5639cbb6b4d0 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.965+0000 7f2f2e7fc700 1 -- 172.21.15.26:0/156952701 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f20004dd0 con 0x5639cbb6b4d0 2023-07-13T22:17:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.965+0000 7f2f2e7fc700 1 -- 172.21.15.26:0/156952701 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20009280 con 0x5639cbb6b4d0 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 -- 172.21.15.26:0/156952701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 msgr2=0x5639cbb6b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f200017c0 tx=0x7f2f20001ad0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 -- 172.21.15.26:0/156952701 shutdown_connections 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb6d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb6b8e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/156952701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb6c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.966+0000 7f2f51fecb80 1 -- 172.21.15.26:0/156952701 >> 172.21.15.26:0/156952701 conn(0x5639cbb60da0 msgr2=0x5639cbb661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.967+0000 7f2f35ffb700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 0x7f2f240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.967+0000 7f2f35ffb700 1 -- 172.21.15.26:0/786110615 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f300ce550 con 0x7f2f381005c0 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.968+0000 7f2f51fecb80 1 -- 172.21.15.26:0/156952701 shutdown_connections 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.968+0000 7f2f51fecb80 1 -- 172.21.15.26:0/156952701 wait complete. 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.969+0000 7f2f51fecb80 1 -- start start 2023-07-13T22:17:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.970+0000 7f2f51fecb80 1 Processor -- start 2023-07-13T22:17:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.973+0000 7f2f37fff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 0x7f2f240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb75700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cba5cac0 con 0x5639cbb6be20 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cba5c7c0 con 0x5639cbb6cc70 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cb940920 con 0x5639cbb6b4d0 2023-07-13T22:17:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.974+0000 7f2f51fecb80 10 client.0 _ll_register_callbacks cb 0x5639cba5a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.983+0000 7f2f37fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.983+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.983+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40992/0 (socket says 172.21.15.26:40992) 2023-07-13T22:17:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.983+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3500422411 learned_addr learned my addr 172.21.15.26:0/3500422411 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3500422411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 msgr2=0x5639cbb751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3500422411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 msgr2=0x5639cbb75700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb75700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3500422411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f20001470 con 0x5639cbb6cc70 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f30097ea0 tx=0x7f2f3009b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ffe1afe0744ce02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3001be90 con 0x5639cbb6cc70 2023-07-13T22:17:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.984+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f3009b9a0 con 0x5639cbb6cc70 2023-07-13T22:17:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.985+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639cba164f0 con 0x5639cbb6cc70 2023-07-13T22:17:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.985+0000 7f2f37fff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 0x7f2f240a3710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800bfd0 tx=0x7f2f2800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.988+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3009f9d0 con 0x5639cbb6cc70 2023-07-13T22:17:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.995+0000 7f2f51fecb80 10 client.25982 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.995+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639cb8d3c60 con 0x5639cbb6cc70 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.995+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f300c7080 con 0x5639cbb6cc70 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.995+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2f3009af00 con 0x5639cbb6cc70 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 10 client.25982 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 20 client.25982 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbb91d80 0x5639cbb94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 4 client.25982 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.996+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639cbab5eb0 con 0x5639cbb91d80 2023-07-13T22:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:20.997+0000 7f2f37fff700 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbb91d80 0x5639cbb94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.011+0000 7f2f37fff700 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbb91d80 0x5639cbb94190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48deb28705f0d50 server_cookie=3f046c7a7acaced9 in_seq=0 out_seq=0 2023-07-13T22:17:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.011+0000 7f2f177fe700 10 client.25982 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.012+0000 7f2f177fe700 1 -- 172.21.15.26:0/3500422411 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5639cbab5eb0 con 0x5639cbb91d80 2023-07-13T22:17:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.012+0000 7f2f177fe700 10 client.25982 handle_command_reply: tid=0 2023-07-13T22:17:20.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.012+0000 7f2f51fecb80 1 client.25982 shutdown 2023-07-13T22:17:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 msgr2=0x5639cbb7c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f30097ea0 tx=0x7f2f3009b7f0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 shutdown_connections 2023-07-13T22:17:21.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbb91d80 0x5639cbb94190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb7c740 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3500422411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb75700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.013+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 >> 172.21.15.26:0/3500422411 conn(0x5639cbb60da0 msgr2=0x5639cbb658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:21.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.014+0000 7f2f177fe700 0 client.25982 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.014+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 shutdown_connections 2023-07-13T22:17:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.014+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3500422411 wait complete. 2023-07-13T22:17:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.015+0000 7f2f51fecb80 20 client.25982 trim_cache size 0 max 16384 2023-07-13T22:17:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.046+0000 7f2f51fecb80 1 Processor -- start 2023-07-13T22:17:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.046+0000 7f2f51fecb80 1 -- start start 2023-07-13T22:17:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb93d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb7c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb609c0 con 0x5639cbb6be20 2023-07-13T22:17:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb62ec0 con 0x5639cbb6cc70 2023-07-13T22:17:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.047+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb760c0 con 0x5639cbb6b4d0 2023-07-13T22:17:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.048+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.048+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40996/0 (socket says 172.21.15.26:40996) 2023-07-13T22:17:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.048+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3516563423 learned_addr learned my addr 172.21.15.26:0/3516563423 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.048+0000 7f2f2f7fe700 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb7c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.049+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3516563423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 msgr2=0x5639cbb93d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.049+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb93d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.049+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3516563423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 msgr2=0x5639cbb7c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.049+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb7c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.049+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/3516563423 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639cbb6d310 con 0x5639cbb6cc70 2023-07-13T22:17:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.060+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009bb0 tx=0x7f2f30005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4382adcbe2b6ab2b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.061+0000 7f2f2effd700 1 -- 172.21.15.26:0/3516563423 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3001be90 con 0x5639cbb6cc70 2023-07-13T22:17:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.061+0000 7f2f2effd700 1 -- 172.21.15.26:0/3516563423 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639cba164f0 con 0x5639cbb6cc70 2023-07-13T22:17:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.061+0000 7f2f2effd700 1 -- 172.21.15.26:0/3516563423 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3009ae90 con 0x5639cbb6cc70 2023-07-13T22:17:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.062+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3516563423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 msgr2=0x5639cbb73ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:21.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.062+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009bb0 tx=0x7f2f30005b90 comp rx=0 tx=0).stop 2023-07-13T22:17:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3516563423 shutdown_connections 2023-07-13T22:17:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6cc70 0x5639cbb73ee0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6b4d0 0x5639cbb93d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/3516563423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6be20 0x5639cbb7c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3516563423 >> 172.21.15.26:0/3516563423 conn(0x5639cbb60da0 msgr2=0x5639cbb8e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3516563423 shutdown_connections 2023-07-13T22:17:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.071+0000 7f2f51fecb80 1 -- 172.21.15.26:0/3516563423 wait complete. 2023-07-13T22:17:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.072+0000 7f2f51fecb80 1 -- start start 2023-07-13T22:17:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.072+0000 7f2f51fecb80 1 Processor -- start 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb7c120 con 0x5639cbb6cc70 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb7c290 con 0x5639cbb6b4d0 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639cbb7c450 con 0x5639cbb6be20 2023-07-13T22:17:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.073+0000 7f2f51fecb80 10 client.0 _ll_register_callbacks cb 0x5639cba5a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.078+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.078+0000 7f2f3cb9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40228/0 (socket says 172.21.15.26:40228) 2023-07-13T22:17:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.078+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/355405903 learned_addr learned my addr 172.21.15.26:0/355405903 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.097+0000 7f2f37fff700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.097+0000 7f2f2f7fe700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb78dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.103+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/355405903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 msgr2=0x5639cbb78dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.103+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb78dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.103+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 msgr2=0x5639cbb78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.103+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.103+0000 7f2f3cb9c700 1 -- 172.21.15.26:0/355405903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639cb8d3c60 con 0x5639cbb6cc70 2023-07-13T22:17:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.152+0000 7f2f3cb9c700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f3009b6a0 tx=0x7f2f3009f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b1fb0e50ee75b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:21.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.161+0000 7f2f177fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.178+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3001be90 con 0x5639cbb6cc70 2023-07-13T22:17:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.178+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f3009b9a0 con 0x5639cbb6cc70 2023-07-13T22:17:21.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.178+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f20004dd0 con 0x5639cbb6cc70 2023-07-13T22:17:21.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.189+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3001ba70 con 0x5639cbb6cc70 2023-07-13T22:17:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.189+0000 7f2f51fecb80 10 client.15774 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:21.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.189+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639cb911250 con 0x5639cbb6cc70 2023-07-13T22:17:21.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:20 smithi117 ceph-mon[129074]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 95 op/s 2023-07-13T22:17:21.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.207+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f300c2020 con 0x5639cbb6cc70 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.207+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2f3009ab20 con 0x5639cbb6cc70 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.207+0000 7f2f51fecb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.207+0000 7f2f51fecb80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.208+0000 7f2f51fecb80 20 client.15774 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.208+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbbe6600 0x5639cbbe89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.208+0000 7f2f51fecb80 4 client.15774 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.208+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639cba78050 con 0x5639cbbe6600 2023-07-13T22:17:21.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.208+0000 7f2f37fff700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbbe6600 0x5639cbbe89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:21.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.213+0000 7f2f177fe700 10 client.15774 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:21.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.214+0000 7f2f37fff700 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbbe6600 0x5639cbbe89e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459c30a80c0d5690 server_cookie=2f892d76efebbd75 in_seq=0 out_seq=0 2023-07-13T22:17:21.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.225+0000 7f2f177fe700 1 -- 172.21.15.26:0/355405903 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639cba78050 con 0x5639cbbe6600 2023-07-13T22:17:21.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.225+0000 7f2f177fe700 10 client.15774 handle_command_reply: tid=0 2023-07-13T22:17:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.227+0000 7f2f51fecb80 1 client.15774 shutdown 2023-07-13T22:17:21.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.228+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 msgr2=0x5639cbb7bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:21.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.228+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f3009b6a0 tx=0x7f2f3009f9d0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 shutdown_connections 2023-07-13T22:17:21.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5639cbb6cc70 0x5639cbb7bbe0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5639cbbe6600 0x5639cbbe89e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5639cbb6b4d0 0x5639cbb78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 --2- 172.21.15.26:0/355405903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5639cbb6be20 0x5639cbb78dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.235+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 >> 172.21.15.26:0/355405903 conn(0x5639cbb60da0 msgr2=0x5639cbb93d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.240+0000 7f2f177fe700 0 client.15774 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.240+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 shutdown_connections 2023-07-13T22:17:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.240+0000 7f2f51fecb80 1 -- 172.21.15.26:0/355405903 wait complete. 2023-07-13T22:17:21.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.241+0000 7f2f51fecb80 20 client.15774 trim_cache size 0 max 16384 2023-07-13T22:17:21.229 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.243+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 msgr2=0x7f2f240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:21.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.243+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 0x7f2f240a3710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800bfd0 tx=0x7f2f2800b410 comp rx=0 tx=0).stop 2023-07-13T22:17:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.243+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 msgr2=0x7f2f381a34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:21.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.244+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000b070 tx=0x7f2f30004d30 comp rx=0 tx=0).stop 2023-07-13T22:17:21.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:20 smithi121 ceph-mon[129187]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 95 op/s 2023-07-13T22:17:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 shutdown_connections 2023-07-13T22:17:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f240a1260 0x7f2f240a3710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f380fea40 0x7f2f3819c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f381005c0 0x7f2f381a34e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 --2- 172.21.15.26:0/786110615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2f380ffc30 0x7f2f3819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.261+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 >> 172.21.15.26:0/786110615 conn(0x7f2f380fa060 msgr2=0x7f2f3810bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.266+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 shutdown_connections 2023-07-13T22:17:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:21.266+0000 7f2f3e5ff700 1 -- 172.21.15.26:0/786110615 wait complete. 2023-07-13T22:17:22.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:17:22.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:17:22.858 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:22 smithi026 ceph-mon[95647]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 277 MiB/s wr, 87 op/s 2023-07-13T22:17:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.977+0000 7f43f37ff700 1 -- 172.21.15.26:0/3300870244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec102ef0 msgr2=0x7f43ec103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.977+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3300870244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec102ef0 0x7f43ec103340 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009990 tx=0x7f43e8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 -- 172.21.15.26:0/3300870244 shutdown_connections 2023-07-13T22:17:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3300870244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec101d00 0x7f43ec102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3300870244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec103880 0x7f43ec10a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3300870244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec102ef0 0x7f43ec103340 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 -- 172.21.15.26:0/3300870244 >> 172.21.15.26:0/3300870244 conn(0x7f43ec0fd360 msgr2=0x7f43ec0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 -- 172.21.15.26:0/3300870244 shutdown_connections 2023-07-13T22:17:22.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.978+0000 7f43f37ff700 1 -- 172.21.15.26:0/3300870244 wait complete. 2023-07-13T22:17:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.979+0000 7f43f37ff700 1 Processor -- start 2023-07-13T22:17:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 -- start start 2023-07-13T22:17:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec102ef0 0x7f43ec197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec103880 0x7f43ec19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec19f5d0 con 0x7f43ec102ef0 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec19f710 con 0x7f43ec103880 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f37ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ec19f8a0 con 0x7f43ec101d00 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.980+0000 7f43f159b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49136/0 (socket says 172.21.15.26:49136) 2023-07-13T22:17:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 -- 172.21.15.26:0/3488030077 learned_addr learned my addr 172.21.15.26:0/3488030077 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec103880 0x7f43ec19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec102ef0 0x7f43ec197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 -- 172.21.15.26:0/3488030077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec103880 msgr2=0x7f43ec19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec103880 0x7f43ec19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 -- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec102ef0 msgr2=0x7f43ec197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec102ef0 0x7f43ec197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 -- 172.21.15.26:0/3488030077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80095f0 con 0x7f43ec101d00 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.981+0000 7f43f159b700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f43e000c990 tx=0x7f43e000cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:22.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.982+0000 7f43de7fc700 1 -- 172.21.15.26:0/3488030077 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0007940 con 0x7f43ec101d00 2023-07-13T22:17:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.982+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec19fb80 con 0x7f43ec101d00 2023-07-13T22:17:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.982+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec1a00e0 con 0x7f43ec101d00 2023-07-13T22:17:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.982+0000 7f43de7fc700 1 -- 172.21.15.26:0/3488030077 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e000f450 con 0x7f43ec101d00 2023-07-13T22:17:22.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.983+0000 7f43de7fc700 1 -- 172.21.15.26:0/3488030077 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0018660 con 0x7f43ec101d00 2023-07-13T22:17:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.988+0000 7f43de7fc700 1 -- 172.21.15.26:0/3488030077 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43e0007aa0 con 0x7f43ec101d00 2023-07-13T22:17:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.989+0000 7f4407040b80 1 Processor -- start 2023-07-13T22:17:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.989+0000 7f4407040b80 1 -- start start 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563550 0x558060563960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 0x558060564760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060468d30 con 0x558060563ea0 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580604bbac0 con 0x558060563550 2023-07-13T22:17:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.990+0000 7f4407040b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580604b4f50 con 0x558060564cf0 2023-07-13T22:17:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f1d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563550 0x558060563960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49140/0 (socket says 172.21.15.26:49140) 2023-07-13T22:17:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 -- 172.21.15.26:0/495740522 learned_addr learned my addr 172.21.15.26:0/495740522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 -- 172.21.15.26:0/495740522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563550 msgr2=0x558060563960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563550 0x558060563960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.991+0000 7f43f0d9a700 1 -- 172.21.15.26:0/495740522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 msgr2=0x558060564760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43f159b700 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 0x558060564760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 0x558060564760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43f0d9a700 1 -- 172.21.15.26:0/495740522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580602e2e60 con 0x558060564cf0 2023-07-13T22:17:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43f159b700 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 0x558060564760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:17:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43de7fc700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 0x7f43d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.992+0000 7f43de7fc700 1 -- 172.21.15.26:0/3488030077 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43e00cc6e0 con 0x7f43ec101d00 2023-07-13T22:17:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.993+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 0x7f43d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.993+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43e8015000 tx=0x7f43e8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd0dc7b95354a2f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.993+0000 7f43d77fe700 1 -- 172.21.15.26:0/495740522 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d070 con 0x558060564cf0 2023-07-13T22:17:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.994+0000 7f43d77fe700 1 -- 172.21.15.26:0/495740522 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e801e040 con 0x558060564cf0 2023-07-13T22:17:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.994+0000 7f43d77fe700 1 -- 172.21.15.26:0/495740522 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8003c30 con 0x558060564cf0 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.994+0000 7f4407040b80 1 -- 172.21.15.26:0/495740522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 msgr2=0x558060565160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.994+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 0x7f43d80a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43e400a4c0 tx=0x7f43e400d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.994+0000 7f4407040b80 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f43e8015000 tx=0x7f43e8009f90 comp rx=0 tx=0).stop 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 -- 172.21.15.26:0/495740522 shutdown_connections 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060564cf0 0x558060565160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563550 0x558060563960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 --2- 172.21.15.26:0/495740522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563ea0 0x558060564760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 -- 172.21.15.26:0/495740522 >> 172.21.15.26:0/495740522 conn(0x558060558e20 msgr2=0x55806055e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 -- 172.21.15.26:0/495740522 shutdown_connections 2023-07-13T22:17:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 -- 172.21.15.26:0/495740522 wait complete. 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.995+0000 7f4407040b80 1 -- start start 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.996+0000 7f4407040b80 1 Processor -- start 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.996+0000 7f4407040b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 0x55806056d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.996+0000 7f4407040b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f4407040b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 0x558060574700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f4407040b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060454b40 con 0x558060564cf0 2023-07-13T22:17:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f4407040b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060454840 con 0x558060563ea0 2023-07-13T22:17:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f4407040b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580603389a0 con 0x558060563550 2023-07-13T22:17:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f4407040b80 10 client.0 _ll_register_callbacks cb 0x5580604527e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f43f0d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f43f0d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57330/0 (socket says 172.21.15.26:57330) 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2352221819 learned_addr learned my addr 172.21.15.26:0/2352221819 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.997+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 0x558060574700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f159b700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 0x55806056d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2352221819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 msgr2=0x55806056d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 0x55806056d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2352221819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 msgr2=0x558060574700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 0x558060574700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2352221819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ec10aa70 con 0x558060563ea0 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c3eeb867cb69910 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d040 con 0x558060563ea0 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.998+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43e8003b30 con 0x558060563ea0 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55806040e570 con 0x558060563ea0 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 0x558060574700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f43f159b700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 0x55806056d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e8005a00 con 0x558060563ea0 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 10 client.26000 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5580602cbce0 con 0x558060563ea0 2023-07-13T22:17:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55806040e570 con 0x558060563ea0 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f43e8004c20 con 0x558060563ea0 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 10 client.26000 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 10 client.26000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:22.999+0000 7f4407040b80 20 client.26000 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558060578a00 0x55806057ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f4407040b80 4 client.26000 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580604adf30 con 0x558060578a00 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f43f159b700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558060578a00 0x55806057ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f43f159b700 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558060578a00 0x55806057ae10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96460f0c49f624cd server_cookie=9a87431d6273e914 in_seq=0 out_seq=0 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.000+0000 7f43d4ff9700 10 client.26000 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.001+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2352221819 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580604adf30 con 0x558060578a00 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.001+0000 7f43d4ff9700 10 client.26000 handle_command_reply: tid=0 2023-07-13T22:17:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.001+0000 7f4407040b80 1 client.26000 shutdown 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.001+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 msgr2=0x55806056d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.001+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e8005740 comp rx=0 tx=0).stop 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 shutdown_connections 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060564cf0 0x558060574700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x558060578a00 0x55806057ae10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563550 0x55806056d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 --2- 172.21.15.26:0/2352221819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060563ea0 0x55806056d6c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:22.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 >> 172.21.15.26:0/2352221819 conn(0x558060558e20 msgr2=0x55806055d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:22.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f43d4ff9700 0 client.26000 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:22.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 shutdown_connections 2023-07-13T22:17:22.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 1 -- 172.21.15.26:0/2352221819 wait complete. 2023-07-13T22:17:22.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.002+0000 7f4407040b80 20 client.26000 trim_cache size 0 max 16384 2023-07-13T22:17:23.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.021+0000 7f4407040b80 1 Processor -- start 2023-07-13T22:17:23.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.021+0000 7f4407040b80 1 -- start start 2023-07-13T22:17:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x55806056b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x5580605746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x558060567b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060555f70 con 0x558060563550 2023-07-13T22:17:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55806055e040 con 0x558060564cf0 2023-07-13T22:17:23.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f4407040b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060575670 con 0x558060563ea0 2023-07-13T22:17:23.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f43f159b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x55806056b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f43f159b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x55806056b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45024/0 (socket says 172.21.15.26:45024) 2023-07-13T22:17:23.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f43f0d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x5580605746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f43f159b700 1 -- 172.21.15.26:0/2379462544 learned_addr learned my addr 172.21.15.26:0/2379462544 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.022+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x558060567b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2379462544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 msgr2=0x558060567b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x558060567b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2379462544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 msgr2=0x55806056b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x55806056b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 -- 172.21.15.26:0/2379462544 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580603092d0 con 0x558060563ea0 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x558060567b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.023+0000 7f43f0d9a700 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x5580605746c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f43e800ea70 tx=0x7f43e80050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869686f06e92689e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.024+0000 7f43d7fff700 1 -- 172.21.15.26:0/2379462544 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e801d070 con 0x558060563ea0 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.024+0000 7f43d7fff700 1 -- 172.21.15.26:0/2379462544 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e800bd90 con 0x558060563ea0 2023-07-13T22:17:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.024+0000 7f43d7fff700 1 -- 172.21.15.26:0/2379462544 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800e760 con 0x558060563ea0 2023-07-13T22:17:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 -- 172.21.15.26:0/2379462544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 msgr2=0x5580605746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x5580605746c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f43e800ea70 tx=0x7f43e80050f0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 -- 172.21.15.26:0/2379462544 shutdown_connections 2023-07-13T22:17:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x55806056b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x558060567b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 --2- 172.21.15.26:0/2379462544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x5580605746c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 -- 172.21.15.26:0/2379462544 >> 172.21.15.26:0/2379462544 conn(0x558060558e20 msgr2=0x55806055d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 -- 172.21.15.26:0/2379462544 shutdown_connections 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.025+0000 7f4407040b80 1 -- 172.21.15.26:0/2379462544 wait complete. 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.026+0000 7f4407040b80 1 -- start start 2023-07-13T22:17:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.026+0000 7f4407040b80 1 Processor -- start 2023-07-13T22:17:23.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x5580605714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x558060571a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558060572100 con 0x558060563550 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580605d55c0 con 0x558060564cf0 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580605d5700 con 0x558060563ea0 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f4407040b80 10 client.0 _ll_register_callbacks cb 0x7f43e8009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57354/0 (socket says 172.21.15.26:57354) 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 -- 172.21.15.26:0/2454293707 learned_addr learned my addr 172.21.15.26:0/2454293707 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 -- 172.21.15.26:0/2454293707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 msgr2=0x558060571a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x558060571a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 -- 172.21.15.26:0/2454293707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 msgr2=0x5580605714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.027+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x5580605714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.028+0000 7f43f1d9c700 1 -- 172.21.15.26:0/2454293707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580602cbce0 con 0x558060564cf0 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.028+0000 7f43f1d9c700 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43e4018720 tx=0x7f43e4004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7951c243eb4c5b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.028+0000 7f43d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.028+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4025660 con 0x558060564cf0 2023-07-13T22:17:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43e40257c0 con 0x558060564cf0 2023-07-13T22:17:23.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43e801e040 con 0x558060564cf0 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43e4025980 con 0x558060564cf0 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 10 client.26006 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5580602e1330 con 0x558060564cf0 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e4032030 con 0x558060564cf0 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f43e4023070 con 0x558060564cf0 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 10 client.26006 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 20 client.26006 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5580605d7270 0x5580605d9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 4 client.26006 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.029+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5580604700d0 con 0x5580605d7270 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.030+0000 7f43f159b700 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5580605d7270 0x5580605d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.030+0000 7f43d4ff9700 10 client.26006 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.030+0000 7f43f159b700 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5580605d7270 0x5580605d9680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad6c3ec0857b995 server_cookie=af1c435b14a5ca1d in_seq=0 out_seq=0 2023-07-13T22:17:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f43d4ff9700 1 -- 172.21.15.26:0/2454293707 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5580604700d0 con 0x5580605d7270 2023-07-13T22:17:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f43d4ff9700 10 client.26006 handle_command_reply: tid=0 2023-07-13T22:17:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 client.26006 shutdown 2023-07-13T22:17:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 msgr2=0x5580605d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f43e4018720 tx=0x7f43e4004af0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 shutdown_connections 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558060563550 0x5580605714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5580605d7270 0x5580605d9680 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558060564cf0 0x5580605d5080 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 --2- 172.21.15.26:0/2454293707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558060563ea0 0x558060571a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 >> 172.21.15.26:0/2454293707 conn(0x558060558e20 msgr2=0x55806056b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.032+0000 7f43d4ff9700 0 client.26006 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.033+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 shutdown_connections 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.033+0000 7f4407040b80 1 -- 172.21.15.26:0/2454293707 wait complete. 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.033+0000 7f4407040b80 20 client.26006 trim_cache size 0 max 16384 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:17:23.023 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:17:23.024 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "626da2ee-e3bf-4999-898f-cc9d0f91879b", 2023-07-13T22:17:23.024 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:17:23.024 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:17:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 msgr2=0x7f43d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 0x7f43d80a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f43e400a4c0 tx=0x7f43e400d040 comp rx=0 tx=0).stop 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 msgr2=0x7f43ec197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f43e000c990 tx=0x7f43e000cca0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 shutdown_connections 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43ec103880 0x7f43ec19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ec101d00 0x7f43ec197a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f43d80a1150 0x7f43d80a3600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 --2- 172.21.15.26:0/3488030077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43ec102ef0 0x7f43ec197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 >> 172.21.15.26:0/3488030077 conn(0x7f43ec0fd360 msgr2=0x7f43ec1062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.034+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 shutdown_connections 2023-07-13T22:17:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.035+0000 7f43f37ff700 1 -- 172.21.15.26:0/3488030077 wait complete. 2023-07-13T22:17:23.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:17:23.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:22 smithi117 ceph-mon[129074]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 277 MiB/s wr, 87 op/s 2023-07-13T22:17:23.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:22 smithi121 ceph-mon[129187]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 277 MiB/s wr, 87 op/s 2023-07-13T22:17:23.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.504+0000 7f32115ff700 1 -- 172.21.15.26:0/3421314112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c071830 msgr2=0x7f320c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.504+0000 7f32115ff700 1 --2- 172.21.15.26:0/3421314112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c071830 0x7f320c10b7d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3200009990 tx=0x7f3200009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.511+0000 7f32115ff700 1 -- 172.21.15.26:0/3421314112 shutdown_connections 2023-07-13T22:17:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.511+0000 7f32115ff700 1 --2- 172.21.15.26:0/3421314112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c10bd60 0x7f320c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.511+0000 7f32115ff700 1 --2- 172.21.15.26:0/3421314112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f320c070ee0 0x7f320c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.511+0000 7f32115ff700 1 --2- 172.21.15.26:0/3421314112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c071830 0x7f320c10b7d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.511+0000 7f32115ff700 1 -- 172.21.15.26:0/3421314112 >> 172.21.15.26:0/3421314112 conn(0x7f320c06ca90 msgr2=0x7f320c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.533+0000 7f32115ff700 1 -- 172.21.15.26:0/3421314112 shutdown_connections 2023-07-13T22:17:23.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.541+0000 7f32115ff700 1 -- 172.21.15.26:0/3421314112 wait complete. 2023-07-13T22:17:23.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.542+0000 7f32115ff700 1 Processor -- start 2023-07-13T22:17:23.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 -- start start 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c070ee0 0x7f320c113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f320c10bd60 0x7f320c1b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c117110 con 0x7f320c070ee0 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.558+0000 7f32115ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c117280 con 0x7f320c10bd60 2023-07-13T22:17:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.559+0000 7f32115ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1b4f10 con 0x7f320c071830 2023-07-13T22:17:23.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.559+0000 7f320a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.559+0000 7f320a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49182/0 (socket says 172.21.15.26:49182) 2023-07-13T22:17:23.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.559+0000 7f320a7fc700 1 -- 172.21.15.26:0/4044423436 learned_addr learned my addr 172.21.15.26:0/4044423436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.560+0000 7f320a7fc700 1 -- 172.21.15.26:0/4044423436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f320c10bd60 msgr2=0x7f320c1b49d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.560+0000 7f320a7fc700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f320c10bd60 0x7f320c1b49d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.560+0000 7f320a7fc700 1 -- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c070ee0 msgr2=0x7f320c113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.560+0000 7f320a7fc700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c070ee0 0x7f320c113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.560+0000 7f320a7fc700 1 -- 172.21.15.26:0/4044423436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000095f0 con 0x7f320c071830 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.562+0000 7f320a7fc700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3200003cd0 tx=0x7f3200003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.563+0000 7f31f3fff700 1 -- 172.21.15.26:0/4044423436 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320001d070 con 0x7f320c071830 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.564+0000 7f31f3fff700 1 -- 172.21.15.26:0/4044423436 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320000f460 con 0x7f320c071830 2023-07-13T22:17:23.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.564+0000 7f31f3fff700 1 -- 172.21.15.26:0/4044423436 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3200022680 con 0x7f320c071830 2023-07-13T22:17:23.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.564+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1b5140 con 0x7f320c071830 2023-07-13T22:17:23.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.564+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1b5530 con 0x7f320c071830 2023-07-13T22:17:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.575+0000 7f31f3fff700 1 -- 172.21.15.26:0/4044423436 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f32000227e0 con 0x7f320c071830 2023-07-13T22:17:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.577+0000 7f31f3fff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 0x7f31f40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.578+0000 7f31f3fff700 1 -- 172.21.15.26:0/4044423436 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32000ce980 con 0x7f320c071830 2023-07-13T22:17:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 Processor -- start 2023-07-13T22:17:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 -- start start 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecef4d0 0x55bcfecef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfecf10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfebf4cb0 con 0x55bcfecf0c70 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfec47a40 con 0x55bcfecef4d0 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.586+0000 7f3224f09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfec40ed0 con 0x55bcfecefe20 2023-07-13T22:17:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.587+0000 7f320a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.587+0000 7f320a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49188/0 (socket says 172.21.15.26:49188) 2023-07-13T22:17:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.587+0000 7f320a7fc700 1 -- 172.21.15.26:0/3174507849 learned_addr learned my addr 172.21.15.26:0/3174507849 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320b7fe700 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfecf10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 -- 172.21.15.26:0/3174507849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecef4d0 msgr2=0x55bcfecef8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecef4d0 0x55bcfecef8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 -- 172.21.15.26:0/3174507849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 msgr2=0x55bcfecf10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfecf10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 -- 172.21.15.26:0/3174507849 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcfea6ede0 con 0x55bcfecefe20 2023-07-13T22:17:23.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f320a7fc700 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3200003ed0 tx=0x7f3200009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30be0691d44b5180 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.589+0000 7f31f17fa700 1 -- 172.21.15.26:0/3174507849 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32000b3070 con 0x55bcfecefe20 2023-07-13T22:17:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.590+0000 7f31f17fa700 1 -- 172.21.15.26:0/3174507849 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320009fe00 con 0x55bcfecefe20 2023-07-13T22:17:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.590+0000 7f31f17fa700 1 -- 172.21.15.26:0/3174507849 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320009b500 con 0x55bcfecefe20 2023-07-13T22:17:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.591+0000 7f3224f09b80 1 -- 172.21.15.26:0/3174507849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 msgr2=0x55bcfecf06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.591+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3200003ed0 tx=0x7f3200009310 comp rx=0 tx=0).stop 2023-07-13T22:17:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.601+0000 7f3224f09b80 1 -- 172.21.15.26:0/3174507849 shutdown_connections 2023-07-13T22:17:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.601+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfecf10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.601+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecef4d0 0x55bcfecef8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.601+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3174507849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfecf06e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.601+0000 7f3224f09b80 1 -- 172.21.15.26:0/3174507849 >> 172.21.15.26:0/3174507849 conn(0x55bcfece4da0 msgr2=0x55bcfecea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.602+0000 7f320affd700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 0x7f31f40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.608+0000 7f3224f09b80 1 -- 172.21.15.26:0/3174507849 shutdown_connections 2023-07-13T22:17:23.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.608+0000 7f320affd700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 0x7f31f40a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55bcfeceb640 tx=0x7f31fc006da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.608+0000 7f3224f09b80 1 -- 172.21.15.26:0/3174507849 wait complete. 2023-07-13T22:17:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.620+0000 7f3224f09b80 1 -- start start 2023-07-13T22:17:23.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.621+0000 7f3224f09b80 1 Processor -- start 2023-07-13T22:17:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.621+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfed1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.621+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecf0c70 0x55bcfed1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.621+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.622+0000 7f3224f09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfebe0ac0 con 0x55bcfed20450 2023-07-13T22:17:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.622+0000 7f3224f09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfebe07c0 con 0x55bcfecf0c70 2023-07-13T22:17:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.622+0000 7f3224f09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfeac4920 con 0x55bcfecefe20 2023-07-13T22:17:23.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.622+0000 7f3224f09b80 10 client.0 _ll_register_callbacks cb 0x55bcfebde760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.625+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.625+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45060/0 (socket says 172.21.15.26:45060) 2023-07-13T22:17:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.625+0000 7f320b7fe700 1 -- 172.21.15.26:0/3790458175 learned_addr learned my addr 172.21.15.26:0/3790458175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.626+0000 7f320b7fe700 1 -- 172.21.15.26:0/3790458175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 msgr2=0x55bcfed1aee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.626+0000 7f320b7fe700 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfed1aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.626+0000 7f320b7fe700 1 -- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecf0c70 msgr2=0x55bcfed1b420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.626+0000 7f320b7fe700 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecf0c70 0x55bcfed1b420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.626+0000 7f320b7fe700 1 -- 172.21.15.26:0/3790458175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1b5530 con 0x55bcfed20450 2023-07-13T22:17:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.627+0000 7f320b7fe700 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f320400c7c0 tx=0x7f320400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0162f5840ca601 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.627+0000 7f31eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.628+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204011070 con 0x55bcfed20450 2023-07-13T22:17:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.628+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3204015410 con 0x55bcfed20450 2023-07-13T22:17:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.628+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcfeb9a4f0 con 0x55bcfed20450 2023-07-13T22:17:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.629+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32040155d0 con 0x55bcfed20450 2023-07-13T22:17:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.629+0000 7f3224f09b80 10 client.15792 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.629+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bcfea57c60 con 0x55bcfed20450 2023-07-13T22:17:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.630+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcfeb9a4f0 con 0x55bcfed20450 2023-07-13T22:17:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.630+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f320401d070 con 0x55bcfed20450 2023-07-13T22:17:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:23.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 20 client.15792 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed270b0 0x55bcfed294c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 4 client.15792 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.631+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcfec39eb0 con 0x55bcfed270b0 2023-07-13T22:17:23.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.632+0000 7f320affd700 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed270b0 0x55bcfed294c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.633+0000 7f320affd700 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed270b0 0x55bcfed294c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edda20d35fdcfa8 server_cookie=797268eaa6c459c0 in_seq=0 out_seq=0 2023-07-13T22:17:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.633+0000 7f31eeff5700 10 client.15792 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.634+0000 7f31eeff5700 1 -- 172.21.15.26:0/3790458175 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bcfec39eb0 con 0x55bcfed270b0 2023-07-13T22:17:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.634+0000 7f31eeff5700 10 client.15792 handle_command_reply: tid=0 2023-07-13T22:17:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.634+0000 7f3224f09b80 1 client.15792 shutdown 2023-07-13T22:17:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.634+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 msgr2=0x55bcfed22830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.634+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f320400c7c0 tx=0x7f320400c8a0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 shutdown_connections 2023-07-13T22:17:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfed20450 0x55bcfed22830 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed270b0 0x55bcfed294c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecefe20 0x55bcfed1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 --2- 172.21.15.26:0/3790458175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecf0c70 0x55bcfed1b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 >> 172.21.15.26:0/3790458175 conn(0x55bcfece4da0 msgr2=0x55bcfece98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.635+0000 7f31eeff5700 0 client.15792 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.636+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 shutdown_connections 2023-07-13T22:17:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.636+0000 7f3224f09b80 1 -- 172.21.15.26:0/3790458175 wait complete. 2023-07-13T22:17:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.636+0000 7f3224f09b80 20 client.15792 trim_cache size 0 max 16384 2023-07-13T22:17:23.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.673+0000 7f3224f09b80 1 Processor -- start 2023-07-13T22:17:23.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.673+0000 7f3224f09b80 1 -- start start 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfed1a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfed20450 0x55bcfed290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfed1bd80 con 0x55bcfecf0c70 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfed24500 con 0x55bcfecefe20 2023-07-13T22:17:23.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f3224f09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfeb9a4f0 con 0x55bcfed20450 2023-07-13T22:17:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f320affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f320affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57416/0 (socket says 172.21.15.26:57416) 2023-07-13T22:17:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.674+0000 7f320affd700 1 -- 172.21.15.26:0/2311684540 learned_addr learned my addr 172.21.15.26:0/2311684540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320b7fe700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfed20450 0x55bcfed290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320a7fc700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfed1a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 -- 172.21.15.26:0/2311684540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfed20450 msgr2=0x55bcfed290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfed20450 0x55bcfed290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 -- 172.21.15.26:0/2311684540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 msgr2=0x55bcfed1a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfed1a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 -- 172.21.15.26:0/2311684540 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcfecea470 con 0x55bcfecefe20 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320a7fc700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfed1a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f320affd700 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f31fc006fd0 tx=0x7f31fc00e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b860c662d3ff55 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.675+0000 7f31f1ffb700 1 -- 172.21.15.26:0/2311684540 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc020e70 con 0x55bcfecefe20 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f31f1ffb700 1 -- 172.21.15.26:0/2311684540 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31fc0205a0 con 0x55bcfecefe20 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f31f1ffb700 1 -- 172.21.15.26:0/2311684540 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0044a0 con 0x55bcfecefe20 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 -- 172.21.15.26:0/2311684540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 msgr2=0x55bcfed36d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f31fc006fd0 tx=0x7f31fc00e000 comp rx=0 tx=0).stop 2023-07-13T22:17:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 -- 172.21.15.26:0/2311684540 shutdown_connections 2023-07-13T22:17:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfed20450 0x55bcfed290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfecefe20 0x55bcfed36d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 --2- 172.21.15.26:0/2311684540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecf0c70 0x55bcfed1a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.676+0000 7f3224f09b80 1 -- 172.21.15.26:0/2311684540 >> 172.21.15.26:0/2311684540 conn(0x55bcfece4da0 msgr2=0x55bcfece98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.677+0000 7f3224f09b80 1 -- 172.21.15.26:0/2311684540 shutdown_connections 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.677+0000 7f3224f09b80 1 -- 172.21.15.26:0/2311684540 wait complete. 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.677+0000 7f3224f09b80 1 -- start start 2023-07-13T22:17:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.678+0000 7f3224f09b80 1 Processor -- start 2023-07-13T22:17:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.678+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 0x55bcfed34c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecf0c70 0x55bcfed351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfed36700 con 0x55bcfecefe20 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfed36870 con 0x55bcfed20450 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcfed36a30 con 0x55bcfecf0c70 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f3224f09b80 10 client.0 _ll_register_callbacks cb 0x7f3204009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.679+0000 7f320a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecf0c70 0x55bcfed351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:57418/0 (socket says 172.21.15.26:57418) 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 -- 172.21.15.26:0/796157931 learned_addr learned my addr 172.21.15.26:0/796157931 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320affd700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 0x55bcfed34c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 -- 172.21.15.26:0/796157931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecf0c70 msgr2=0x55bcfed351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecf0c70 0x55bcfed351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 -- 172.21.15.26:0/796157931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 msgr2=0x55bcfed34c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 0x55bcfed34c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320b7fe700 1 -- 172.21.15.26:0/796157931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31fc00c040 con 0x55bcfed20450 2023-07-13T22:17:23.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.680+0000 7f320affd700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 0x55bcfed34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.681+0000 7f320b7fe700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3204000f80 tx=0x7f320400c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b723f72a57e2fad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f31eeff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204011040 con 0x55bcfed20450 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f320009fe00 con 0x55bcfed20450 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f320401e3a0 con 0x55bcfed20450 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320401ee70 con 0x55bcfed20450 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f3224f09b80 10 client.26012 fetch_fsmap learned FSMap version 28 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.682+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bcfea95250 con 0x55bcfed20450 2023-07-13T22:17:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.683+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320009fe00 con 0x55bcfed20450 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.683+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f320401d030 con 0x55bcfed20450 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 28 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 20 client.26012 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed7c560 0x55bcfed7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 4 client.26012 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:17:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bcfebfc050 con 0x55bcfed7c560 2023-07-13T22:17:23.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.684+0000 7f320affd700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed7c560 0x55bcfed7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:23.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.687+0000 7f320affd700 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed7c560 0x55bcfed7e970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aab8b24bdcbedc5 server_cookie=be051790088a9160 in_seq=0 out_seq=0 2023-07-13T22:17:23.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.687+0000 7f31eeff5700 10 client.26012 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.687+0000 7f31eeff5700 1 -- 172.21.15.26:0/796157931 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bcfebfc050 con 0x55bcfed7c560 2023-07-13T22:17:23.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.687+0000 7f31eeff5700 10 client.26012 handle_command_reply: tid=0 2023-07-13T22:17:23.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.687+0000 7f3224f09b80 1 client.26012 shutdown 2023-07-13T22:17:23.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.693+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 msgr2=0x55bcfed361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.693+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3204000f80 tx=0x7f320400c340 comp rx=0 tx=0).stop 2023-07-13T22:17:23.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 shutdown_connections 2023-07-13T22:17:23.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bcfecefe20 0x55bcfed34c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55bcfed7c560 0x55bcfed7e970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcfed20450 0x55bcfed361c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 --2- 172.21.15.26:0/796157931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcfecf0c70 0x55bcfed351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.694+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 >> 172.21.15.26:0/796157931 conn(0x55bcfece4da0 msgr2=0x55bcfed290c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.695+0000 7f31eeff5700 0 client.26012 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:23.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.695+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 shutdown_connections 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout: "626da2ee-e3bf-4999-898f-cc9d0f91879b": { 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:17:23.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "626da2ee-e3bf-4999-898f-cc9d0f91879b", 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.695+0000 7f3224f09b80 1 -- 172.21.15.26:0/796157931 wait complete. 2023-07-13T22:17:23.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.695+0000 7f3224f09b80 20 client.26012 trim_cache size 0 max 16384 2023-07-13T22:17:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 msgr2=0x7f31f40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 0x7f31f40a3800 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55bcfeceb640 tx=0x7f31fc006da0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 msgr2=0x7f320c114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:23.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3200003cd0 tx=0x7f3200003db0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 shutdown_connections 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f31f40a1350 0x7f31f40a3800 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.703+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320c070ee0 0x7f320c113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.704+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f320c10bd60 0x7f320c1b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.704+0000 7f32115ff700 1 --2- 172.21.15.26:0/4044423436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f320c071830 0x7f320c114030 secure :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3200003cd0 tx=0x7f3200003db0 comp rx=0 tx=0).stop 2023-07-13T22:17:23.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.704+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 >> 172.21.15.26:0/4044423436 conn(0x7f320c06ca90 msgr2=0x7f320c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.705+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 shutdown_connections 2023-07-13T22:17:23.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:23.705+0000 7f32115ff700 1 -- 172.21.15.26:0/4044423436 wait complete. 2023-07-13T22:17:23.729 INFO:tasks.cephfs.filesystem:scrub status for tag:626da2ee-e3bf-4999-898f-cc9d0f91879b - {'path': '/', 'tag': '626da2ee-e3bf-4999-898f-cc9d0f91879b', 'options': 'recursive,force'} 2023-07-13T22:17:24.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:17:24.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:17:24.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:17:24.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.192 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:23 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:17:24.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:17:24.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:17:24.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:23 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:17:24.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:17:24.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:17:24.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:24.243 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:23 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:17:25.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:24 smithi117 ceph-mon[129074]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 107 op/s 2023-07-13T22:17:25.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:24 smithi026 ceph-mon[95647]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 107 op/s 2023-07-13T22:17:25.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:24 smithi121 ceph-mon[129187]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 330 MiB/s wr, 107 op/s 2023-07-13T22:17:27.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:26 smithi117 ceph-mon[129074]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:17:27.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:26 smithi026 ceph-mon[95647]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:17:27.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:26 smithi121 ceph-mon[129187]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:17:28.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:28 smithi117 ceph-mon[129074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:17:28.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:28 smithi026 ceph-mon[95647]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:17:28.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:28 smithi121 ceph-mon[129187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:17:29.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:29 smithi117 ceph-mon[129074]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 300 MiB/s wr, 99 op/s 2023-07-13T22:17:29.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:29 smithi026 ceph-mon[95647]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 300 MiB/s wr, 99 op/s 2023-07-13T22:17:29.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:29 smithi121 ceph-mon[129187]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 300 MiB/s wr, 99 op/s 2023-07-13T22:17:31.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:31 smithi117 ceph-mon[129074]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 98 op/s 2023-07-13T22:17:31.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:31 smithi026 ceph-mon[95647]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 98 op/s 2023-07-13T22:17:31.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:31 smithi121 ceph-mon[129187]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 299 MiB/s wr, 98 op/s 2023-07-13T22:17:32.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:32 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:17:32.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:32 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:17:32.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:32 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:17:33.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:33 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:17:33.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:33 smithi117 ceph-mon[129074]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 MiB/s wr, 91 op/s 2023-07-13T22:17:33.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:33 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:17:33.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:33 smithi026 ceph-mon[95647]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 MiB/s wr, 91 op/s 2023-07-13T22:17:33.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:33 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:17:33.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:33 smithi121 ceph-mon[129187]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 MiB/s wr, 91 op/s 2023-07-13T22:17:35.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:35 smithi121 ceph-mon[129187]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 324 MiB/s wr, 109 op/s 2023-07-13T22:17:35.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:35 smithi117 ceph-mon[129074]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 324 MiB/s wr, 109 op/s 2023-07-13T22:17:35.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:35 smithi026 ceph-mon[95647]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 324 MiB/s wr, 109 op/s 2023-07-13T22:17:37.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:37 smithi117 ceph-mon[129074]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 93 op/s 2023-07-13T22:17:37.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:37 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:17:37.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:37 smithi026 ceph-mon[95647]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 93 op/s 2023-07-13T22:17:37.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:37 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:17:37.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:37 smithi121 ceph-mon[129187]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 93 op/s 2023-07-13T22:17:37.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:37 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:17:39.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:39 smithi117 ceph-mon[129074]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:17:39.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:39 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:17:39.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:39 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:17:39.692 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:39 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:17:39.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:39 smithi026 ceph-mon[95647]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:17:39.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:17:39.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:17:39.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:39 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:17:39.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:39 smithi121 ceph-mon[129187]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 99 op/s 2023-07-13T22:17:39.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:17:39.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:17:39.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:39 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:17:41.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:41 smithi117 ceph-mon[129074]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:17:41.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:41 smithi026 ceph-mon[95647]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:17:41.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:41 smithi121 ceph-mon[129187]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:17:43.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:43 smithi117 ceph-mon[129074]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 278 MiB/s wr, 89 op/s 2023-07-13T22:17:43.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:43 smithi026 ceph-mon[95647]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 278 MiB/s wr, 89 op/s 2023-07-13T22:17:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:43 smithi121 ceph-mon[129187]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 278 MiB/s wr, 89 op/s 2023-07-13T22:17:44.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:44 smithi117 ceph-mon[129074]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:17:44.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:44 smithi026 ceph-mon[95647]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:17:44.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:44 smithi121 ceph-mon[129187]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:17:47.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:46 smithi117 ceph-mon[129074]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 85 op/s 2023-07-13T22:17:47.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:46 smithi026 ceph-mon[95647]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 85 op/s 2023-07-13T22:17:47.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:46 smithi121 ceph-mon[129187]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 85 op/s 2023-07-13T22:17:49.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:48 smithi117 ceph-mon[129074]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 90 op/s 2023-07-13T22:17:49.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:48 smithi026 ceph-mon[95647]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 90 op/s 2023-07-13T22:17:49.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:48 smithi121 ceph-mon[129187]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 90 op/s 2023-07-13T22:17:51.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:50 smithi117 ceph-mon[129074]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 87 op/s 2023-07-13T22:17:51.208 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:50 smithi026 ceph-mon[95647]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 87 op/s 2023-07-13T22:17:51.242 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:50 smithi121 ceph-mon[129187]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 87 op/s 2023-07-13T22:17:53.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:52 smithi117 ceph-mon[129074]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 81 op/s 2023-07-13T22:17:53.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:52 smithi026 ceph-mon[95647]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 81 op/s 2023-07-13T22:17:53.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:52 smithi121 ceph-mon[129187]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 257 MiB/s wr, 81 op/s 2023-07-13T22:17:53.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:17:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.242+0000 7f5f33fff700 1 -- 172.21.15.26:0/425451745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f34071b20 msgr2=0x7f5f3410b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.242+0000 7f5f33fff700 1 --2- 172.21.15.26:0/425451745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f34071b20 0x7f5f3410b530 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5f280098f0 tx=0x7f5f28009c00 comp rx=0 tx=0).stop 2023-07-13T22:17:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 -- 172.21.15.26:0/425451745 shutdown_connections 2023-07-13T22:17:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 --2- 172.21.15.26:0/425451745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f3410bba0 0x7f5f3410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 --2- 172.21.15.26:0/425451745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f340711d0 0x7f5f340715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 --2- 172.21.15.26:0/425451745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f34071b20 0x7f5f3410b530 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 -- 172.21.15.26:0/425451745 >> 172.21.15.26:0/425451745 conn(0x7f5f3406c8b0 msgr2=0x7f5f3406ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 -- 172.21.15.26:0/425451745 shutdown_connections 2023-07-13T22:17:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.245+0000 7f5f33fff700 1 -- 172.21.15.26:0/425451745 wait complete. 2023-07-13T22:17:54.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.247+0000 7f5f33fff700 1 Processor -- start 2023-07-13T22:17:54.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 -- start start 2023-07-13T22:17:54.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f340711d0 0x7f5f34116ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f3410bba0 0x7f5f34113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341b4850 con 0x7f5f340711d0 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341b4990 con 0x7f5f34071b20 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.248+0000 7f5f33fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341b4b50 con 0x7f5f3410bba0 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40136/0 (socket says 172.21.15.26:40136) 2023-07-13T22:17:54.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 -- 172.21.15.26:0/699075047 learned_addr learned my addr 172.21.15.26:0/699075047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 -- 172.21.15.26:0/699075047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f3410bba0 msgr2=0x7f5f34113fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f3410bba0 0x7f5f34113fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 -- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f340711d0 msgr2=0x7f5f34116ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.249+0000 7f5f327fc700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f340711d0 0x7f5f34116ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.250+0000 7f5f327fc700 1 -- 172.21.15.26:0/699075047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009520 con 0x7f5f34071b20 2023-07-13T22:17:54.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.250+0000 7f5f327fc700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800fbd0 tx=0x7f5f2800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.253+0000 7f5f23fff700 1 -- 172.21.15.26:0/699075047 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2801c070 con 0x7f5f34071b20 2023-07-13T22:17:54.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.253+0000 7f5f23fff700 1 -- 172.21.15.26:0/699075047 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f28005530 con 0x7f5f34071b20 2023-07-13T22:17:54.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.253+0000 7f5f23fff700 1 -- 172.21.15.26:0/699075047 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28021460 con 0x7f5f34071b20 2023-07-13T22:17:54.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.253+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f341b4dd0 con 0x7f5f34071b20 2023-07-13T22:17:54.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.253+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f341b51a0 con 0x7f5f34071b20 2023-07-13T22:17:54.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.283+0000 7f5f4d13fb80 1 Processor -- start 2023-07-13T22:17:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.294+0000 7f5f4d13fb80 1 -- start start 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f23fff700 1 -- 172.21.15.26:0/699075047 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f28003680 con 0x7f5f34071b20 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aab720 0x55afa2aabb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2aac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa29b0cb0 con 0x55afa2aace30 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2a03a40 con 0x55afa2aac070 2023-07-13T22:17:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.295+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa29fced0 con 0x55afa2aab720 2023-07-13T22:17:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.296+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.296+0000 7f5f327fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45370/0 (socket says 172.21.15.26:45370) 2023-07-13T22:17:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.296+0000 7f5f327fc700 1 -- 172.21.15.26:0/2928397995 learned_addr learned my addr 172.21.15.26:0/2928397995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.297+0000 7f5f327fc700 1 -- 172.21.15.26:0/2928397995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aab720 msgr2=0x55afa2aabb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.297+0000 7f5f327fc700 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aab720 0x55afa2aabb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.297+0000 7f5f327fc700 1 -- 172.21.15.26:0/2928397995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 msgr2=0x55afa2aac8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.297+0000 7f5f327fc700 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2aac8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.297+0000 7f5f327fc700 1 -- 172.21.15.26:0/2928397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afa282ade0 con 0x55afa2aace30 2023-07-13T22:17:54.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.298+0000 7f5f23fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 0x7f5f1c0a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.298+0000 7f5f23fff700 1 -- 172.21.15.26:0/699075047 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f280ccfb0 con 0x7f5f34071b20 2023-07-13T22:17:54.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.299+0000 7f5f327fc700 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f280d0e00 tx=0x7f5f280d0e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad4ebeff64ad035 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.300+0000 7f5f217fa700 1 -- 172.21.15.26:0/2928397995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280967d0 con 0x55afa2aace30 2023-07-13T22:17:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.300+0000 7f5f217fa700 1 -- 172.21.15.26:0/2928397995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f280d0a40 con 0x55afa2aace30 2023-07-13T22:17:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.300+0000 7f5f217fa700 1 -- 172.21.15.26:0/2928397995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28099fd0 con 0x55afa2aace30 2023-07-13T22:17:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.301+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/2928397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 msgr2=0x55afa2aad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.301+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5f280d0e00 tx=0x7f5f280d0e30 comp rx=0 tx=0).stop 2023-07-13T22:17:54.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.303+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/2928397995 shutdown_connections 2023-07-13T22:17:54.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.303+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aace30 0x55afa2aad2a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.303+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aab720 0x55afa2aabb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.303+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/2928397995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2aac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.303+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/2928397995 >> 172.21.15.26:0/2928397995 conn(0x55afa2aa0ff0 msgr2=0x55afa2aa6460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.304+0000 7f5f337fe700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 0x7f5f1c0a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.305+0000 7f5f337fe700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 0x7f5f1c0a38e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0095b0 tx=0x7f5f2c009220 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.305+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/2928397995 shutdown_connections 2023-07-13T22:17:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.305+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/2928397995 wait complete. 2023-07-13T22:17:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.306+0000 7f5f4d13fb80 1 -- start start 2023-07-13T22:17:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.307+0000 7f5f4d13fb80 1 Processor -- start 2023-07-13T22:17:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.307+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2ab5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa299cac0 con 0x55afa2aab720 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa299c7c0 con 0x55afa2aac070 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2880920 con 0x55afa2aace30 2023-07-13T22:17:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.308+0000 7f5f4d13fb80 10 client.0 _ll_register_callbacks cb 0x55afa299a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:54.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60854/0 (socket says 172.21.15.26:60854) 2023-07-13T22:17:54.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 -- 172.21.15.26:0/34474976 learned_addr learned my addr 172.21.15.26:0/34474976 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:54.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f327fc700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2ab5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 -- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 msgr2=0x55afa2ab5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2ab5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 -- 172.21.15.26:0/34474976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 msgr2=0x55afa2ab5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.309+0000 7f5f337fe700 1 -- 172.21.15.26:0/34474976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f341b51a0 con 0x55afa2aace30 2023-07-13T22:17:54.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.310+0000 7f5f337fe700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0179b0 tx=0x7f5f2c004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd47d210776dc65 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.313+0000 7f5f1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:54.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.313+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01c070 con 0x55afa2aace30 2023-07-13T22:17:54.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.313+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f2c020410 con 0x55afa2aace30 2023-07-13T22:17:54.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.313+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afa29f5eb0 con 0x55afa2aace30 2023-07-13T22:17:54.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.340+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f2c0205d0 con 0x55afa2aace30 2023-07-13T22:17:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.351+0000 7f5f4d13fb80 10 client.25993 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.351+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55afa2813c60 con 0x55afa2aace30 2023-07-13T22:17:54.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55afa29f5eb0 con 0x55afa2aace30 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f2c028070 con 0x55afa2aace30 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 10 client.25993 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 20 client.25993 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2ac0e10 0x55afa2ac3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 4 client.25993 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.367+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afa29b8050 con 0x55afa2ac0e10 2023-07-13T22:17:54.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.368+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2ac0e10 0x55afa2ac3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.403+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2ac0e10 0x55afa2ac3220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381f5730c2ae0408 server_cookie=98e4c06cc099cee2 in_seq=0 out_seq=0 2023-07-13T22:17:54.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.403+0000 7f5f1a7fc700 10 client.25993 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.425+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/34474976 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55afa29b8050 con 0x55afa2ac0e10 2023-07-13T22:17:54.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.425+0000 7f5f1a7fc700 10 client.25993 handle_command_reply: tid=0 2023-07-13T22:17:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 client.25993 shutdown 2023-07-13T22:17:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 msgr2=0x55afa2abc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0179b0 tx=0x7f5f2c004ce0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 shutdown_connections 2023-07-13T22:17:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2ac0e10 0x55afa2ac3220 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abc5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/34474976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2ab5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.430+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 >> 172.21.15.26:0/34474976 conn(0x55afa2aa0ff0 msgr2=0x55afa2aa5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.436+0000 7f5f1a7fc700 0 client.25993 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:54.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.445+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 shutdown_connections 2023-07-13T22:17:54.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.447+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/34474976 wait complete. 2023-07-13T22:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.450+0000 7f5f4d13fb80 20 client.25993 trim_cache size 0 max 16384 2023-07-13T22:17:54.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.487+0000 7f5f4d13fb80 1 Processor -- start 2023-07-13T22:17:54.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 -- start start 2023-07-13T22:17:54.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2abc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2aa0bc0 con 0x55afa2aab720 2023-07-13T22:17:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2aa7500 con 0x55afa2aac070 2023-07-13T22:17:54.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.488+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2aa0e00 con 0x55afa2aace30 2023-07-13T22:17:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60868/0 (socket says 172.21.15.26:60868) 2023-07-13T22:17:54.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 -- 172.21.15.26:0/849755982 learned_addr learned my addr 172.21.15.26:0/849755982 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 -- 172.21.15.26:0/849755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 msgr2=0x55afa2abc4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2abc4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 -- 172.21.15.26:0/849755982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 msgr2=0x55afa2ab3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 -- 172.21.15.26:0/849755982 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afa2851250 con 0x55afa2aace30 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.489+0000 7f5f337fe700 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c016fa0 tx=0x7f5f2c016fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f227305a0d3c083 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.490+0000 7f5f21ffb700 1 -- 172.21.15.26:0/849755982 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01c030 con 0x55afa2aace30 2023-07-13T22:17:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.490+0000 7f5f21ffb700 1 -- 172.21.15.26:0/849755982 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f2c00f040 con 0x55afa2aace30 2023-07-13T22:17:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.490+0000 7f5f21ffb700 1 -- 172.21.15.26:0/849755982 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c01e570 con 0x55afa2aace30 2023-07-13T22:17:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/849755982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 msgr2=0x55afa2abce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c016fa0 tx=0x7f5f2c016fd0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/849755982 shutdown_connections 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aab720 0x55afa2ab3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2abce90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/849755982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aac070 0x55afa2abc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/849755982 >> 172.21.15.26:0/849755982 conn(0x55afa2aa0ff0 msgr2=0x55afa2ac7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/849755982 shutdown_connections 2023-07-13T22:17:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.491+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/849755982 wait complete. 2023-07-13T22:17:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.492+0000 7f5f4d13fb80 1 -- start start 2023-07-13T22:17:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.492+0000 7f5f4d13fb80 1 Processor -- start 2023-07-13T22:17:54.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aac070 0x55afa2ab8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2ab9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2ab97a0 con 0x55afa2aac070 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2ab9910 con 0x55afa2aab720 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afa2b14ce0 con 0x55afa2aace30 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f4d13fb80 10 client.0 _ll_register_callbacks cb 0x7f5f2c007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:54.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f32ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40164/0 (socket says 172.21.15.26:40164) 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.493+0000 7f5f32ffd700 1 -- 172.21.15.26:0/70082972 learned_addr learned my addr 172.21.15.26:0/70082972 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f327fc700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aac070 0x55afa2ab8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 -- 172.21.15.26:0/70082972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 msgr2=0x55afa2ab9260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2ab9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 -- 172.21.15.26:0/70082972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aac070 msgr2=0x55afa2ab8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:54.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aac070 0x55afa2ab8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 -- 172.21.15.26:0/70082972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afa2813c60 con 0x55afa2aab720 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.494+0000 7f5f32ffd700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005950 tx=0x7f5f24016420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafcc09fe3a45b3e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f1a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24016d50 con 0x55afa2aab720 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afa29564f0 con 0x55afa2aab720 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f24004830 con 0x55afa2aab720 2023-07-13T22:17:54.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f240049f0 con 0x55afa2aab720 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f4d13fb80 10 client.26021 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.495+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55afa28292b0 con 0x55afa2aab720 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55afa29564f0 con 0x55afa2aab720 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f240104c0 con 0x55afa2aab720 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 10 client.26021 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 20 client.26021 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:54.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2b16750 0x55afa2b18b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 4 client.26021 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:17:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.496+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55afa284bf70 con 0x55afa2b16750 2023-07-13T22:17:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.497+0000 7f5f327fc700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2b16750 0x55afa2b18b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.497+0000 7f5f327fc700 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2b16750 0x55afa2b18b60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7208ec9b87a9b4aa server_cookie=b2b78d9c1688879a in_seq=0 out_seq=0 2023-07-13T22:17:54.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.497+0000 7f5f1a7fc700 10 client.26021 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:54.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f1a7fc700 1 -- 172.21.15.26:0/70082972 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55afa284bf70 con 0x55afa2b16750 2023-07-13T22:17:54.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f1a7fc700 10 client.26021 handle_command_reply: tid=0 2023-07-13T22:17:54.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 client.26021 shutdown 2023-07-13T22:17:54.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 msgr2=0x55afa2abc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f24005950 tx=0x7f5f24016420 comp rx=0 tx=0).stop 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 shutdown_connections 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55afa2b16750 0x55afa2b18b60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afa2aace30 0x55afa2ab9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55afa2aab720 0x55afa2abc280 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 --2- 172.21.15.26:0/70082972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55afa2aac070 0x55afa2ab8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 >> 172.21.15.26:0/70082972 conn(0x55afa2aa0ff0 msgr2=0x55afa2ab3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.499+0000 7f5f1a7fc700 0 client.26021 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.500+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 shutdown_connections 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.500+0000 7f5f4d13fb80 1 -- 172.21.15.26:0/70082972 wait complete. 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.500+0000 7f5f4d13fb80 20 client.26021 trim_cache size 0 max 16384 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:17:54.490 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:17:54.491 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:17:54.491 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:17:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 msgr2=0x7f5f1c0a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 0x7f5f1c0a38e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0095b0 tx=0x7f5f2c009220 comp rx=0 tx=0).stop 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 msgr2=0x7f5f34113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f2800fbd0 tx=0x7f5f2800b8d0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 shutdown_connections 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f1c0a1430 0x7f5f1c0a38e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f340711d0 0x7f5f34116ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f3410bba0 0x7f5f34113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 --2- 172.21.15.26:0/699075047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f34071b20 0x7f5f34113950 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:54.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.501+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 >> 172.21.15.26:0/699075047 conn(0x7f5f3406c8b0 msgr2=0x7f5f3406dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:54.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.502+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 shutdown_connections 2023-07-13T22:17:54.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:54.502+0000 7f5f33fff700 1 -- 172.21.15.26:0/699075047 wait complete. 2023-07-13T22:17:54.528 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:17:54.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:17:54.914 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:54 smithi026 ceph-mon[95647]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 313 MiB/s wr, 99 op/s 2023-07-13T22:17:55.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.041+0000 7fa2201ff700 1 -- 172.21.15.26:0/2698674793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 msgr2=0x7fa21810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:55.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.041+0000 7fa2201ff700 1 --2- 172.21.15.26:0/2698674793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa21810b7d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa208009990 tx=0x7fa208009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 -- 172.21.15.26:0/2698674793 shutdown_connections 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 --2- 172.21.15.26:0/2698674793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa21810bd60 0x7fa21810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 --2- 172.21.15.26:0/2698674793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2180712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 --2- 172.21.15.26:0/2698674793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa21810b7d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 -- 172.21.15.26:0/2698674793 >> 172.21.15.26:0/2698674793 conn(0x7fa21806ca40 msgr2=0x7fa21806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 -- 172.21.15.26:0/2698674793 shutdown_connections 2023-07-13T22:17:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.042+0000 7fa2201ff700 1 -- 172.21.15.26:0/2698674793 wait complete. 2023-07-13T22:17:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.044+0000 7fa2201ff700 1 Processor -- start 2023-07-13T22:17:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.044+0000 7fa2201ff700 1 -- start start 2023-07-13T22:17:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa2181a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa21810bd60 0x7fa2181abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2181ac230 con 0x7fa21810bd60 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21810ec40 con 0x7fa218070ed0 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.045+0000 7fa2201ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21810edd0 con 0x7fa218071820 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:55.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40184/0 (socket says 172.21.15.26:40184) 2023-07-13T22:17:55.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 -- 172.21.15.26:0/376115323 learned_addr learned my addr 172.21.15.26:0/376115323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:55.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21d79a700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa2181a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:55.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 -- 172.21.15.26:0/376115323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 msgr2=0x7fa2181a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa2181a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 -- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa21810bd60 msgr2=0x7fa2181abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa21810bd60 0x7fa2181abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.046+0000 7fa21df9b700 1 -- 172.21.15.26:0/376115323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080095f0 con 0x7fa218070ed0 2023-07-13T22:17:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.047+0000 7fa21df9b700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9a0 tx=0x7fa21400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.047+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214009790 con 0x7fa218070ed0 2023-07-13T22:17:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.047+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21810f0b0 con 0x7fa218070ed0 2023-07-13T22:17:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.048+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa21400cbd0 con 0x7fa218070ed0 2023-07-13T22:17:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.048+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214018680 con 0x7fa218070ed0 2023-07-13T22:17:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.049+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21810f690 con 0x7fa218070ed0 2023-07-13T22:17:55.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.053+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21811a160 con 0x7fa218070ed0 2023-07-13T22:17:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.056+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa21400cd40 con 0x7fa218070ed0 2023-07-13T22:17:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.059+0000 7fa20effd700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 0x7fa2040a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.059+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa214014070 con 0x7fa218070ed0 2023-07-13T22:17:55.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.062+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa2140963d0 con 0x7fa218070ed0 2023-07-13T22:17:55.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.071+0000 7fa21d79a700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 0x7fa2040a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:55.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.075+0000 7fa21d79a700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 0x7fa2040a3750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b2e0 tx=0x7fa208005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:55.191 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:54 smithi117 ceph-mon[129074]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 313 MiB/s wr, 99 op/s 2023-07-13T22:17:55.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:54 smithi121 ceph-mon[129187]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 313 MiB/s wr, 99 op/s 2023-07-13T22:17:55.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.488+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa21810ff00 con 0x7fa218070ed0 2023-07-13T22:17:55.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.489+0000 7fa20effd700 1 -- 172.21.15.26:0/376115323 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fa2140999f0 con 0x7fa218070ed0 2023-07-13T22:17:55.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:17:55.477 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 msgr2=0x7fa2040a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 0x7fa2040a3750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa20800b2e0 tx=0x7fa208005c00 comp rx=0 tx=0).stop 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 msgr2=0x7fa2181a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa21400e9a0 tx=0x7fa21400ed60 comp rx=0 tx=0).stop 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 shutdown_connections 2023-07-13T22:17:55.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2040a12a0 0x7fa2040a3750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa21810bd60 0x7fa2181abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa218070ed0 0x7fa2181a46b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 --2- 172.21.15.26:0/376115323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa218071820 0x7fa2181a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:55.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.491+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 >> 172.21.15.26:0/376115323 conn(0x7fa21806ca40 msgr2=0x7fa21806dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:55.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.492+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 shutdown_connections 2023-07-13T22:17:55.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:55.492+0000 7fa2201ff700 1 -- 172.21.15.26:0/376115323 wait complete. 2023-07-13T22:17:55.481 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:17:55.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:17:56.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:55 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/376115323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:56.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 -- 172.21.15.26:0/1180667988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a4076d80 msgr2=0x7f05a4079230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1180667988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a4076d80 0x7f05a4079230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f059c009e00 tx=0x7f059c00b780 comp rx=0 tx=0).stop 2023-07-13T22:17:56.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 -- 172.21.15.26:0/1180667988 shutdown_connections 2023-07-13T22:17:56.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1180667988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a4076d80 0x7f05a4079230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1180667988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a40710c0 0x7f05a40714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1180667988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4076710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 -- 172.21.15.26:0/1180667988 >> 172.21.15.26:0/1180667988 conn(0x7f05a406c8b0 msgr2=0x7f05a406ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.220+0000 7f05a3fff700 1 -- 172.21.15.26:0/1180667988 shutdown_connections 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.221+0000 7f05a3fff700 1 -- 172.21.15.26:0/1180667988 wait complete. 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.222+0000 7f05a3fff700 1 Processor -- start 2023-07-13T22:17:56.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.222+0000 7f05a3fff700 1 -- start start 2023-07-13T22:17:56.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.223+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.223+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60924/0 (socket says 172.21.15.26:60924) 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a2ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45442/0 (socket says 172.21.15.26:45442) 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a4076d80 0x7f05a41b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a3fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41ae270 con 0x7f05a40710c0 2023-07-13T22:17:56.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a3fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41ae3e0 con 0x7f05a4076d80 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a3fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a41ae5a0 con 0x7f05a4071a10 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.224+0000 7f05a27fc700 1 -- 172.21.15.26:0/1156148298 learned_addr learned my addr 172.21.15.26:0/1156148298 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 -- 172.21.15.26:0/1156148298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a4076d80 msgr2=0x7f05a41b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a4076d80 0x7f05a41b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 -- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 msgr2=0x7f05a40821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 -- 172.21.15.26:0/1156148298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c005e70 con 0x7f05a4071a10 2023-07-13T22:17:56.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.225+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.226+0000 7f058bfff700 1 -- 172.21.15.26:0/1156148298 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800cbd0 con 0x7f05a4071a10 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.226+0000 7f058bfff700 1 -- 172.21.15.26:0/1156148298 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f059800cd30 con 0x7f05a4071a10 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.226+0000 7f058bfff700 1 -- 172.21.15.26:0/1156148298 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980055a0 con 0x7f05a4071a10 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.226+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a41ae880 con 0x7f05a4071a10 2023-07-13T22:17:56.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.226+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a41aed40 con 0x7f05a4071a10 2023-07-13T22:17:56.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.233+0000 7f058bfff700 1 -- 172.21.15.26:0/1156148298 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0598004750 con 0x7f05a4071a10 2023-07-13T22:17:56.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.233+0000 7f05bcee6b80 1 Processor -- start 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.234+0000 7f05bcee6b80 1 -- start start 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.234+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f7f6d0 0x558db2f7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.235+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.235+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.235+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2e84cb0 con 0x558db2f80e10 2023-07-13T22:17:56.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.235+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2ed7a40 con 0x558db2f80020 2023-07-13T22:17:56.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.235+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2ed0ed0 con 0x558db2f7f6d0 2023-07-13T22:17:56.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.238+0000 7f058bfff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 0x7f058c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.238+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.238+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45458/0 (socket says 172.21.15.26:45458) 2023-07-13T22:17:56.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.238+0000 7f05a27fc700 1 -- 172.21.15.26:0/810972136 learned_addr learned my addr 172.21.15.26:0/810972136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.238+0000 7f058bfff700 1 -- 172.21.15.26:0/1156148298 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0598014070 con 0x7f05a4071a10 2023-07-13T22:17:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 -- 172.21.15.26:0/810972136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f7f6d0 msgr2=0x558db2f7fae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f7f6d0 0x558db2f7fae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 -- 172.21.15.26:0/810972136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 msgr2=0x558db2f808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 -- 172.21.15.26:0/810972136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db2cfede0 con 0x558db2f80e10 2023-07-13T22:17:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:17:56.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.239+0000 7f05a27fc700 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f059800c5b0 tx=0x7f0598009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b77c6d03b61255 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.240+0000 7f05a37fe700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 0x7f058c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.240+0000 7f05897fa700 1 -- 172.21.15.26:0/810972136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598099ec0 con 0x558db2f80e10 2023-07-13T22:17:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.240+0000 7f05897fa700 1 -- 172.21.15.26:0/810972136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f059809def0 con 0x558db2f80e10 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.240+0000 7f05897fa700 1 -- 172.21.15.26:0/810972136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059809a380 con 0x558db2f80e10 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 -- 172.21.15.26:0/810972136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 msgr2=0x558db2f812b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f059800c5b0 tx=0x7f0598009740 comp rx=0 tx=0).stop 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 -- 172.21.15.26:0/810972136 shutdown_connections 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f812b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f7f6d0 0x558db2f7fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/810972136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.241+0000 7f05bcee6b80 1 -- 172.21.15.26:0/810972136 >> 172.21.15.26:0/810972136 conn(0x558db2f74ff0 msgr2=0x558db2f7a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.242+0000 7f05a37fe700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 0x7f058c0a37f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f059c00bb50 tx=0x7f059c00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.249+0000 7f05bcee6b80 1 -- 172.21.15.26:0/810972136 shutdown_connections 2023-07-13T22:17:56.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:55 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/376115323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.260+0000 7f05bcee6b80 1 -- 172.21.15.26:0/810972136 wait complete. 2023-07-13T22:17:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.260+0000 7f05bcee6b80 1 -- start start 2023-07-13T22:17:56.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.261+0000 7f05bcee6b80 1 Processor -- start 2023-07-13T22:17:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f7f6d0 0x558db2f893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f90940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2e70ac0 con 0x558db2f80e10 2023-07-13T22:17:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2e707c0 con 0x558db2f7f6d0 2023-07-13T22:17:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2d54920 con 0x558db2f80020 2023-07-13T22:17:56.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.262+0000 7f05bcee6b80 10 client.0 _ll_register_callbacks cb 0x558db2e6e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:56.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.263+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f90940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.264+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.265+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f90940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45470/0 (socket says 172.21.15.26:45470) 2023-07-13T22:17:56.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.265+0000 7f05a37fe700 1 -- 172.21.15.26:0/1546318908 learned_addr learned my addr 172.21.15.26:0/1546318908 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 -- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f7f6d0 msgr2=0x558db2f893c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f7f6d0 0x558db2f893c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 -- 172.21.15.26:0/1546318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 msgr2=0x558db2f90940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f90940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 -- 172.21.15.26:0/1546318908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a41aed40 con 0x558db2f80020 2023-07-13T22:17:56.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.266+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f89900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0598096930 tx=0x7f0598005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc08aa2749b05c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.269+0000 7f0586ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:56.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.269+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059809a320 con 0x558db2f80020 2023-07-13T22:17:56.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.269+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05980a2310 con 0x558db2f80020 2023-07-13T22:17:56.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.269+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558db2e2a4f0 con 0x558db2f80020 2023-07-13T22:17:56.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.270+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05980a1e80 con 0x558db2f80020 2023-07-13T22:17:56.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.272+0000 7f05bcee6b80 10 client.26008 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:56.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.272+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558db2ce7c60 con 0x558db2f80020 2023-07-13T22:17:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.273+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558db2e2a4f0 con 0x558db2f80020 2023-07-13T22:17:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.273+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0598094ad0 con 0x558db2f80020 2023-07-13T22:17:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.274+0000 7f05bcee6b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.274+0000 7f05bcee6b80 10 client.26008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:17:56.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.274+0000 7f05bcee6b80 20 client.26008 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.275+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2fa5c80 0x558db2fa8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.275+0000 7f05bcee6b80 4 client.26008 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.275+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558db2ec9eb0 con 0x558db2fa5c80 2023-07-13T22:17:56.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.275+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2fa5c80 0x558db2fa8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.277+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2fa5c80 0x558db2fa8090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e78ddf92886e5780 server_cookie=6c3927f087f9e4d in_seq=0 out_seq=0 2023-07-13T22:17:56.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.277+0000 7f0586ff5700 10 client.26008 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.278+0000 7f0586ff5700 1 -- 172.21.15.26:0/1546318908 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558db2ec9eb0 con 0x558db2fa5c80 2023-07-13T22:17:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.278+0000 7f0586ff5700 10 client.26008 handle_command_reply: tid=0 2023-07-13T22:17:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.278+0000 7f05bcee6b80 1 client.26008 shutdown 2023-07-13T22:17:56.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 msgr2=0x558db2f89900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f89900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0598096930 tx=0x7f0598005040 comp rx=0 tx=0).stop 2023-07-13T22:17:56.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 shutdown_connections 2023-07-13T22:17:56.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2fa5c80 0x558db2fa8090 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f80e10 0x558db2f90940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f7f6d0 0x558db2f893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1546318908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f89900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 >> 172.21.15.26:0/1546318908 conn(0x558db2f74ff0 msgr2=0x558db2f79b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.279+0000 7f0586ff5700 0 client.26008 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.280+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 shutdown_connections 2023-07-13T22:17:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.280+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1546318908 wait complete. 2023-07-13T22:17:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.280+0000 7f05bcee6b80 20 client.26008 trim_cache size 0 max 16384 2023-07-13T22:17:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.310+0000 7f05bcee6b80 1 Processor -- start 2023-07-13T22:17:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.310+0000 7f05bcee6b80 1 -- start start 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.310+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f88a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.310+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.311+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80e10 0x558db2fac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.311+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2f875f0 con 0x558db2f7f6d0 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.311+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2facef0 con 0x558db2f80020 2023-07-13T22:17:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.311+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2f7b670 con 0x558db2f80e10 2023-07-13T22:17:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.311+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40242/0 (socket says 172.21.15.26:40242) 2023-07-13T22:17:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 -- 172.21.15.26:0/824533647 learned_addr learned my addr 172.21.15.26:0/824533647 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a37fe700 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80e10 0x558db2fac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 -- 172.21.15.26:0/824533647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80e10 msgr2=0x558db2fac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80e10 0x558db2fac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 -- 172.21.15.26:0/824533647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 msgr2=0x558db2f88a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f88a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.312+0000 7f05a27fc700 1 -- 172.21.15.26:0/824533647 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db2d25250 con 0x558db2f80020 2023-07-13T22:17:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.313+0000 7f05a27fc700 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f059809a080 tx=0x7f0598009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac53c8992fe1052e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.315+0000 7f0589ffb700 1 -- 172.21.15.26:0/824533647 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980a2310 con 0x558db2f80020 2023-07-13T22:17:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.315+0000 7f0589ffb700 1 -- 172.21.15.26:0/824533647 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0598094ad0 con 0x558db2f80020 2023-07-13T22:17:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.315+0000 7f0589ffb700 1 -- 172.21.15.26:0/824533647 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980991d0 con 0x558db2f80020 2023-07-13T22:17:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 -- 172.21.15.26:0/824533647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 msgr2=0x558db2f90900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f059809a080 tx=0x7f0598009740 comp rx=0 tx=0).stop 2023-07-13T22:17:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 -- 172.21.15.26:0/824533647 shutdown_connections 2023-07-13T22:17:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f88a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80e10 0x558db2fac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/824533647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80020 0x558db2f90900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.316+0000 7f05bcee6b80 1 -- 172.21.15.26:0/824533647 >> 172.21.15.26:0/824533647 conn(0x558db2f74ff0 msgr2=0x558db2f7a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.317+0000 7f05bcee6b80 1 -- 172.21.15.26:0/824533647 shutdown_connections 2023-07-13T22:17:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.317+0000 7f05bcee6b80 1 -- 172.21.15.26:0/824533647 wait complete. 2023-07-13T22:17:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.318+0000 7f05bcee6b80 1 -- start start 2023-07-13T22:17:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.318+0000 7f05bcee6b80 1 Processor -- start 2023-07-13T22:17:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f8c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80e10 0x558db2f8f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2f8fb20 con 0x558db2f7f6d0 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2f8fc90 con 0x558db2f80e10 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db2f8fe50 con 0x558db2f80020 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.319+0000 7f05bcee6b80 10 client.0 _ll_register_callbacks cb 0x7f0598099d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:56.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.320+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.320+0000 7f05a37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80e10 0x558db2f8f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.320+0000 7f05a27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60960/0 (socket says 172.21.15.26:60960) 2023-07-13T22:17:56.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.320+0000 7f05a27fc700 1 -- 172.21.15.26:0/1054135626 learned_addr learned my addr 172.21.15.26:0/1054135626 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.323+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f8c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 -- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80e10 msgr2=0x558db2f8f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80e10 0x558db2f8f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 -- 172.21.15.26:0/1054135626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 msgr2=0x558db2f8c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f8c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 -- 172.21.15.26:0/1054135626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558db2ce7c60 con 0x558db2f80020 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f05a27fc700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05980a1de0 tx=0x7f0598096330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb3fafe03799ae6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.324+0000 7f0586ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598094c40 con 0x558db2f80020 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05980a2310 con 0x558db2f80020 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f059809def0 con 0x558db2f80020 2023-07-13T22:17:56.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0598016e30 con 0x558db2f80020 2023-07-13T22:17:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f05bcee6b80 10 client.26014 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.325+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558db2cfd2b0 con 0x558db2f80020 2023-07-13T22:17:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f059809def0 con 0x558db2f80020 2023-07-13T22:17:56.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f05980caa70 con 0x558db2f80020 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 10 client.26014 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 20 client.26014 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2ff9e30 0x558db2ffc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 4 client.26014 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:56.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.326+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558db2e8c050 con 0x558db2ff9e30 2023-07-13T22:17:56.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.331+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2ff9e30 0x558db2ffc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.331+0000 7f0586ff5700 10 client.26014 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.331+0000 7f05a2ffd700 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2ff9e30 0x558db2ffc210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de1b5509306de590 server_cookie=b075baab40907a45 in_seq=0 out_seq=0 2023-07-13T22:17:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f0586ff5700 1 -- 172.21.15.26:0/1054135626 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558db2e8c050 con 0x558db2ff9e30 2023-07-13T22:17:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f0586ff5700 10 client.26014 handle_command_reply: tid=0 2023-07-13T22:17:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f05bcee6b80 1 client.26014 shutdown 2023-07-13T22:17:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 msgr2=0x558db2f8c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f05980a1de0 tx=0x7f0598096330 comp rx=0 tx=0).stop 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 shutdown_connections 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.332+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558db2ff9e30 0x558db2ffc210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558db2f7f6d0 0x558db2f8c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558db2f80e10 0x558db2f8f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 --2- 172.21.15.26:0/1054135626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558db2f80020 0x558db2f8c5c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 >> 172.21.15.26:0/1054135626 conn(0x558db2f74ff0 msgr2=0x558db2fa7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f0586ff5700 0 client.26014 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:17:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 shutdown_connections 2023-07-13T22:17:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 1 -- 172.21.15.26:0/1054135626 wait complete. 2023-07-13T22:17:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.333+0000 7f05bcee6b80 20 client.26014 trim_cache size 0 max 16384 2023-07-13T22:17:56.322 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.334+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 msgr2=0x7f058c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.334+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 0x7f058c0a37f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f059c00bb50 tx=0x7f059c00b040 comp rx=0 tx=0).stop 2023-07-13T22:17:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.334+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 msgr2=0x7f05a4082700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.334+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f059800e9c0 tx=0x7f059800ed80 comp rx=0 tx=0).stop 2023-07-13T22:17:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 shutdown_connections 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f058c0a1340 0x7f058c0a37f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05a40710c0 0x7f05a40821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a4076d80 0x7f05a41b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 --2- 172.21.15.26:0/1156148298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05a4071a10 0x7f05a4082700 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 >> 172.21.15.26:0/1156148298 conn(0x7f05a406c8b0 msgr2=0x7f05a406db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 shutdown_connections 2023-07-13T22:17:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:56.335+0000 7f05a3fff700 1 -- 172.21.15.26:0/1156148298 wait complete. 2023-07-13T22:17:56.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:17:56.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:55 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/376115323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:17:56.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:56 smithi026 ceph-mon[95647]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 81 op/s 2023-07-13T22:17:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 -- 172.21.15.26:0/129329559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104071200 msgr2=0x7fc104071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/129329559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104071200 0x7fc104071610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc100009990 tx=0x7fc100009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 -- 172.21.15.26:0/129329559 shutdown_connections 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/129329559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc104079690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/129329559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104071200 0x7fc104071610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/129329559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc104074850 0x7fc104076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.016+0000 7fc10c3ff700 1 -- 172.21.15.26:0/129329559 >> 172.21.15.26:0/129329559 conn(0x7fc10406ca90 msgr2=0x7fc10406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.017+0000 7fc10c3ff700 1 -- 172.21.15.26:0/129329559 shutdown_connections 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.017+0000 7fc10c3ff700 1 -- 172.21.15.26:0/129329559 wait complete. 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 Processor -- start 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 -- start start 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc104071200 0x7fc1040823b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 0x7fc1040828f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc10412f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10412fd70 con 0x7fc104071200 2023-07-13T22:17:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10412fee0 con 0x7fc104074850 2023-07-13T22:17:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.019+0000 7fc10c3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041300a0 con 0x7fc104077210 2023-07-13T22:17:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.020+0000 7fc10999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 0x7fc1040828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.020+0000 7fc10999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 0x7fc1040828f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40282/0 (socket says 172.21.15.26:40282) 2023-07-13T22:17:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.020+0000 7fc10999a700 1 -- 172.21.15.26:0/589843696 learned_addr learned my addr 172.21.15.26:0/589843696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.020+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc10412f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 -- 172.21.15.26:0/589843696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 msgr2=0x7fc1040828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 0x7fc1040828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 -- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc104071200 msgr2=0x7fc1040823b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc104071200 0x7fc1040823b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 -- 172.21.15.26:0/589843696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1000095f0 con 0x7fc104077210 2023-07-13T22:17:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.022+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc10412f7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00efc0 tx=0x7fc0fc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.023+0000 7fc0fb7fe700 1 -- 172.21.15.26:0/589843696 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc018070 con 0x7fc104077210 2023-07-13T22:17:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.023+0000 7fc0fb7fe700 1 -- 172.21.15.26:0/589843696 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0fc00f040 con 0x7fc104077210 2023-07-13T22:17:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.023+0000 7fc0fb7fe700 1 -- 172.21.15.26:0/589843696 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00ce10 con 0x7fc104077210 2023-07-13T22:17:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.023+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1041302c0 con 0x7fc104077210 2023-07-13T22:17:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.023+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc104130800 con 0x7fc104077210 2023-07-13T22:17:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.032+0000 7fc11fdf7b80 1 Processor -- start 2023-07-13T22:17:57.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 -- start start 2023-07-13T22:17:57.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fc4d0 0x555fcf7fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf7fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf701cb0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf754a40 con 0x555fcf7fce20 2023-07-13T22:17:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf74ded0 con 0x555fcf7fdc70 2023-07-13T22:17:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.044+0000 7fc0fb7fe700 1 -- 172.21.15.26:0/589843696 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc0fc004000 con 0x7fc104077210 2023-07-13T22:17:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.047+0000 7fc0fb7fe700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 0x7fc0f00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.047+0000 7fc0fb7fe700 1 -- 172.21.15.26:0/589843696 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0fc0cc8a0 con 0x7fc104077210 2023-07-13T22:17:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.048+0000 7fc10a19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.048+0000 7fc10a19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40294/0 (socket says 172.21.15.26:40294) 2023-07-13T22:17:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.048+0000 7fc10a19b700 1 -- 172.21.15.26:0/1075688613 learned_addr learned my addr 172.21.15.26:0/1075688613 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.048+0000 7fc10999a700 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf7fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.050+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 0x7fc0f00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.052+0000 7fc10a99c700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 0x7fc0f00a3800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0090a0 tx=0x7fc0fc009f70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.052+0000 7fc10a19b700 1 -- 172.21.15.26:0/1075688613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 msgr2=0x555fcf7fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.052+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf7fe0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.053+0000 7fc10a19b700 1 -- 172.21.15.26:0/1075688613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fc4d0 msgr2=0x555fcf7fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.053+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fc4d0 0x555fcf7fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.053+0000 7fc10a19b700 1 -- 172.21.15.26:0/1075688613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fcf57bde0 con 0x555fcf7fce20 2023-07-13T22:17:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.053+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc10000b680 tx=0x7fc100009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1545215b860dce6b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.058+0000 7fc0f8bfe700 1 -- 172.21.15.26:0/1075688613 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10001d070 con 0x555fcf7fce20 2023-07-13T22:17:57.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.058+0000 7fc0f8bfe700 1 -- 172.21.15.26:0/1075688613 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10000bd90 con 0x555fcf7fce20 2023-07-13T22:17:57.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.058+0000 7fc0f8bfe700 1 -- 172.21.15.26:0/1075688613 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100021410 con 0x555fcf7fce20 2023-07-13T22:17:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1075688613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 msgr2=0x555fcf7fd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc10000b680 tx=0x7fc100009550 comp rx=0 tx=0).stop 2023-07-13T22:17:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1075688613 shutdown_connections 2023-07-13T22:17:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fc4d0 0x555fcf7fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf7fe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1075688613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf7fd6e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.064+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1075688613 >> 172.21.15.26:0/1075688613 conn(0x555fcf7f1da0 msgr2=0x555fcf7f71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.065+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1075688613 shutdown_connections 2023-07-13T22:17:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.065+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1075688613 wait complete. 2023-07-13T22:17:57.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.066+0000 7fc11fdf7b80 1 -- start start 2023-07-13T22:17:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.066+0000 7fc11fdf7b80 1 Processor -- start 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf8066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fdc70 0x555fcf80d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf6edac0 con 0x555fcf7fce20 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf6ed7c0 con 0x555fcf7fdc70 2023-07-13T22:17:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf5d1920 con 0x555fcf7fc4d0 2023-07-13T22:17:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.067+0000 7fc11fdf7b80 10 client.0 _ll_register_callbacks cb 0x555fcf6eb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:57.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32780/0 (socket says 172.21.15.26:32780) 2023-07-13T22:17:57.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 -- 172.21.15.26:0/1542228599 learned_addr learned my addr 172.21.15.26:0/1542228599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fdc70 0x555fcf80d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 -- 172.21.15.26:0/1542228599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fdc70 msgr2=0x555fcf80d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fdc70 0x555fcf80d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 -- 172.21.15.26:0/1542228599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 msgr2=0x555fcf8066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf8066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.073+0000 7fc10999a700 1 -- 172.21.15.26:0/1542228599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10407a0c0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.074+0000 7fc10999a700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0f404d5d0 tx=0x7fc0f404c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5766227e878e3f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.094+0000 7fc0edffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.094+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4055b10 con 0x555fcf7fc4d0 2023-07-13T22:17:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.094+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0f4055c70 con 0x555fcf7fc4d0 2023-07-13T22:17:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.095+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fcf6a74f0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.099+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f404db50 con 0x555fcf7fc4d0 2023-07-13T22:17:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.099+0000 7fc11fdf7b80 10 client.26026 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.099+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555fcf564c60 con 0x555fcf7fc4d0 2023-07-13T22:17:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.100+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f406b030 con 0x555fcf7fc4d0 2023-07-13T22:17:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.100+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc0f4064020 con 0x555fcf7fc4d0 2023-07-13T22:17:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 10 client.26026 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 10 client.26026 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 20 client.26026 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf811f60 0x555fcf814370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 4 client.26026 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.101+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fcf746eb0 con 0x555fcf811f60 2023-07-13T22:17:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.102+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf811f60 0x555fcf814370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.108+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf811f60 0x555fcf814370 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=62d9758cda048dc0 server_cookie=f01e8e048b85233d in_seq=0 out_seq=0 2023-07-13T22:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.113+0000 7fc0edffb700 10 client.26026 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.113+0000 7fc0edffb700 1 -- 172.21.15.26:0/1542228599 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fcf746eb0 con 0x555fcf811f60 2023-07-13T22:17:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.113+0000 7fc0edffb700 10 client.26026 handle_command_reply: tid=0 2023-07-13T22:17:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.113+0000 7fc11fdf7b80 1 client.26026 shutdown 2023-07-13T22:17:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 msgr2=0x555fcf806160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0f404d5d0 tx=0x7fc0f404c040 comp rx=0 tx=0).stop 2023-07-13T22:17:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 shutdown_connections 2023-07-13T22:17:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf811f60 0x555fcf814370 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fdc70 0x555fcf80d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf806160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1542228599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf8066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.114+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 >> 172.21.15.26:0/1542228599 conn(0x555fcf7f1da0 msgr2=0x555fcf7f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.115+0000 7fc0edffb700 0 client.26026 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:57.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.123+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 shutdown_connections 2023-07-13T22:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.123+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1542228599 wait complete. 2023-07-13T22:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.124+0000 7fc11fdf7b80 20 client.26026 trim_cache size 0 max 16384 2023-07-13T22:17:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.155+0000 7fc11fdf7b80 1 Processor -- start 2023-07-13T22:17:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.155+0000 7fc11fdf7b80 1 -- start start 2023-07-13T22:17:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf813f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf80d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fdc70 0x555fcf804010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf7f8c60 con 0x555fcf7fdc70 2023-07-13T22:17:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf7fc8a0 con 0x555fcf7fce20 2023-07-13T22:17:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf7f36d0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.156+0000 7fc10999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf813f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.157+0000 7fc10a19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf80d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.157+0000 7fc10a19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf80d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40320/0 (socket says 172.21.15.26:40320) 2023-07-13T22:17:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.157+0000 7fc10a19b700 1 -- 172.21.15.26:0/4189024847 learned_addr learned my addr 172.21.15.26:0/4189024847 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.157+0000 7fc10999a700 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fdc70 0x555fcf804010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 -- 172.21.15.26:0/4189024847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 msgr2=0x555fcf80d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf80d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 -- 172.21.15.26:0/4189024847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fdc70 msgr2=0x555fcf804010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fdc70 0x555fcf804010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 -- 172.21.15.26:0/4189024847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fcf5a2250 con 0x555fcf7fc4d0 2023-07-13T22:17:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc10999a700 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf813f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc0f405afd0 tx=0x7fc0f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=867585e4017eb55f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.158+0000 7fc0f93ff700 1 -- 172.21.15.26:0/4189024847 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f404d600 con 0x555fcf7fc4d0 2023-07-13T22:17:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.159+0000 7fc0f93ff700 1 -- 172.21.15.26:0/4189024847 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f406b070 con 0x555fcf7fc4d0 2023-07-13T22:17:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.159+0000 7fc0f93ff700 1 -- 172.21.15.26:0/4189024847 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40593f0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/4189024847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 msgr2=0x555fcf813f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf813f70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc0f405afd0 tx=0x7fc0f4004750 comp rx=0 tx=0).stop 2023-07-13T22:17:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/4189024847 shutdown_connections 2023-07-13T22:17:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fdc70 0x555fcf804010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fc4d0 0x555fcf813f70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/4189024847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fce20 0x555fcf80d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/4189024847 >> 172.21.15.26:0/4189024847 conn(0x555fcf7f1da0 msgr2=0x555fcf80dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/4189024847 shutdown_connections 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.160+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/4189024847 wait complete. 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.161+0000 7fc11fdf7b80 1 -- start start 2023-07-13T22:17:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.161+0000 7fc11fdf7b80 1 Processor -- start 2023-07-13T22:17:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 0x555fcf8098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf809e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf80ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf80d360 con 0x555fcf7fce20 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf80d4d0 con 0x555fcf7fc4d0 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fcf80d690 con 0x555fcf7fdc70 2023-07-13T22:17:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.163+0000 7fc11fdf7b80 10 client.0 _ll_register_callbacks cb 0x7fc0f405c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.166+0000 7fc10999a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf80ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.166+0000 7fc10999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 0x555fcf8098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.166+0000 7fc10999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 0x555fcf8098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40328/0 (socket says 172.21.15.26:40328) 2023-07-13T22:17:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.166+0000 7fc10999a700 1 -- 172.21.15.26:0/1704925924 learned_addr learned my addr 172.21.15.26:0/1704925924 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf809e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10999a700 1 -- 172.21.15.26:0/1704925924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 msgr2=0x555fcf8098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10999a700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 0x555fcf8098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10999a700 1 -- 172.21.15.26:0/1704925924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 msgr2=0x555fcf809e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10999a700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf809e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.167+0000 7fc10999a700 1 -- 172.21.15.26:0/1704925924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fcf564c60 con 0x555fcf7fdc70 2023-07-13T22:17:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.168+0000 7fc10999a700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf80ce20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0f404afd0 tx=0x7fc0f404d4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6c76c8d627093d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.168+0000 7fc0edffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.168+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4049a60 con 0x555fcf7fdc70 2023-07-13T22:17:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.168+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc0f4000e40 con 0x555fcf7fdc70 2023-07-13T22:17:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.169+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc10000bd90 con 0x555fcf7fdc70 2023-07-13T22:17:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.169+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0f40593f0 con 0x555fcf7fdc70 2023-07-13T22:17:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.170+0000 7fc11fdf7b80 10 client.26032 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.170+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555fcf57a2b0 con 0x555fcf7fdc70 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.170+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f4088030 con 0x555fcf7fdc70 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc0f4055260 con 0x555fcf7fdc70 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 10 client.26032 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 10 client.26032 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 20 client.26032 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:57.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf866e00 0x555fcf869210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 4 client.26032 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.171+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fcf709050 con 0x555fcf866e00 2023-07-13T22:17:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.172+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf866e00 0x555fcf869210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.173+0000 7fc10a19b700 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf866e00 0x555fcf869210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=593aaac1d0580434 server_cookie=dee4c6757fb2f396 in_seq=0 out_seq=0 2023-07-13T22:17:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.173+0000 7fc0edffb700 10 client.26032 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc0edffb700 1 -- 172.21.15.26:0/1704925924 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fcf709050 con 0x555fcf866e00 2023-07-13T22:17:57.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc0edffb700 10 client.26032 handle_command_reply: tid=0 2023-07-13T22:17:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 client.26032 shutdown 2023-07-13T22:17:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 msgr2=0x555fcf80ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf80ce20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc0f404afd0 tx=0x7fc0f404d4e0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 shutdown_connections 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x555fcf866e00 0x555fcf869210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555fcf7fdc70 0x555fcf80ce20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555fcf7fc4d0 0x555fcf8098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 --2- 172.21.15.26:0/1704925924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555fcf7fce20 0x555fcf809e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 >> 172.21.15.26:0/1704925924 conn(0x555fcf7f1da0 msgr2=0x555fcf813f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.174+0000 7fc0edffb700 0 client.26032 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:17:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.175+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 shutdown_connections 2023-07-13T22:17:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.175+0000 7fc11fdf7b80 1 -- 172.21.15.26:0/1704925924 wait complete. 2023-07-13T22:17:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.176+0000 7fc11fdf7b80 20 client.26032 trim_cache size 0 max 16384 2023-07-13T22:17:57.164 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.188+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 msgr2=0x7fc0f00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.188+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 0x7fc0f00a3800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0090a0 tx=0x7fc0fc009f70 comp rx=0 tx=0).stop 2023-07-13T22:17:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.188+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 msgr2=0x7fc10412f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.188+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc10412f7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00efc0 tx=0x7fc0fc00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 shutdown_connections 2023-07-13T22:17:57.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0f00a1350 0x7fc0f00a3800 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc104071200 0x7fc1040823b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc104077210 0x7fc10412f7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 --2- 172.21.15.26:0/589843696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc104074850 0x7fc1040828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.189+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 >> 172.21.15.26:0/589843696 conn(0x7fc10406ca90 msgr2=0x7fc104078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.191+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 shutdown_connections 2023-07-13T22:17:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.191+0000 7fc10c3ff700 1 -- 172.21.15.26:0/589843696 wait complete. 2023-07-13T22:17:57.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:17:57.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:56 smithi121 ceph-mon[129187]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 81 op/s 2023-07-13T22:17:57.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:56 smithi117 ceph-mon[129074]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 260 MiB/s wr, 81 op/s 2023-07-13T22:17:57.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.923+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2466806520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 msgr2=0x7fd5d40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.923+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2466806520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d40712f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b0a0 tx=0x7fd5c400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2466806520 shutdown_connections 2023-07-13T22:17:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2466806520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 0x7fd5d410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2466806520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d40712f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2466806520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 0x7fd5d410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2466806520 >> 172.21.15.26:0/2466806520 conn(0x7fd5d406ca90 msgr2=0x7fd5d406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2466806520 shutdown_connections 2023-07-13T22:17:57.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.924+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2466806520 wait complete. 2023-07-13T22:17:57.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.927+0000 7fd5d8fff700 1 Processor -- start 2023-07-13T22:17:57.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 -- start start 2023-07-13T22:17:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 0x7fd5d41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 0x7fd5d41abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41ac240 con 0x7fd5d410bd60 2023-07-13T22:17:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41ac380 con 0x7fd5d4071830 2023-07-13T22:17:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.928+0000 7fd5d8fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41ac510 con 0x7fd5d4070ee0 2023-07-13T22:17:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32812/0 (socket says 172.21.15.26:32812) 2023-07-13T22:17:57.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 -- 172.21.15.26:0/2202781649 learned_addr learned my addr 172.21.15.26:0/2202781649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 0x7fd5d41abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 0x7fd5d41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 -- 172.21.15.26:0/2202781649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 msgr2=0x7fd5d41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 0x7fd5d41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 -- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 msgr2=0x7fd5d41abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 0x7fd5d41abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.929+0000 7fd5d259c700 1 -- 172.21.15.26:0/2202781649 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4009d00 con 0x7fd5d4070ee0 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b070 tx=0x7fd5c40094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5cbfff700 1 -- 172.21.15.26:0/2202781649 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400e040 con 0x7fd5d4070ee0 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d41ac790 con 0x7fd5d4070ee0 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5cbfff700 1 -- 172.21.15.26:0/2202781649 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5c401d930 con 0x7fd5d4070ee0 2023-07-13T22:17:57.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5cbfff700 1 -- 172.21.15.26:0/2202781649 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c401b990 con 0x7fd5d4070ee0 2023-07-13T22:17:57.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.930+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d41acc90 con 0x7fd5d4070ee0 2023-07-13T22:17:57.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.936+0000 7fd5cbfff700 1 -- 172.21.15.26:0/2202781649 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd5c401daa0 con 0x7fd5d4070ee0 2023-07-13T22:17:57.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.938+0000 7fd5ec9b7b80 1 Processor -- start 2023-07-13T22:17:57.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 -- start start 2023-07-13T22:17:57.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5cbfff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 0x7fd5c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5cbfff700 1 -- 172.21.15.26:0/2202781649 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5c40cdca0 con 0x7fd5d4070ee0 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc4bbcb0 con 0x55fddc5b64d0 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc50ea40 con 0x55fddc5b7c70 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.940+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc507ed0 con 0x55fddc5b6e20 2023-07-13T22:17:57.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.941+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.941+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32820/0 (socket says 172.21.15.26:32820) 2023-07-13T22:17:57.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.941+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3011220087 learned_addr learned my addr 172.21.15.26:0/3011220087 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.942+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3011220087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 msgr2=0x55fddc5b80e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:57.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.942+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5b80e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.942+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3011220087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 msgr2=0x55fddc5b68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:17:57.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.942+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5b68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.942+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3011220087 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fddc335de0 con 0x55fddc5b6e20 2023-07-13T22:17:57.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.943+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00bd20 tx=0x7fd5bc00f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cb9b50c555520d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.943+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 0x7fd5c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.944+0000 7fd5c8d99700 1 -- 172.21.15.26:0/3011220087 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00fe50 con 0x55fddc5b6e20 2023-07-13T22:17:57.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.944+0000 7fd5c8d99700 1 -- 172.21.15.26:0/3011220087 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5bc00be70 con 0x55fddc5b6e20 2023-07-13T22:17:57.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.944+0000 7fd5c8d99700 1 -- 172.21.15.26:0/3011220087 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0107c0 con 0x55fddc5b6e20 2023-07-13T22:17:57.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3011220087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 msgr2=0x55fddc5b76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00bd20 tx=0x7fd5bc00f640 comp rx=0 tx=0).stop 2023-07-13T22:17:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 0x7fd5c00a36d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006b30 tx=0x7fd5cc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3011220087 shutdown_connections 2023-07-13T22:17:57.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3011220087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5b76e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.945+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3011220087 >> 172.21.15.26:0/3011220087 conn(0x55fddc5abda0 msgr2=0x55fddc5b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.946+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3011220087 shutdown_connections 2023-07-13T22:17:57.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.946+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3011220087 wait complete. 2023-07-13T22:17:57.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.946+0000 7fd5ec9b7b80 1 -- start start 2023-07-13T22:17:57.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.947+0000 7fd5ec9b7b80 1 Processor -- start 2023-07-13T22:17:57.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.947+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5c0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.947+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.947+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc4a7ac0 con 0x55fddc5b7c70 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc4a77c0 con 0x55fddc5b64d0 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc38b920 con 0x55fddc5b6e20 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5ec9b7b80 10 client.0 _ll_register_callbacks cb 0x55fddc4a5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32836/0 (socket says 172.21.15.26:32836) 2023-07-13T22:17:57.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3712750034 learned_addr learned my addr 172.21.15.26:0/3712750034 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.948+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5d259c700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5c0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 msgr2=0x55fddc5c0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5c0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3712750034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 msgr2=0x55fddc5c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/3712750034 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc00b980 con 0x55fddc5b6e20 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.949+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005910 tx=0x7fd5bc00fe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3de8631f4fe68868 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc027070 con 0x55fddc5b6e20 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5bc005410 con 0x55fddc5b6e20 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fddc4614f0 con 0x55fddc5b6e20 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5bc01e070 con 0x55fddc5b6e20 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5ec9b7b80 10 client.26044 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:57.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.950+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fddc31ec60 con 0x55fddc5b6e20 2023-07-13T22:17:57.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fddc4614f0 con 0x55fddc5b6e20 2023-07-13T22:17:57.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5bc02a030 con 0x55fddc5b6e20 2023-07-13T22:17:57.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:57.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 10 client.26044 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:17:57.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 20 client.26044 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:57.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc5cbea0 0x55fddc5ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 4 client.26044 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:57.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.951+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fddc500eb0 con 0x55fddc5cbea0 2023-07-13T22:17:57.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.958+0000 7fd5d259c700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc5cbea0 0x55fddc5ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.959+0000 7fd5d259c700 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc5cbea0 0x55fddc5ce2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36fd7e389cde2189 server_cookie=d428e7b44267c1ca in_seq=0 out_seq=0 2023-07-13T22:17:57.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.959+0000 7fd5b9ffb700 10 client.26044 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:57.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.960+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/3712750034 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fddc500eb0 con 0x55fddc5cbea0 2023-07-13T22:17:57.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.960+0000 7fd5b9ffb700 10 client.26044 handle_command_reply: tid=0 2023-07-13T22:17:57.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.960+0000 7fd5ec9b7b80 1 client.26044 shutdown 2023-07-13T22:17:57.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.960+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 msgr2=0x55fddc5c0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005910 tx=0x7fd5bc00fe80 comp rx=0 tx=0).stop 2023-07-13T22:17:57.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 shutdown_connections 2023-07-13T22:17:57.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc5cbea0 0x55fddc5ce2b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5c0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/3712750034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c0750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 >> 172.21.15.26:0/3712750034 conn(0x55fddc5abda0 msgr2=0x55fddc5b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5b9ffb700 0 client.26044 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 shutdown_connections 2023-07-13T22:17:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.961+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/3712750034 wait complete. 2023-07-13T22:17:57.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.962+0000 7fd5ec9b7b80 20 client.26044 trim_cache size 0 max 16384 2023-07-13T22:17:57.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.991+0000 7fd5ec9b7b80 1 Processor -- start 2023-07-13T22:17:57.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.991+0000 7fd5ec9b7b80 1 -- start start 2023-07-13T22:17:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5bded0 con 0x55fddc5b7c70 2023-07-13T22:17:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5cbbf0 con 0x55fddc5b64d0 2023-07-13T22:17:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.992+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5b8400 con 0x55fddc5b6e20 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.993+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.993+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32844/0 (socket says 172.21.15.26:32844) 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.993+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/769941674 learned_addr learned my addr 172.21.15.26:0/769941674 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.994+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.994+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/769941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 msgr2=0x55fddc5bf8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.994+0000 7fd5d259c700 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.994+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.995+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/769941674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 msgr2=0x55fddc5c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.995+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.995+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/769941674 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fddc35c250 con 0x55fddc5b6e20 2023-07-13T22:17:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.995+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0183a0 tx=0x7fd5bc00f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=304844c6f0166b59 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.996+0000 7fd5c959a700 1 -- 172.21.15.26:0/769941674 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc027070 con 0x55fddc5b6e20 2023-07-13T22:17:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.996+0000 7fd5c959a700 1 -- 172.21.15.26:0/769941674 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5bc01f040 con 0x55fddc5b6e20 2023-07-13T22:17:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.996+0000 7fd5c959a700 1 -- 172.21.15.26:0/769941674 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc01e030 con 0x55fddc5b6e20 2023-07-13T22:17:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/769941674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 msgr2=0x55fddc5c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0183a0 tx=0x7fd5bc00f640 comp rx=0 tx=0).stop 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/769941674 shutdown_connections 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b7c70 0x55fddc5c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b64d0 0x55fddc5bf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/769941674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c76b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.997+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/769941674 >> 172.21.15.26:0/769941674 conn(0x55fddc5abda0 msgr2=0x55fddc5d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.998+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/769941674 shutdown_connections 2023-07-13T22:17:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.998+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/769941674 wait complete. 2023-07-13T22:17:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.998+0000 7fd5ec9b7b80 1 -- start start 2023-07-13T22:17:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.999+0000 7fd5ec9b7b80 1 Processor -- start 2023-07-13T22:17:57.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.999+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.999+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:57.999+0000 7fd5ec9b7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.000+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5c5dc0 con 0x55fddc5b64d0 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.000+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5c5f30 con 0x55fddc5b7c70 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.000+0000 7fd5ec9b7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fddc5c60f0 con 0x55fddc5b6e20 2023-07-13T22:17:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.000+0000 7fd5ec9b7b80 10 client.0 _ll_register_callbacks cb 0x7fd5bc00bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:57.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5c9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32852/0 (socket says 172.21.15.26:32852) 2023-07-13T22:17:57.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/2310002482 learned_addr learned my addr 172.21.15.26:0/2310002482 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:57.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5d2d9d700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 msgr2=0x55fddc5c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.001+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.002+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/2310002482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 msgr2=0x55fddc5c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:57.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.002+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.002+0000 7fd5c9d9b700 1 -- 172.21.15.26:0/2310002482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fddc31ec60 con 0x55fddc5b6e20 2023-07-13T22:17:57.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.002+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:17:57.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.002+0000 7fd5c9d9b700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005a80 tx=0x7fd5bc005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4a71331173248b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:57.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.003+0000 7fd5b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.003+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc01e070 con 0x55fddc5b6e20 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.003+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd5bc000e40 con 0x55fddc5b6e20 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.003+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5bc00be70 con 0x55fddc5b6e20 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.004+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5bc027040 con 0x55fddc5b6e20 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.004+0000 7fd5ec9b7b80 10 client.26050 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:57.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.004+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fddc3342b0 con 0x55fddc5b6e20 2023-07-13T22:17:57.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.004+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5bc00be70 con 0x55fddc5b6e20 2023-07-13T22:17:57.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd5bc00f760 con 0x55fddc5b6e20 2023-07-13T22:17:57.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:57.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 10 client.26050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:17:57.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 20 client.26050 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:57.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc620c20 0x55fddc623000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:57.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 4 client.26050 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:57.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.006+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fddc4c3050 con 0x55fddc620c20 2023-07-13T22:17:57.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.007+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc620c20 0x55fddc623000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:57.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.008+0000 7fd5d259c700 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc620c20 0x55fddc623000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deeaf9612d8c6fd5 server_cookie=e846acfccef6f6e1 in_seq=0 out_seq=0 2023-07-13T22:17:57.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.008+0000 7fd5b9ffb700 10 client.26050 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:57.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.008+0000 7fd5b9ffb700 1 -- 172.21.15.26:0/2310002482 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fddc4c3050 con 0x55fddc620c20 2023-07-13T22:17:57.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.008+0000 7fd5b9ffb700 10 client.26050 handle_command_reply: tid=0 2023-07-13T22:17:57.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.008+0000 7fd5ec9b7b80 1 client.26050 shutdown 2023-07-13T22:17:57.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 msgr2=0x55fddc5c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005a80 tx=0x7fd5bc005640 comp rx=0 tx=0).stop 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 shutdown_connections 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fddc5b64d0 0x55fddc5c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55fddc620c20 0x55fddc623000 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fddc5b7c70 0x55fddc5c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 --2- 172.21.15.26:0/2310002482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fddc5b6e20 0x55fddc5c7610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:57.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 >> 172.21.15.26:0/2310002482 conn(0x55fddc5abda0 msgr2=0x55fddc5c8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:57.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.009+0000 7fd5b9ffb700 0 client.26050 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:17:57.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.010+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 shutdown_connections 2023-07-13T22:17:57.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.010+0000 7fd5ec9b7b80 1 -- 172.21.15.26:0/2310002482 wait complete. 2023-07-13T22:17:57.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.010+0000 7fd5ec9b7b80 20 client.26050 trim_cache size 0 max 16384 2023-07-13T22:17:57.999 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:57.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 msgr2=0x7fd5c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:57.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 0x7fd5c00a36d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc006b30 tx=0x7fd5cc008040 comp rx=0 tx=0).stop 2023-07-13T22:17:58.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 msgr2=0x7fd5d41a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b070 tx=0x7fd5c40094f0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 shutdown_connections 2023-07-13T22:17:58.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd5c00a1220 0x7fd5c00a36d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5d410bd60 0x7fd5d41abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5d4070ee0 0x7fd5d41a46c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 --2- 172.21.15.26:0/2202781649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5d4071830 0x7fd5d41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.012+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 >> 172.21.15.26:0/2202781649 conn(0x7fd5d406ca90 msgr2=0x7fd5d406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.013+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 shutdown_connections 2023-07-13T22:17:58.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.013+0000 7fd5d8fff700 1 -- 172.21.15.26:0/2202781649 wait complete. 2023-07-13T22:17:58.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:17:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.808+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/395062719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04070ee0 msgr2=0x7f4b040712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.808+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/395062719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04070ee0 0x7f4b040712f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009990 tx=0x7f4b00009ca0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.819+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/395062719 shutdown_connections 2023-07-13T22:17:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.819+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/395062719 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b0410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.819+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/395062719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04070ee0 0x7f4b040712f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.819+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/395062719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04071830 0x7f4b0410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.819+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/395062719 >> 172.21.15.26:0/395062719 conn(0x7f4b0406ca90 msgr2=0x7f4b0406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:58.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.830+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/395062719 shutdown_connections 2023-07-13T22:17:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.830+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/395062719 wait complete. 2023-07-13T22:17:58.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.832+0000 7f4b0b3ff700 1 Processor -- start 2023-07-13T22:17:58.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.832+0000 7f4b0b3ff700 1 -- start start 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.832+0000 7f4b0b3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04070ee0 0x7f4b041a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.833+0000 7f4b0b3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04071830 0x7f4b041a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.833+0000 7f4b0b3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.833+0000 7f4b0b3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b041ac240 con 0x7f4b0410bd60 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.833+0000 7f4b0b3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0410ec40 con 0x7f4b04070ee0 2023-07-13T22:17:58.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.833+0000 7f4b0b3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0410edd0 con 0x7f4b04071830 2023-07-13T22:17:58.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.834+0000 7f4b0999c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.835+0000 7f4b0999c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45566/0 (socket says 172.21.15.26:45566) 2023-07-13T22:17:58.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.835+0000 7f4b0999c700 1 -- 172.21.15.26:0/2693742699 learned_addr learned my addr 172.21.15.26:0/2693742699 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.840+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04070ee0 0x7f4b041a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.840+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04071830 0x7f4b041a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.844+0000 7f4b0999c700 1 -- 172.21.15.26:0/2693742699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04071830 msgr2=0x7f4b041a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.844+0000 7f4b0999c700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04071830 0x7f4b041a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.844+0000 7f4b0999c700 1 -- 172.21.15.26:0/2693742699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04070ee0 msgr2=0x7f4b041a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.844+0000 7f4b0999c700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04070ee0 0x7f4b041a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.844+0000 7f4b0999c700 1 -- 172.21.15.26:0/2693742699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000095f0 con 0x7f4b0410bd60 2023-07-13T22:17:58.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.848+0000 7f4b0999c700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b970 tx=0x7f4afc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:58.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.853+0000 7f4afa7fc700 1 -- 172.21.15.26:0/2693742699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00c9b0 con 0x7f4b0410bd60 2023-07-13T22:17:58.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.853+0000 7f4afa7fc700 1 -- 172.21.15.26:0/2693742699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4afc014440 con 0x7f4b0410bd60 2023-07-13T22:17:58.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.853+0000 7f4afa7fc700 1 -- 172.21.15.26:0/2693742699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc0075d0 con 0x7f4b0410bd60 2023-07-13T22:17:58.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.854+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0410f0b0 con 0x7f4b0410bd60 2023-07-13T22:17:58.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.854+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0410f570 con 0x7f4b0410bd60 2023-07-13T22:17:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.870+0000 7f4b1edc1b80 1 Processor -- start 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.870+0000 7f4afa7fc700 1 -- 172.21.15.26:0/2693742699 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4afc00cb10 con 0x7f4b0410bd60 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.870+0000 7f4b1edc1b80 1 -- start start 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.870+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf04fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.871+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf04fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.872+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf04fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.872+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf0400cb0 con 0x55dcf04fcc70 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.872+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf0453a40 con 0x55dcf04fbe20 2023-07-13T22:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.872+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf044ced0 con 0x55dcf04fb4d0 2023-07-13T22:17:58.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.873+0000 7f4afa7fc700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 0x7f4af00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.873+0000 7f4afa7fc700 1 -- 172.21.15.26:0/2693742699 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4afc006e40 con 0x7f4b0410bd60 2023-07-13T22:17:58.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.875+0000 7f4b0999c700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 0x7f4af00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.880+0000 7f4b0999c700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 0x7f4af00a3750 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0a0ed0 tx=0x7f4afc004be0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:58.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0919b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf04fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf04fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf04fb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32884/0 (socket says 172.21.15.26:32884) 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0899a700 1 -- 172.21.15.26:0/4223194989 learned_addr learned my addr 172.21.15.26:0/4223194989 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0919b700 1 -- 172.21.15.26:0/4223194989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 msgr2=0x55dcf04fb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0919b700 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf04fb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.882+0000 7f4b0919b700 1 -- 172.21.15.26:0/4223194989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 msgr2=0x55dcf04fd0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.883+0000 7f4b0899a700 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf04fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.883+0000 7f4b0919b700 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf04fd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.883+0000 7f4b0919b700 1 -- 172.21.15.26:0/4223194989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf027ade0 con 0x55dcf04fbe20 2023-07-13T22:17:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.892+0000 7f4b0919b700 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf04fc6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b00003720 tx=0x7f4b00009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9682fe4d0a014f30 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.893+0000 7f4aef7fe700 1 -- 172.21.15.26:0/4223194989 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000b9d0 con 0x55dcf04fbe20 2023-07-13T22:17:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.893+0000 7f4aef7fe700 1 -- 172.21.15.26:0/4223194989 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b0001e040 con 0x55dcf04fbe20 2023-07-13T22:17:58.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.893+0000 7f4aef7fe700 1 -- 172.21.15.26:0/4223194989 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00004cd0 con 0x55dcf04fbe20 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.894+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/4223194989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 msgr2=0x55dcf04fc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.894+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf04fc6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b00003720 tx=0x7f4b00009f00 comp rx=0 tx=0).stop 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/4223194989 shutdown_connections 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf04fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf04fb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/4223194989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf04fc6e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/4223194989 >> 172.21.15.26:0/4223194989 conn(0x55dcf04f0da0 msgr2=0x55dcf04f61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:58.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.902+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/4223194989 shutdown_connections 2023-07-13T22:17:58.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.912+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/4223194989 wait complete. 2023-07-13T22:17:58.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.912+0000 7f4b1edc1b80 1 -- start start 2023-07-13T22:17:58.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.913+0000 7f4b1edc1b80 1 Processor -- start 2023-07-13T22:17:58.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fbe20 0x55dcf0505690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf03ecac0 con 0x55dcf04fcc70 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf03ec7c0 con 0x55dcf04fb4d0 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf02d0920 con 0x55dcf04fbe20 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b1edc1b80 10 client.0 _ll_register_callbacks cb 0x55dcf03ea760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:58.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.914+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40410/0 (socket says 172.21.15.26:40410) 2023-07-13T22:17:58.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 -- 172.21.15.26:0/2863397831 learned_addr learned my addr 172.21.15.26:0/2863397831 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:58.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fbe20 0x55dcf0505690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 -- 172.21.15.26:0/2863397831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fbe20 msgr2=0x55dcf0505690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fbe20 0x55dcf0505690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 -- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 msgr2=0x55dcf050c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.915+0000 7f4b0899a700 1 -- 172.21.15.26:0/2863397831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b04117640 con 0x55dcf04fb4d0 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.916+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4af40139a0 tx=0x7f4af4013cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9c92593827e47d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.916+0000 7f4aecff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:58.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.916+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4011ce0 con 0x55dcf04fb4d0 2023-07-13T22:17:58.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.916+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4af4011e40 con 0x55dcf04fb4d0 2023-07-13T22:17:58.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.917+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcf03a64f0 con 0x55dcf04fb4d0 2023-07-13T22:17:58.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.920+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af401c440 con 0x55dcf04fb4d0 2023-07-13T22:17:58.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.920+0000 7f4b1edc1b80 10 client.26075 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:58.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.920+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dcf0263c60 con 0x55dcf04fb4d0 2023-07-13T22:17:58.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.921+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dcf03a64f0 con 0x55dcf04fb4d0 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4af4026670 con 0x55dcf04fb4d0 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 10 client.26075 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 20 client.26075 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0510ef0 0x55dcf0513300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 4 client.26075 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:58.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.922+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcf0445eb0 con 0x55dcf0510ef0 2023-07-13T22:17:58.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.923+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0510ef0 0x55dcf0513300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.924+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0510ef0 0x55dcf0513300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7bef2e573810438 server_cookie=640910e78e470846 in_seq=0 out_seq=0 2023-07-13T22:17:58.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.924+0000 7f4aecff9700 10 client.26075 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.926+0000 7f4aecff9700 1 -- 172.21.15.26:0/2863397831 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dcf0445eb0 con 0x55dcf0510ef0 2023-07-13T22:17:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.926+0000 7f4aecff9700 10 client.26075 handle_command_reply: tid=0 2023-07-13T22:17:58.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.926+0000 7f4b1edc1b80 1 client.26075 shutdown 2023-07-13T22:17:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.926+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 msgr2=0x55dcf0505150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.926+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4af40139a0 tx=0x7f4af4013cb0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 shutdown_connections 2023-07-13T22:17:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0510ef0 0x55dcf0513300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fb4d0 0x55dcf0505150 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2863397831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fbe20 0x55dcf0505690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 >> 172.21.15.26:0/2863397831 conn(0x55dcf04f0da0 msgr2=0x55dcf04f58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:58.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4aecff9700 0 client.26075 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 shutdown_connections 2023-07-13T22:17:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.927+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2863397831 wait complete. 2023-07-13T22:17:58.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.928+0000 7f4b1edc1b80 20 client.26075 trim_cache size 0 max 16384 2023-07-13T22:17:58.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.969+0000 7f4b1edc1b80 1 Processor -- start 2023-07-13T22:17:58.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 -- start start 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fb4d0 0x55dcf0502fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fcc70 0x55dcf050c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf04fc570 con 0x55dcf04fb4d0 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf04f2ec0 con 0x55dcf04fbe20 2023-07-13T22:17:58.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.974+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf04f7a00 con 0x55dcf04fcc70 2023-07-13T22:17:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.985+0000 7f4b0919b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.985+0000 7f4b0919b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40424/0 (socket says 172.21.15.26:40424) 2023-07-13T22:17:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.985+0000 7f4b0919b700 1 -- 172.21.15.26:0/1332452891 learned_addr learned my addr 172.21.15.26:0/1332452891 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.985+0000 7f4b0899a700 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fcc70 0x55dcf050c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.985+0000 7f4b0899a700 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fb4d0 0x55dcf0502fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 -- 172.21.15.26:0/1332452891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fcc70 msgr2=0x55dcf050c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fcc70 0x55dcf050c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 -- 172.21.15.26:0/1332452891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fb4d0 msgr2=0x55dcf0502fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fb4d0 0x55dcf0502fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 -- 172.21.15.26:0/1332452891 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcf02a1250 con 0x55dcf04fbe20 2023-07-13T22:17:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.986+0000 7f4b0919b700 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009990 tx=0x7f4b00005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab44efaf48969274 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:58.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.998+0000 7f4aeffff700 1 -- 172.21.15.26:0/1332452891 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00032b60 con 0x55dcf04fbe20 2023-07-13T22:17:58.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.998+0000 7f4aeffff700 1 -- 172.21.15.26:0/1332452891 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b00032cc0 con 0x55dcf04fbe20 2023-07-13T22:17:58.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:58.998+0000 7f4aeffff700 1 -- 172.21.15.26:0/1332452891 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000fa80 con 0x55dcf04fbe20 2023-07-13T22:17:58.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.006+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/1332452891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 msgr2=0x55dcf0503510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:58.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.006+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009990 tx=0x7f4b00005940 comp rx=0 tx=0).stop 2023-07-13T22:17:58.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.009+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/1332452891 shutdown_connections 2023-07-13T22:17:58.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.009+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fb4d0 0x55dcf0502fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.009+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fcc70 0x55dcf050c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.009+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/1332452891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0503510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:58.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.009+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/1332452891 >> 172.21.15.26:0/1332452891 conn(0x55dcf04f0da0 msgr2=0x55dcf0512f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.022+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/1332452891 shutdown_connections 2023-07-13T22:17:59.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.023+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/1332452891 wait complete. 2023-07-13T22:17:59.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.023+0000 7f4b1edc1b80 1 -- start start 2023-07-13T22:17:59.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 Processor -- start 2023-07-13T22:17:59.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0508930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf050be90 con 0x55dcf04fcc70 2023-07-13T22:17:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf050c000 con 0x55dcf04fbe20 2023-07-13T22:17:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcf050c1c0 con 0x55dcf04fb4d0 2023-07-13T22:17:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.024+0000 7f4b1edc1b80 10 client.0 _ll_register_callbacks cb 0x7f4af400ce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.026+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.026+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.027+0000 7f4b0899a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32904/0 (socket says 172.21.15.26:32904) 2023-07-13T22:17:59.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.027+0000 7f4b0899a700 1 -- 172.21.15.26:0/2989846713 learned_addr learned my addr 172.21.15.26:0/2989846713 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.032+0000 7f4b0899a700 1 -- 172.21.15.26:0/2989846713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 msgr2=0x55dcf0508930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:59.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.032+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0508930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.032+0000 7f4b0899a700 1 -- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 msgr2=0x55dcf050b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:59.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.032+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.032+0000 7f4b0899a700 1 -- 172.21.15.26:0/2989846713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b00027040 con 0x55dcf04fb4d0 2023-07-13T22:17:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.048+0000 7f4b0899a700 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4af4016040 tx=0x7f4af4013cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8d5b79d0abfd94a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.048+0000 7f4aecff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.065+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4004230 con 0x55dcf04fb4d0 2023-07-13T22:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.065+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4af4000e40 con 0x55dcf04fb4d0 2023-07-13T22:17:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.066+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b0001e040 con 0x55dcf04fb4d0 2023-07-13T22:17:59.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.069+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af4005cd0 con 0x55dcf04fb4d0 2023-07-13T22:17:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.072+0000 7f4b1edc1b80 10 client.26068 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:59.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.072+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dcf02792b0 con 0x55dcf04fb4d0 2023-07-13T22:17:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.076+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b0001e040 con 0x55dcf04fb4d0 2023-07-13T22:17:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.077+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4af4004850 con 0x55dcf04fb4d0 2023-07-13T22:17:59.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.077+0000 7f4b1edc1b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.078+0000 7f4b1edc1b80 10 client.26068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:17:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.078+0000 7f4b1edc1b80 20 client.26068 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.078+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0565a60 0x55dcf0567e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.078+0000 7f4b1edc1b80 4 client.26068 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.078+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcf0408050 con 0x55dcf0565a60 2023-07-13T22:17:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.084+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0565a60 0x55dcf0567e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.085+0000 7f4b0919b700 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0565a60 0x55dcf0567e70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38d9ea5671e8be1e server_cookie=5e8e715e393146d in_seq=0 out_seq=0 2023-07-13T22:17:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.085+0000 7f4aecff9700 10 client.26068 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.096+0000 7f4aecff9700 1 -- 172.21.15.26:0/2989846713 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcf0408050 con 0x55dcf0565a60 2023-07-13T22:17:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.096+0000 7f4aecff9700 10 client.26068 handle_command_reply: tid=0 2023-07-13T22:17:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.096+0000 7f4b1edc1b80 1 client.26068 shutdown 2023-07-13T22:17:59.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.096+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 msgr2=0x55dcf05083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.096+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4af4016040 tx=0x7f4af4013cb0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 shutdown_connections 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dcf0565a60 0x55dcf0567e70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dcf04fcc70 0x55dcf050b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dcf04fb4d0 0x55dcf05083f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 --2- 172.21.15.26:0/2989846713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcf04fbe20 0x55dcf0508930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.102+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 >> 172.21.15.26:0/2989846713 conn(0x55dcf04f0da0 msgr2=0x55dcf0502fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.103+0000 7f4aecff9700 0 client.26068 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:17:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.103+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 shutdown_connections 2023-07-13T22:17:59.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.104+0000 7f4b1edc1b80 1 -- 172.21.15.26:0/2989846713 wait complete. 2023-07-13T22:17:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.104+0000 7f4b1edc1b80 20 client.26068 trim_cache size 0 max 16384 2023-07-13T22:17:59.093 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.110+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 msgr2=0x7f4af00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.110+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 0x7f4af00a3750 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0a0ed0 tx=0x7f4afc004be0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.110+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 msgr2=0x7f4b041abc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.110+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b970 tx=0x7f4afc00bd30 comp rx=0 tx=0).stop 2023-07-13T22:17:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 shutdown_connections 2023-07-13T22:17:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4af00a12a0 0x7f4af00a3750 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b0410bd60 0x7f4b041abc70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b04070ee0 0x7f4b041a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 --2- 172.21.15.26:0/2693742699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b04071830 0x7f4b041a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.111+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 >> 172.21.15.26:0/2693742699 conn(0x7f4b0406ca90 msgr2=0x7f4b0406dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.115+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 shutdown_connections 2023-07-13T22:17:59.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.115+0000 7f4b0b3ff700 1 -- 172.21.15.26:0/2693742699 wait complete. 2023-07-13T22:17:59.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:17:59.381 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:17:59 smithi026 ceph-mon[95647]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 86 op/s 2023-07-13T22:17:59.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:17:59 smithi117 ceph-mon[129074]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 86 op/s 2023-07-13T22:17:59.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:17:59 smithi121 ceph-mon[129187]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 271 MiB/s wr, 86 op/s 2023-07-13T22:17:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.705+0000 7f4f0fdff700 1 -- 172.21.15.26:0/119822414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 msgr2=0x7f4f080712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.705+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/119822414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 0x7f4f080712f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000b0a0 tx=0x7f4f0000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.719+0000 7f4f0fdff700 1 -- 172.21.15.26:0/119822414 shutdown_connections 2023-07-13T22:17:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.719+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/119822414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f0810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.719+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/119822414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 0x7f4f080712f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.719+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/119822414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f08071830 0x7f4f0810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.719+0000 7f4f0fdff700 1 -- 172.21.15.26:0/119822414 >> 172.21.15.26:0/119822414 conn(0x7f4f0806ca90 msgr2=0x7f4f0806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.720+0000 7f4f0fdff700 1 -- 172.21.15.26:0/119822414 shutdown_connections 2023-07-13T22:17:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.720+0000 7f4f0fdff700 1 -- 172.21.15.26:0/119822414 wait complete. 2023-07-13T22:17:59.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.722+0000 7f4f0fdff700 1 Processor -- start 2023-07-13T22:17:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.722+0000 7f4f0fdff700 1 -- start start 2023-07-13T22:17:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.722+0000 7f4f0fdff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 0x7f4f08113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.723+0000 7f4f0fdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f08071830 0x7f4f081140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.723+0000 7f4f0fdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.723+0000 7f4f0fdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081146c0 con 0x7f4f0810bd60 2023-07-13T22:17:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.723+0000 7f4f0fdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08114830 con 0x7f4f08071830 2023-07-13T22:17:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.723+0000 7f4f0fdff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081b4ea0 con 0x7f4f08070ee0 2023-07-13T22:17:59.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.724+0000 7f4f0e39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.724+0000 7f4f0e39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45620/0 (socket says 172.21.15.26:45620) 2023-07-13T22:17:59.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.724+0000 7f4f0e39c700 1 -- 172.21.15.26:0/339693589 learned_addr learned my addr 172.21.15.26:0/339693589 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.725+0000 7f4f0e39c700 1 -- 172.21.15.26:0/339693589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 msgr2=0x7f4f08113bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:17:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.725+0000 7f4f0e39c700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 0x7f4f08113bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.725+0000 7f4f0e39c700 1 -- 172.21.15.26:0/339693589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f08071830 msgr2=0x7f4f081140f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:17:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.725+0000 7f4f0e39c700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f08071830 0x7f4f081140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.725+0000 7f4f0e39c700 1 -- 172.21.15.26:0/339693589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f00009d00 con 0x7f4f0810bd60 2023-07-13T22:17:59.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.726+0000 7f4f0e39c700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b550 tx=0x7f4f0400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.727+0000 7f4efeffd700 1 -- 172.21.15.26:0/339693589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04011830 con 0x7f4f0810bd60 2023-07-13T22:17:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.727+0000 7f4efeffd700 1 -- 172.21.15.26:0/339693589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f04011e50 con 0x7f4f0810bd60 2023-07-13T22:17:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.727+0000 7f4efeffd700 1 -- 172.21.15.26:0/339693589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f04010ea0 con 0x7f4f0810bd60 2023-07-13T22:17:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.727+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f081b5180 con 0x7f4f0810bd60 2023-07-13T22:17:59.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.727+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f081b56c0 con 0x7f4f0810bd60 2023-07-13T22:17:59.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.738+0000 7f4f237bfb80 1 Processor -- start 2023-07-13T22:17:59.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.739+0000 7f4efeffd700 1 -- 172.21.15.26:0/339693589 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f04011990 con 0x7f4f0810bd60 2023-07-13T22:17:59.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.742+0000 7f4efeffd700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 0x7f4ef40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.743+0000 7f4efeffd700 1 -- 172.21.15.26:0/339693589 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f040cc240 con 0x7f4f0810bd60 2023-07-13T22:17:59.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 -- start start 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645f4d0 0x55beb645f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb64610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6364cb0 con 0x55beb645f4d0 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb63b7a40 con 0x55beb645fe20 2023-07-13T22:17:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.744+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb63b0ed0 con 0x55beb6460c70 2023-07-13T22:17:59.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.745+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.745+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 0x7f4ef40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.745+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40468/0 (socket says 172.21.15.26:40468) 2023-07-13T22:17:59.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.745+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3278228370 learned_addr learned my addr 172.21.15.26:0/3278228370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3278228370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 msgr2=0x55beb64610e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:59.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb64610e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3278228370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645f4d0 msgr2=0x55beb645f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645f4d0 0x55beb645f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3278228370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb61dede0 con 0x55beb645fe20 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.746+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80137b0 tx=0x7f4ef8013ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4ca67db15e0de9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.747+0000 7f4ef3fff700 1 -- 172.21.15.26:0/3278228370 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8018070 con 0x55beb645fe20 2023-07-13T22:17:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.747+0000 7f4ef3fff700 1 -- 172.21.15.26:0/3278228370 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef80130d0 con 0x55beb645fe20 2023-07-13T22:17:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.747+0000 7f4ef3fff700 1 -- 172.21.15.26:0/3278228370 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80117c0 con 0x55beb645fe20 2023-07-13T22:17:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.748+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3278228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 msgr2=0x55beb64606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.748+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80137b0 tx=0x7f4ef8013ac0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.749+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 0x7f4ef40a3800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013fd0 tx=0x7f4ef801d420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.750+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3278228370 shutdown_connections 2023-07-13T22:17:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.750+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645f4d0 0x55beb645f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.750+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb64610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.750+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3278228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645fe20 0x55beb64606e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.750+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3278228370 >> 172.21.15.26:0/3278228370 conn(0x55beb6454da0 msgr2=0x55beb645a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.751+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3278228370 shutdown_connections 2023-07-13T22:17:59.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.751+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3278228370 wait complete. 2023-07-13T22:17:59.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.758+0000 7f4f237bfb80 1 -- start start 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.759+0000 7f4f237bfb80 1 Processor -- start 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb64690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb6470630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6350ac0 con 0x55beb645fe20 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb63507c0 con 0x55beb645f4d0 2023-07-13T22:17:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6234920 con 0x55beb6460c70 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.760+0000 7f4f237bfb80 10 client.0 _ll_register_callbacks cb 0x55beb634e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.761+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.761+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45624/0 (socket says 172.21.15.26:45624) 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.761+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3646225426 learned_addr learned my addr 172.21.15.26:0/3646225426 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.762+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3646225426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 msgr2=0x55beb6470630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.762+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb6470630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.762+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 msgr2=0x55beb64690b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.762+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb64690b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.762+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3646225426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef8013460 con 0x55beb645fe20 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.763+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80059b0 tx=0x7f4ef8015050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5534429a5f31ae28 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.763+0000 7f4ef17fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.764+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8018040 con 0x55beb645fe20 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.764+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4ef801dab0 con 0x55beb645fe20 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.764+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beb630a4f0 con 0x55beb645fe20 2023-07-13T22:17:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.764+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ef8010d10 con 0x55beb645fe20 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4f237bfb80 10 client.15855 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55beb61c7c60 con 0x55beb645fe20 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef8036030 con 0x55beb645fe20 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4ef80104f0 con 0x55beb645fe20 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4f237bfb80 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4f237bfb80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.765+0000 7f4f237bfb80 20 client.15855 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.766+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb6496f10 0x55beb64993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.766+0000 7f4f237bfb80 4 client.15855 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:17:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.766+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55beb63a9eb0 con 0x55beb6496f10 2023-07-13T22:17:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.768+0000 7f4f0db9b700 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb6496f10 0x55beb64993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.770+0000 7f4f0db9b700 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb6496f10 0x55beb64993c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de541c96cef4fd31 server_cookie=5528aa986d0eb788 in_seq=0 out_seq=0 2023-07-13T22:17:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.770+0000 7f4ef17fa700 10 client.15855 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.783+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3646225426 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55beb63a9eb0 con 0x55beb6496f10 2023-07-13T22:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.783+0000 7f4ef17fa700 10 client.15855 handle_command_reply: tid=0 2023-07-13T22:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.784+0000 7f4f237bfb80 1 client.15855 shutdown 2023-07-13T22:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.785+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 msgr2=0x55beb64695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.785+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80059b0 tx=0x7f4ef8015050 comp rx=0 tx=0).stop 2023-07-13T22:17:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 shutdown_connections 2023-07-13T22:17:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb6496f10 0x55beb64993c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb6460c70 0x55beb6470630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb64690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3646225426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb64695f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 >> 172.21.15.26:0/3646225426 conn(0x55beb6454da0 msgr2=0x55beb64598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4ef17fa700 0 client.15855 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 shutdown_connections 2023-07-13T22:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.790+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3646225426 wait complete. 2023-07-13T22:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.791+0000 7f4f237bfb80 20 client.15855 trim_cache size 0 max 16384 2023-07-13T22:17:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.831+0000 7f4f237bfb80 1 Processor -- start 2023-07-13T22:17:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 -- start start 2023-07-13T22:17:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645f4d0 0x55beb6463a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb6460c70 0x55beb6470550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6468c80 con 0x55beb645fe20 2023-07-13T22:17:59.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb645b4c0 con 0x55beb6460c70 2023-07-13T22:17:59.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.832+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6469fb0 con 0x55beb645f4d0 2023-07-13T22:17:59.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.833+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.833+0000 7f4f0d39a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45640/0 (socket says 172.21.15.26:45640) 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.833+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3739173661 learned_addr learned my addr 172.21.15.26:0/3739173661 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3739173661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645f4d0 msgr2=0x55beb6463a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645f4d0 0x55beb6463a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3739173661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb6460c70 msgr2=0x55beb6470550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb6460c70 0x55beb6470550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 -- 172.21.15.26:0/3739173661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beb6205250 con 0x55beb645fe20 2023-07-13T22:17:59.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.834+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80094d0 tx=0x7f4ef8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e04de6f2b61af3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.846+0000 7f4efcbff700 1 -- 172.21.15.26:0/3739173661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8018070 con 0x55beb645fe20 2023-07-13T22:17:59.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.846+0000 7f4efcbff700 1 -- 172.21.15.26:0/3739173661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef8036040 con 0x55beb645fe20 2023-07-13T22:17:59.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.847+0000 7f4efcbff700 1 -- 172.21.15.26:0/3739173661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80053c0 con 0x55beb645fe20 2023-07-13T22:17:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.848+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3739173661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 msgr2=0x55beb6463f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.848+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80094d0 tx=0x7f4ef8009710 comp rx=0 tx=0).stop 2023-07-13T22:17:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.852+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3739173661 shutdown_connections 2023-07-13T22:17:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.852+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb6460c70 0x55beb6470550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.852+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645f4d0 0x55beb6463a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.852+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3739173661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb645fe20 0x55beb6463f70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.852+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3739173661 >> 172.21.15.26:0/3739173661 conn(0x55beb6454da0 msgr2=0x55beb6470f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.857+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3739173661 shutdown_connections 2023-07-13T22:17:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.858+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3739173661 wait complete. 2023-07-13T22:17:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.858+0000 7f4f237bfb80 1 -- start start 2023-07-13T22:17:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.859+0000 7f4f237bfb80 1 Processor -- start 2023-07-13T22:17:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.859+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645fe20 0x55beb646c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb6460c70 0x55beb646f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb646fe30 con 0x55beb6460c70 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb646ffa0 con 0x55beb645f4d0 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beb6470160 con 0x55beb645fe20 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f237bfb80 10 client.0 _ll_register_callbacks cb 0x7f4ef8013680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:17:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f0db9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f0db9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40498/0 (socket says 172.21.15.26:40498) 2023-07-13T22:17:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.860+0000 7f4f0db9b700 1 -- 172.21.15.26:0/3904650055 learned_addr learned my addr 172.21.15.26:0/3904650055 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:17:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.861+0000 7f4f0db9b700 1 -- 172.21.15.26:0/3904650055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645fe20 msgr2=0x55beb646c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:17:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.861+0000 7f4f0db9b700 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645fe20 0x55beb646c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.861+0000 7f4f0db9b700 1 -- 172.21.15.26:0/3904650055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb6460c70 msgr2=0x55beb646f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:17:59.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.861+0000 7f4f0db9b700 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb6460c70 0x55beb646f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.861+0000 7f4f0db9b700 1 -- 172.21.15.26:0/3904650055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55beb61c7c60 con 0x55beb645f4d0 2023-07-13T22:17:59.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.871+0000 7f4f0db9b700 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009cd0 tx=0x7f4f00009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe69800e870a8870 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:17:59.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.871+0000 7f4ef17fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:17:59.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.872+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000e040 con 0x55beb645f4d0 2023-07-13T22:17:59.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.872+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f0001fd50 con 0x55beb645f4d0 2023-07-13T22:17:59.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.873+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ef80130d0 con 0x55beb645f4d0 2023-07-13T22:17:59.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.882+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f00008f10 con 0x55beb645f4d0 2023-07-13T22:17:59.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.901+0000 7f4f237bfb80 10 client.26087 fetch_fsmap learned FSMap version 29 2023-07-13T22:17:59.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.901+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55beb61dd2b0 con 0x55beb645f4d0 2023-07-13T22:17:59.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ef80130d0 con 0x55beb645f4d0 2023-07-13T22:17:59.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f0000b0d0 con 0x55beb645f4d0 2023-07-13T22:17:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 10 client.26087 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:17:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 10 client.26087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:17:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 20 client.26087 populate_metadata read hostname 'smithi026' 2023-07-13T22:17:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb64ebf40 0x55beb64ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:17:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 4 client.26087 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:17:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.910+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55beb636c050 con 0x55beb64ebf40 2023-07-13T22:17:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.911+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb64ebf40 0x55beb64ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:17:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.912+0000 7f4ef17fa700 10 client.26087 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:59.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.912+0000 7f4f0d39a700 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb64ebf40 0x55beb64ee350 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c56a7e228f5de1b server_cookie=e6e29e06b462c058 in_seq=0 out_seq=0 2023-07-13T22:17:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.912+0000 7f4ef17fa700 1 -- 172.21.15.26:0/3904650055 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55beb636c050 con 0x55beb64ebf40 2023-07-13T22:17:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.912+0000 7f4ef17fa700 10 client.26087 handle_command_reply: tid=0 2023-07-13T22:17:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.912+0000 7f4f237bfb80 1 client.26087 shutdown 2023-07-13T22:17:59.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 msgr2=0x55beb646c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009cd0 tx=0x7f4f00009540 comp rx=0 tx=0).stop 2023-07-13T22:17:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 shutdown_connections 2023-07-13T22:17:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55beb6460c70 0x55beb646f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55beb64ebf40 0x55beb64ee350 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55beb645f4d0 0x55beb646c390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 --2- 172.21.15.26:0/3904650055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb645fe20 0x55beb646c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 >> 172.21.15.26:0/3904650055 conn(0x55beb6454da0 msgr2=0x55beb6463a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4ef17fa700 0 client.26087 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.913+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 shutdown_connections 2023-07-13T22:17:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.914+0000 7f4f237bfb80 1 -- 172.21.15.26:0/3904650055 wait complete. 2023-07-13T22:17:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.915+0000 7f4f237bfb80 20 client.26087 trim_cache size 0 max 16384 2023-07-13T22:17:59.903 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 msgr2=0x7f4ef40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 0x7f4ef40a3800 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8013fd0 tx=0x7f4ef801d420 comp rx=0 tx=0).stop 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 msgr2=0x7f4f081b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400b550 tx=0x7f4f0400b910 comp rx=0 tx=0).stop 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 shutdown_connections 2023-07-13T22:17:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4ef40a1350 0x7f4ef40a3800 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f0810bd60 0x7f4f081b4910 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f08070ee0 0x7f4f08113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 --2- 172.21.15.26:0/339693589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f08071830 0x7f4f081140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 >> 172.21.15.26:0/339693589 conn(0x7f4f0806ca90 msgr2=0x7f4f0806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.916+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 shutdown_connections 2023-07-13T22:17:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:17:59.917+0000 7f4f0fdff700 1 -- 172.21.15.26:0/339693589 wait complete. 2023-07-13T22:18:00.941 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:18:00.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:18:01.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:01 smithi117 ceph-mon[129074]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:18:01.462 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:01 smithi026 ceph-mon[95647]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:18:01.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:01 smithi121 ceph-mon[129187]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:18:01.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 -- 172.21.15.26:0/688078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834072f70 msgr2=0x7fd8340733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 --2- 172.21.15.26:0/688078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834072f70 0x7fd8340733e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098f0 tx=0x7fd828009c00 comp rx=0 tx=0).stop 2023-07-13T22:18:01.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 -- 172.21.15.26:0/688078790 shutdown_connections 2023-07-13T22:18:01.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 --2- 172.21.15.26:0/688078790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834074550 0x7fd8340729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 --2- 172.21.15.26:0/688078790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 0x7fd834109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 --2- 172.21.15.26:0/688078790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834072f70 0x7fd8340733e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.681+0000 7fd838fff700 1 -- 172.21.15.26:0/688078790 >> 172.21.15.26:0/688078790 conn(0x7fd8340fb0e0 msgr2=0x7fd8340fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.682+0000 7fd838fff700 1 -- 172.21.15.26:0/688078790 shutdown_connections 2023-07-13T22:18:01.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.682+0000 7fd838fff700 1 -- 172.21.15.26:0/688078790 wait complete. 2023-07-13T22:18:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 Processor -- start 2023-07-13T22:18:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 -- start start 2023-07-13T22:18:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 0x7fd83419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 0x7fd83419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 0x7fd8341a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341a3a90 con 0x7fd834072f70 2023-07-13T22:18:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341a3bd0 con 0x7fd834074550 2023-07-13T22:18:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.684+0000 7fd838fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341a3d60 con 0x7fd8341079a0 2023-07-13T22:18:01.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.685+0000 7fd83259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 0x7fd83419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.685+0000 7fd83259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 0x7fd83419bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45668/0 (socket says 172.21.15.26:45668) 2023-07-13T22:18:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.685+0000 7fd83259c700 1 -- 172.21.15.26:0/1174034985 learned_addr learned my addr 172.21.15.26:0/1174034985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 0x7fd83419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd832d9d700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 0x7fd8341a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 -- 172.21.15.26:0/1174034985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 msgr2=0x7fd8341a34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 0x7fd8341a34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 -- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 msgr2=0x7fd83419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 0x7fd83419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.686+0000 7fd831d9b700 1 -- 172.21.15.26:0/1174034985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009520 con 0x7fd834074550 2023-07-13T22:18:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd831d9b700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 0x7fd83419c450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd828009230 tx=0x7fd82800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd8237fe700 1 -- 172.21.15.26:0/1174034985 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82801c070 con 0x7fd834074550 2023-07-13T22:18:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd8237fe700 1 -- 172.21.15.26:0/1174034985 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82800f840 con 0x7fd834074550 2023-07-13T22:18:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd8237fe700 1 -- 172.21.15.26:0/1174034985 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003c40 con 0x7fd834074550 2023-07-13T22:18:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8341a3fe0 con 0x7fd834074550 2023-07-13T22:18:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.687+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8341a4350 con 0x7fd834074550 2023-07-13T22:18:01.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.693+0000 7fd8237fe700 1 -- 172.21.15.26:0/1174034985 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd828005040 con 0x7fd834074550 2023-07-13T22:18:01.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.695+0000 7fd84c90ab80 1 Processor -- start 2023-07-13T22:18:01.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.695+0000 7fd84c90ab80 1 -- start start 2023-07-13T22:18:01.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3ea0 0x559c600f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c600f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5fff8d30 con 0x559c600f4cf0 2023-07-13T22:18:01.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6004bac0 con 0x559c600f3ea0 2023-07-13T22:18:01.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c60044f50 con 0x559c600f3550 2023-07-13T22:18:01.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32968/0 (socket says 172.21.15.26:32968) 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.697+0000 7fd832d9d700 1 -- 172.21.15.26:0/3880477176 learned_addr learned my addr 172.21.15.26:0/3880477176 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd83259c700 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3ea0 0x559c600f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd831d9b700 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c600f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd832d9d700 1 -- 172.21.15.26:0/3880477176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3ea0 msgr2=0x559c600f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd832d9d700 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3ea0 0x559c600f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd832d9d700 1 -- 172.21.15.26:0/3880477176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 msgr2=0x559c600f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd8237fe700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 0x7fd81c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd832d9d700 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c600f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.698+0000 7fd832d9d700 1 -- 172.21.15.26:0/3880477176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c5fe72e60 con 0x559c600f3550 2023-07-13T22:18:01.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.699+0000 7fd8237fe700 1 -- 172.21.15.26:0/1174034985 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd8280cd710 con 0x7fd834074550 2023-07-13T22:18:01.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.699+0000 7fd832d9d700 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d30 tx=0x7fd82c0078a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7524eb43c1118e10 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.699+0000 7fd820ff9700 1 -- 172.21.15.26:0/3880477176 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c014610 con 0x559c600f3550 2023-07-13T22:18:01.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.700+0000 7fd820ff9700 1 -- 172.21.15.26:0/3880477176 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c00f050 con 0x559c600f3550 2023-07-13T22:18:01.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.700+0000 7fd820ff9700 1 -- 172.21.15.26:0/3880477176 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c013a80 con 0x559c600f3550 2023-07-13T22:18:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.700+0000 7fd83259c700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 0x7fd81c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 -- 172.21.15.26:0/3880477176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 msgr2=0x559c600f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d30 tx=0x7fd82c0078a0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 -- 172.21.15.26:0/3880477176 shutdown_connections 2023-07-13T22:18:01.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c600f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3550 0x559c600f3960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/3880477176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3ea0 0x559c600f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 -- 172.21.15.26:0/3880477176 >> 172.21.15.26:0/3880477176 conn(0x559c600e8e20 msgr2=0x559c600ee250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 -- 172.21.15.26:0/3880477176 shutdown_connections 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.701+0000 7fd84c90ab80 1 -- 172.21.15.26:0/3880477176 wait complete. 2023-07-13T22:18:01.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.702+0000 7fd83259c700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 0x7fd81c0a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd824005950 tx=0x7fd824009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.702+0000 7fd84c90ab80 1 -- start start 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.702+0000 7fd84c90ab80 1 Processor -- start 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600fd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ffe4b40 con 0x559c600f4cf0 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5ffe4840 con 0x559c600f3550 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c5fec89a0 con 0x559c600f3ea0 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.703+0000 7fd84c90ab80 10 client.0 _ll_register_callbacks cb 0x559c5ffe27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.704+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.704+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45688/0 (socket says 172.21.15.26:45688) 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.704+0000 7fd832d9d700 1 -- 172.21.15.26:0/565815016 learned_addr learned my addr 172.21.15.26:0/565815016 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:01.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.704+0000 7fd83259c700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd831d9b700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600fd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd832d9d700 1 -- 172.21.15.26:0/565815016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 msgr2=0x559c600fd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd832d9d700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600fd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd832d9d700 1 -- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 msgr2=0x559c600fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd832d9d700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd832d9d700 1 -- 172.21.15.26:0/565815016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c009960 con 0x559c600f4cf0 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd83259c700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:18:01.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.705+0000 7fd831d9b700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.706+0000 7fd832d9d700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d00 tx=0x7fd82c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887f317c0b191966 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.706+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.706+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c013e10 con 0x559c600f4cf0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.706+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd82c014410 con 0x559c600f4cf0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.706+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c5ff9e570 con 0x559c600f4cf0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd82c02d030 con 0x559c600f4cf0 2023-07-13T22:18:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 10 client.15867 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:01.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c5fe5bce0 con 0x559c600f4cf0 2023-07-13T22:18:01.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c5ff9e570 con 0x559c600f4cf0 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd82c027750 con 0x559c600f4cf0 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 20 client.15867 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c60108f00 0x559c6010b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 4 client.15867 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:01.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.707+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c6003df30 con 0x559c60108f00 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.708+0000 7fd83259c700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c60108f00 0x559c6010b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.709+0000 7fd83259c700 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c60108f00 0x559c6010b310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3fffef28789ff9 server_cookie=e1514e753caa4613 in_seq=0 out_seq=0 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.709+0000 7fd819ffb700 10 client.15867 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.709+0000 7fd819ffb700 1 -- 172.21.15.26:0/565815016 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c6003df30 con 0x559c60108f00 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd819ffb700 10 client.15867 handle_command_reply: tid=0 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 client.15867 shutdown 2023-07-13T22:18:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 msgr2=0x559c60104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009d00 tx=0x7fd82c009110 comp rx=0 tx=0).stop 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 shutdown_connections 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c60108f00 0x559c6010b310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/565815016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd819ffb700 0 client.15867 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.710+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 >> 172.21.15.26:0/565815016 conn(0x559c600e8e20 msgr2=0x559c600ed930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.711+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 shutdown_connections 2023-07-13T22:18:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.711+0000 7fd84c90ab80 1 -- 172.21.15.26:0/565815016 wait complete. 2023-07-13T22:18:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.711+0000 7fd84c90ab80 20 client.15867 trim_cache size 0 max 16384 2023-07-13T22:18:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 Processor -- start 2023-07-13T22:18:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 -- start start 2023-07-13T22:18:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c600fc7c0 con 0x559c600f4cf0 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c600e8a20 con 0x559c600f3550 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.749+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c600fc5b0 con 0x559c600f3ea0 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.752+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.752+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45698/0 (socket says 172.21.15.26:45698) 2023-07-13T22:18:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.752+0000 7fd832d9d700 1 -- 172.21.15.26:0/2053546780 learned_addr learned my addr 172.21.15.26:0/2053546780 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 -- 172.21.15.26:0/2053546780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 msgr2=0x559c600f8170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600f8170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 -- 172.21.15.26:0/2053546780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 msgr2=0x559c600f7c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600f7c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 -- 172.21.15.26:0/2053546780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c600f5390 con 0x559c600f4cf0 2023-07-13T22:18:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.756+0000 7fd832d9d700 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009250 tx=0x7fd82c006180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb08ec48419ec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.757+0000 7fd8217fa700 1 -- 172.21.15.26:0/2053546780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c02d070 con 0x559c600f4cf0 2023-07-13T22:18:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.757+0000 7fd8217fa700 1 -- 172.21.15.26:0/2053546780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c013580 con 0x559c600f4cf0 2023-07-13T22:18:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.757+0000 7fd8217fa700 1 -- 172.21.15.26:0/2053546780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c031600 con 0x559c600f4cf0 2023-07-13T22:18:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2053546780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 msgr2=0x559c60104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009250 tx=0x7fd82c006180 comp rx=0 tx=0).stop 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2053546780 shutdown_connections 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f4cf0 0x559c60104820 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f3550 0x559c600f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2053546780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c600f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.758+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2053546780 >> 172.21.15.26:0/2053546780 conn(0x559c600e8e20 msgr2=0x559c600fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.759+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2053546780 shutdown_connections 2023-07-13T22:18:01.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.759+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2053546780 wait complete. 2023-07-13T22:18:01.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.759+0000 7fd84c90ab80 1 -- start start 2023-07-13T22:18:01.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 Processor -- start 2023-07-13T22:18:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 0x559c600ff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c601047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f4cf0 0x559c600ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c600ffe00 con 0x559c600f3550 2023-07-13T22:18:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6016ddc0 con 0x559c600f4cf0 2023-07-13T22:18:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6016df00 con 0x559c600f3ea0 2023-07-13T22:18:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.760+0000 7fd84c90ab80 10 client.0 _ll_register_callbacks cb 0x7fd82c004eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:01.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.762+0000 7fd832d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f4cf0 0x559c600ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.762+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c601047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.762+0000 7fd831d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c601047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:32990/0 (socket says 172.21.15.26:32990) 2023-07-13T22:18:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.762+0000 7fd831d9b700 1 -- 172.21.15.26:0/2746532638 learned_addr learned my addr 172.21.15.26:0/2746532638 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.762+0000 7fd83259c700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 0x559c600ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 -- 172.21.15.26:0/2746532638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 msgr2=0x559c601047a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c601047a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 -- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f4cf0 msgr2=0x559c600ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f4cf0 0x559c600ff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 -- 172.21.15.26:0/2746532638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c5fe5bce0 con 0x559c600f3550 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd83259c700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 0x559c600ff230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559c6010bbc0 tx=0x7fd824017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f36cb2b30071b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.763+0000 7fd819ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:18:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824023040 con 0x559c600f3550 2023-07-13T22:18:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd82402a430 con 0x559c600f3550 2023-07-13T22:18:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd82c00f050 con 0x559c600f3550 2023-07-13T22:18:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd824017070 con 0x559c600f3550 2023-07-13T22:18:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd84c90ab80 10 client.15873 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.764+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559c5fe992d0 con 0x559c600f3550 2023-07-13T22:18:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd82c00f050 con 0x559c600f3550 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd824032a70 con 0x559c600f3550 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 20 client.15873 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c6016f9e0 0x559c60171df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 4 client.15873 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:18:01.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.765+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559c600000d0 con 0x559c6016f9e0 2023-07-13T22:18:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.766+0000 7fd831d9b700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c6016f9e0 0x559c60171df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.766+0000 7fd831d9b700 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c6016f9e0 0x559c60171df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e51cb453eaaca59 server_cookie=d277a12e8e12cb51 in_seq=0 out_seq=0 2023-07-13T22:18:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.766+0000 7fd819ffb700 10 client.15873 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:01.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.767+0000 7fd819ffb700 1 -- 172.21.15.26:0/2746532638 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559c600000d0 con 0x559c6016f9e0 2023-07-13T22:18:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.767+0000 7fd819ffb700 10 client.15873 handle_command_reply: tid=0 2023-07-13T22:18:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.768+0000 7fd84c90ab80 1 client.15873 shutdown 2023-07-13T22:18:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.768+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 msgr2=0x559c600ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.768+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 0x559c600ff230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x559c6010bbc0 tx=0x7fd824017000 comp rx=0 tx=0).stop 2023-07-13T22:18:01.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.769+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 shutdown_connections 2023-07-13T22:18:01.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559c600f3550 0x559c600ff230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559c6016f9e0 0x559c60171df0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559c600f4cf0 0x559c600ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 --2- 172.21.15.26:0/2746532638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c600f3ea0 0x559c601047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd819ffb700 0 client.15873 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 >> 172.21.15.26:0/2746532638 conn(0x559c600e8e20 msgr2=0x559c600f7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 shutdown_connections 2023-07-13T22:18:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 1 -- 172.21.15.26:0/2746532638 wait complete. 2023-07-13T22:18:01.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.770+0000 7fd84c90ab80 20 client.15873 trim_cache size 0 max 16384 2023-07-13T22:18:01.759 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:18:01.759 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:18:01.759 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "e09acbec-0d71-45fb-8697-72a287b9adfb", 2023-07-13T22:18:01.760 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:18:01.760 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:18:01.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.771+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 msgr2=0x7fd81c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.771+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 0x7fd81c0a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd824005950 tx=0x7fd824009260 comp rx=0 tx=0).stop 2023-07-13T22:18:01.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.771+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 msgr2=0x7fd83419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.771+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 0x7fd83419c450 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd828009230 tx=0x7fd82800b790 comp rx=0 tx=0).stop 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 shutdown_connections 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd81c0a11d0 0x7fd81c0a3680 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd824005950 tx=0x7fd824009260 comp rx=0 tx=0).stop 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd834072f70 0x7fd83419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8341079a0 0x7fd8341a34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 --2- 172.21.15.26:0/1174034985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd834074550 0x7fd83419c450 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd828009230 tx=0x7fd82800b790 comp rx=0 tx=0).stop 2023-07-13T22:18:01.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.772+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 >> 172.21.15.26:0/1174034985 conn(0x7fd8340fb0e0 msgr2=0x7fd8340fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:01.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.774+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 shutdown_connections 2023-07-13T22:18:01.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:01.776+0000 7fd838fff700 1 -- 172.21.15.26:0/1174034985 wait complete. 2023-07-13T22:18:01.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:18:02.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.502+0000 7f157ebff700 1 -- 172.21.15.26:0/3002398434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 msgr2=0x7f15780712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.502+0000 7f157ebff700 1 --2- 172.21.15.26:0/3002398434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 0x7f15780712f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f157000b0a0 tx=0x7f157000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.504+0000 7f157ebff700 1 -- 172.21.15.26:0/3002398434 shutdown_connections 2023-07-13T22:18:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.504+0000 7f157ebff700 1 --2- 172.21.15.26:0/3002398434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f157810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.504+0000 7f157ebff700 1 --2- 172.21.15.26:0/3002398434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 0x7f15780712f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.504+0000 7f157ebff700 1 --2- 172.21.15.26:0/3002398434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f157810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.504+0000 7f157ebff700 1 -- 172.21.15.26:0/3002398434 >> 172.21.15.26:0/3002398434 conn(0x7f157806ca90 msgr2=0x7f157806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.505+0000 7f157ebff700 1 -- 172.21.15.26:0/3002398434 shutdown_connections 2023-07-13T22:18:02.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.505+0000 7f157ebff700 1 -- 172.21.15.26:0/3002398434 wait complete. 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.507+0000 7f157ebff700 1 Processor -- start 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.507+0000 7f157ebff700 1 -- start start 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 0x7f1578114bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f1578115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f1578116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578116780 con 0x7f157810bd60 2023-07-13T22:18:02.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781168f0 con 0x7f1578070ee0 2023-07-13T22:18:02.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.508+0000 7f157ebff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578116a60 con 0x7f1578071830 2023-07-13T22:18:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.509+0000 7f1577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f1578115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.510+0000 7f1577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f1578115100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33004/0 (socket says 172.21.15.26:33004) 2023-07-13T22:18:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.510+0000 7f1577fff700 1 -- 172.21.15.26:0/1931868198 learned_addr learned my addr 172.21.15.26:0/1931868198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:02.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.512+0000 7f157d19c700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f1578116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.513+0000 7f157d19c700 1 -- 172.21.15.26:0/1931868198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 msgr2=0x7f1578115100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.513+0000 7f157d19c700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f1578115100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.513+0000 7f157d19c700 1 -- 172.21.15.26:0/1931868198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 msgr2=0x7f1578114bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.513+0000 7f157d19c700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 0x7f1578114bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.513+0000 7f157d19c700 1 -- 172.21.15.26:0/1931868198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009d00 con 0x7f157810bd60 2023-07-13T22:18:02.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.514+0000 7f157d19c700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f1578116180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f156800b3f0 tx=0x7f156800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.515+0000 7f1575ffb700 1 -- 172.21.15.26:0/1931868198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568011830 con 0x7f157810bd60 2023-07-13T22:18:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.515+0000 7f1575ffb700 1 -- 172.21.15.26:0/1931868198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1568011e50 con 0x7f157810bd60 2023-07-13T22:18:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.515+0000 7f1575ffb700 1 -- 172.21.15.26:0/1931868198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800f2e0 con 0x7f157810bd60 2023-07-13T22:18:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.515+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1578116d40 con 0x7f157810bd60 2023-07-13T22:18:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.515+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578117280 con 0x7f157810bd60 2023-07-13T22:18:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.534+0000 7f159246fb80 1 Processor -- start 2023-07-13T22:18:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.535+0000 7f159246fb80 1 -- start start 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a4f4d0 0x561704a4f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a50c70 0x561704a510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704954cb0 con 0x561704a50c70 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617049a7a40 con 0x561704a4f4d0 2023-07-13T22:18:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.536+0000 7f159246fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617049a0ed0 con 0x561704a4fe20 2023-07-13T22:18:02.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.537+0000 7f157c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.537+0000 7f157c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33016/0 (socket says 172.21.15.26:33016) 2023-07-13T22:18:02.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.537+0000 7f157c99b700 1 -- 172.21.15.26:0/11501627 learned_addr learned my addr 172.21.15.26:0/11501627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:02.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.537+0000 7f1577fff700 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a4f4d0 0x561704a4f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 -- 172.21.15.26:0/11501627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a4f4d0 msgr2=0x561704a4f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a4f4d0 0x561704a4f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 -- 172.21.15.26:0/11501627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a50c70 msgr2=0x561704a510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a50c70 0x561704a510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 -- 172.21.15.26:0/11501627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617047cede0 con 0x561704a4fe20 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.538+0000 7f157c99b700 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1570006070 tx=0x7f15700096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79828c2ded79d42e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.539+0000 7f1566ffd700 1 -- 172.21.15.26:0/11501627 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000e040 con 0x561704a4fe20 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.539+0000 7f1566ffd700 1 -- 172.21.15.26:0/11501627 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1570007960 con 0x561704a4fe20 2023-07-13T22:18:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.539+0000 7f1566ffd700 1 -- 172.21.15.26:0/11501627 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157001dbe0 con 0x561704a4fe20 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.540+0000 7f159246fb80 1 -- 172.21.15.26:0/11501627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 msgr2=0x561704a506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.540+0000 7f159246fb80 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1570006070 tx=0x7f15700096e0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.541+0000 7f159246fb80 1 -- 172.21.15.26:0/11501627 shutdown_connections 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.541+0000 7f159246fb80 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a50c70 0x561704a510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.541+0000 7f159246fb80 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a4f4d0 0x561704a4f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.541+0000 7f159246fb80 1 --2- 172.21.15.26:0/11501627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a506e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.541+0000 7f159246fb80 1 -- 172.21.15.26:0/11501627 >> 172.21.15.26:0/11501627 conn(0x561704a44da0 msgr2=0x561704a4a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.542+0000 7f1575ffb700 1 -- 172.21.15.26:0/1931868198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1568011990 con 0x7f157810bd60 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.545+0000 7f159246fb80 1 -- 172.21.15.26:0/11501627 shutdown_connections 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.545+0000 7f1575ffb700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 0x7f15600a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.545+0000 7f1575ffb700 1 -- 172.21.15.26:0/1931868198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15680cc1d0 con 0x7f157810bd60 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.545+0000 7f159246fb80 1 -- 172.21.15.26:0/11501627 wait complete. 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.546+0000 7f159246fb80 1 -- start start 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.546+0000 7f159246fb80 1 Processor -- start 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f157c99b700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 0x7f15600a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a59360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a59d10 0x561704a60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704940ac0 con 0x561704a4fe20 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617049407c0 con 0x561704a59d10 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704824920 con 0x561704a598a0 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.547+0000 7f159246fb80 10 client.0 _ll_register_callbacks cb 0x56170493e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.548+0000 7f157d19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a59d10 0x561704a60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.548+0000 7f1577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.548+0000 7f1577fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33022/0 (socket says 172.21.15.26:33022) 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.548+0000 7f1577fff700 1 -- 172.21.15.26:0/1615407873 learned_addr learned my addr 172.21.15.26:0/1615407873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 -- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a59d10 msgr2=0x561704a60e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a59d10 0x561704a60e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 -- 172.21.15.26:0/1615407873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 msgr2=0x561704a59360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a59360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 -- 172.21.15.26:0/1615407873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578117640 con 0x561704a598a0 2023-07-13T22:18:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f1577fff700 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f156c005ed0 tx=0x7f156c010aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ff7598750655fb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f155bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.549+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004440 con 0x561704a598a0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.550+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f156c017030 con 0x561704a598a0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.550+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617048fa4f0 con 0x561704a598a0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.550+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156c003680 con 0x561704a598a0 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.550+0000 7f159246fb80 10 client.26092 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.550+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5617047b7c60 con 0x561704a598a0 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5617048fa4f0 con 0x561704a598a0 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f156c011b00 con 0x561704a598a0 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 10 client.26092 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 20 client.26092 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704a659b0 0x561704a67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 4 client.26092 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.551+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561704999eb0 con 0x561704a659b0 2023-07-13T22:18:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.552+0000 7f157c99b700 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704a659b0 0x561704a67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.553+0000 7f157c99b700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 0x7f15600a3710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f157001d7b0 tx=0x7f15700094f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.553+0000 7f157c99b700 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704a659b0 0x561704a67dc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385530f9e0b0d359 server_cookie=728e7aca06069a24 in_seq=0 out_seq=0 2023-07-13T22:18:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.553+0000 7f155bfff700 10 client.26092 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.554+0000 7f155bfff700 1 -- 172.21.15.26:0/1615407873 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561704999eb0 con 0x561704a659b0 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.554+0000 7f155bfff700 10 client.26092 handle_command_reply: tid=0 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.554+0000 7f159246fb80 1 client.26092 shutdown 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 msgr2=0x561704a5e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f156c005ed0 tx=0x7f156c010aa0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 shutdown_connections 2023-07-13T22:18:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a59360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704a659b0 0x561704a67dc0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a59d10 0x561704a60e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 --2- 172.21.15.26:0/1615407873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a598a0 0x561704a5e8f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 >> 172.21.15.26:0/1615407873 conn(0x561704a44da0 msgr2=0x561704a498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.555+0000 7f155bfff700 0 client.26092 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.566+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 shutdown_connections 2023-07-13T22:18:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.566+0000 7f159246fb80 1 -- 172.21.15.26:0/1615407873 wait complete. 2023-07-13T22:18:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.568+0000 7f159246fb80 20 client.26092 trim_cache size 0 max 16384 2023-07-13T22:18:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.605+0000 7f159246fb80 1 Processor -- start 2023-07-13T22:18:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 -- start start 2023-07-13T22:18:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a59d10 0x561704a59130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a501f0 con 0x561704a4fe20 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a5a140 con 0x561704a598a0 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.606+0000 7f159246fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a59670 con 0x561704a59d10 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.608+0000 7f1577fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.608+0000 7f1577fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40604/0 (socket says 172.21.15.26:40604) 2023-07-13T22:18:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.608+0000 7f1577fff700 1 -- 172.21.15.26:0/2239610210 learned_addr learned my addr 172.21.15.26:0/2239610210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 -- 172.21.15.26:0/2239610210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a59d10 msgr2=0x561704a59130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:18:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a59d10 0x561704a59130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 -- 172.21.15.26:0/2239610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 msgr2=0x561704a60e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 -- 172.21.15.26:0/2239610210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617047f5250 con 0x561704a598a0 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.609+0000 7f1577fff700 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f156c01c4e0 tx=0x7f156c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ac13069f7c95ca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.610+0000 7f15677fe700 1 -- 172.21.15.26:0/2239610210 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004440 con 0x561704a598a0 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.610+0000 7f15677fe700 1 -- 172.21.15.26:0/2239610210 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156c01c600 con 0x561704a598a0 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.610+0000 7f15677fe700 1 -- 172.21.15.26:0/2239610210 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c011aa0 con 0x561704a598a0 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.611+0000 7f159246fb80 1 -- 172.21.15.26:0/2239610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 msgr2=0x561704a46310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.611+0000 7f159246fb80 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f156c01c4e0 tx=0x7f156c000bc0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 -- 172.21.15.26:0/2239610210 shutdown_connections 2023-07-13T22:18:02.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a4fe20 0x561704a60e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a59d10 0x561704a59130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 --2- 172.21.15.26:0/2239610210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a46310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 -- 172.21.15.26:0/2239610210 >> 172.21.15.26:0/2239610210 conn(0x561704a44da0 msgr2=0x561704a6c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 -- 172.21.15.26:0/2239610210 shutdown_connections 2023-07-13T22:18:02.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.612+0000 7f159246fb80 1 -- 172.21.15.26:0/2239610210 wait complete. 2023-07-13T22:18:02.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.613+0000 7f159246fb80 1 -- start start 2023-07-13T22:18:02.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.613+0000 7f159246fb80 1 Processor -- start 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a73a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a59d10 0x561704a74a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a74f80 con 0x561704a59d10 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a750f0 con 0x561704a598a0 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561704a752b0 con 0x561704a4fe20 2023-07-13T22:18:02.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f159246fb80 10 client.0 _ll_register_callbacks cb 0x7f156c00bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f157c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f157c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:33028/0 (socket says 172.21.15.26:33028) 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.614+0000 7f157c99b700 1 -- 172.21.15.26:0/2258122136 learned_addr learned my addr 172.21.15.26:0/2258122136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f1577fff700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a73a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 -- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 msgr2=0x561704a73a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a73a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 -- 172.21.15.26:0/2258122136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a59d10 msgr2=0x561704a74a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a59d10 0x561704a74a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 -- 172.21.15.26:0/2258122136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617047b7c60 con 0x561704a4fe20 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.615+0000 7f157c99b700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1570009fd0 tx=0x7f15700096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0475bd018c7d45 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.619+0000 7f155bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.619+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000e040 con 0x561704a4fe20 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.619+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1570033280 con 0x561704a4fe20 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.619+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1570007960 con 0x561704a4fe20 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.620+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1570007800 con 0x561704a4fe20 2023-07-13T22:18:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.620+0000 7f159246fb80 10 client.26095 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:02.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.620+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5617047cd2b0 con 0x561704a4fe20 2023-07-13T22:18:02.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.620+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1570007960 con 0x561704a4fe20 2023-07-13T22:18:02.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.620+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f157001dac0 con 0x561704a4fe20 2023-07-13T22:18:02.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 10 client.26095 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:02.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:02.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 20 client.26095 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704abab00 0x561704abcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 4 client.26095 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.621+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56170495c050 con 0x561704abab00 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.622+0000 7f1577fff700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704abab00 0x561704abcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.623+0000 7f1577fff700 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704abab00 0x561704abcf10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a44209f14648035 server_cookie=a4063bd29a6cb7c9 in_seq=0 out_seq=0 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.623+0000 7f155bfff700 10 client.26095 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.623+0000 7f155bfff700 1 -- 172.21.15.26:0/2258122136 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56170495c050 con 0x561704abab00 2023-07-13T22:18:02.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f155bfff700 10 client.26095 handle_command_reply: tid=0 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 client.26095 shutdown 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 msgr2=0x561704a734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1570009fd0 tx=0x7f15700096e0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 shutdown_connections 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561704a59d10 0x561704a74a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x561704abab00 0x561704abcf10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561704a4fe20 0x561704a734f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 --2- 172.21.15.26:0/2258122136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561704a598a0 0x561704a73a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.624+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 >> 172.21.15.26:0/2258122136 conn(0x561704a44da0 msgr2=0x561704a57240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.625+0000 7f155bfff700 0 client.26095 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:02.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.625+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 shutdown_connections 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.625+0000 7f159246fb80 1 -- 172.21.15.26:0/2258122136 wait complete. 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.626+0000 7f159246fb80 20 client.26095 trim_cache size 0 max 16384 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stdout: "e09acbec-0d71-45fb-8697-72a287b9adfb": { 2023-07-13T22:18:02.616 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:18:02.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "e09acbec-0d71-45fb-8697-72a287b9adfb", 2023-07-13T22:18:02.617 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:18:02.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:18:02.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:18:02.617 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:18:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.639+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 msgr2=0x7f15600a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.639+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 0x7f15600a3710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f157001d7b0 tx=0x7f15700094f0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.639+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 msgr2=0x7f1578116180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.639+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f1578116180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f156800b3f0 tx=0x7f156800b7b0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 shutdown_connections 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15600a1260 0x7f15600a3710 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f157810bd60 0x7f1578116180 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1578070ee0 0x7f1578114bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 --2- 172.21.15.26:0/1931868198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1578071830 0x7f1578115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.641+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 >> 172.21.15.26:0/1931868198 conn(0x7f157806ca90 msgr2=0x7f157806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.642+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 shutdown_connections 2023-07-13T22:18:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:02.642+0000 7f157ebff700 1 -- 172.21.15.26:0/1931868198 wait complete. 2023-07-13T22:18:02.701 INFO:tasks.cephfs.filesystem:scrub status for tag:e09acbec-0d71-45fb-8697-72a287b9adfb - {'path': '/', 'tag': 'e09acbec-0d71-45fb-8697-72a287b9adfb', 'options': 'recursive,force'} 2023-07-13T22:18:03.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:03 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.462 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:18:03.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:18:03.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:18:03.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:18:03.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.464 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:03 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 85 op/s 2023-07-13T22:18:03.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:18:03.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:18:03.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:18:03.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:03.492 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:03 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:05.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:05 smithi117 ceph-mon[129074]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:18:05.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:05 smithi026 ceph-mon[95647]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:18:05.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:05 smithi121 ceph-mon[129187]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 317 MiB/s wr, 101 op/s 2023-07-13T22:18:07.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:07 smithi121 ceph-mon[129187]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:18:07.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:07 smithi117 ceph-mon[129074]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:18:07.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:07 smithi026 ceph-mon[95647]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 273 MiB/s wr, 85 op/s 2023-07-13T22:18:09.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:09 smithi117 ceph-mon[129074]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:18:09.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:09 smithi026 ceph-mon[95647]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:18:09.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:09 smithi121 ceph-mon[129187]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:18:11.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:11 smithi117 ceph-mon[129074]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 295 MiB/s wr, 92 op/s 2023-07-13T22:18:11.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:11 smithi026 ceph-mon[95647]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 295 MiB/s wr, 92 op/s 2023-07-13T22:18:11.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:11 smithi121 ceph-mon[129187]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 295 MiB/s wr, 92 op/s 2023-07-13T22:18:12.692 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:12 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:18:12.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:12 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:18:12.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:12 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:18:13.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:13 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:18:13.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:13 smithi117 ceph-mon[129074]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:18:13.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:13 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:18:13.718 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:13 smithi026 ceph-mon[95647]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:18:13.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:13 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:18:13.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:13 smithi121 ceph-mon[129187]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 85 op/s 2023-07-13T22:18:15.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:15 smithi121 ceph-mon[129187]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 334 MiB/s wr, 105 op/s 2023-07-13T22:18:15.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:15 smithi117 ceph-mon[129074]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 334 MiB/s wr, 105 op/s 2023-07-13T22:18:15.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:15 smithi026 ceph-mon[95647]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 334 MiB/s wr, 105 op/s 2023-07-13T22:18:17.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:17 smithi117 ceph-mon[129074]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 89 op/s 2023-07-13T22:18:17.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:17 smithi026 ceph-mon[95647]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 89 op/s 2023-07-13T22:18:17.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:17 smithi121 ceph-mon[129187]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 89 op/s 2023-07-13T22:18:19.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:19 smithi117 ceph-mon[129074]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 297 MiB/s wr, 95 op/s 2023-07-13T22:18:19.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:19 smithi026 ceph-mon[95647]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 297 MiB/s wr, 95 op/s 2023-07-13T22:18:19.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:19 smithi121 ceph-mon[129187]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 297 MiB/s wr, 95 op/s 2023-07-13T22:18:21.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:20 smithi117 ceph-mon[129074]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 313 MiB/s wr, 98 op/s 2023-07-13T22:18:21.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:20 smithi026 ceph-mon[95647]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 313 MiB/s wr, 98 op/s 2023-07-13T22:18:21.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:20 smithi121 ceph-mon[129187]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 313 MiB/s wr, 98 op/s 2023-07-13T22:18:23.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:22 smithi117 ceph-mon[129074]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 293 MiB/s wr, 92 op/s 2023-07-13T22:18:23.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:22 smithi026 ceph-mon[95647]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 293 MiB/s wr, 92 op/s 2023-07-13T22:18:23.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:22 smithi121 ceph-mon[129187]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 293 MiB/s wr, 92 op/s 2023-07-13T22:18:25.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:24 smithi117 ceph-mon[129074]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 350 MiB/s wr, 110 op/s 2023-07-13T22:18:25.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:24 smithi026 ceph-mon[95647]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 350 MiB/s wr, 110 op/s 2023-07-13T22:18:25.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:24 smithi121 ceph-mon[129187]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 350 MiB/s wr, 110 op/s 2023-07-13T22:18:27.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:27 smithi117 ceph-mon[129074]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 289 MiB/s wr, 90 op/s 2023-07-13T22:18:27.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:27 smithi026 ceph-mon[95647]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 289 MiB/s wr, 90 op/s 2023-07-13T22:18:27.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:27 smithi121 ceph-mon[129187]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 289 MiB/s wr, 90 op/s 2023-07-13T22:18:29.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:29 smithi117 ceph-mon[129074]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:18:29.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:29 smithi026 ceph-mon[95647]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:18:29.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:29 smithi121 ceph-mon[129187]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:18:31.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:31 smithi117 ceph-mon[129074]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:18:31.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:31 smithi026 ceph-mon[95647]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:18:31.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:31 smithi121 ceph-mon[129187]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:18:32.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:18:33.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:33 smithi117 ceph-mon[129074]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 MiB/s wr, 93 op/s 2023-07-13T22:18:33.465 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:33 smithi026 ceph-mon[95647]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 MiB/s wr, 93 op/s 2023-07-13T22:18:33.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:33 smithi121 ceph-mon[129187]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 MiB/s wr, 93 op/s 2023-07-13T22:18:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 -- 172.21.15.26:0/2980062024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 msgr2=0x7f849c107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2980062024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 0x7f849c107e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f848c00f490 tx=0x7f848c00f7a0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 -- 172.21.15.26:0/2980062024 shutdown_connections 2023-07-13T22:18:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2980062024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2980062024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 0x7f849c107e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2980062024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849c070c70 0x7f849c0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.597+0000 7f84a0fff700 1 -- 172.21.15.26:0/2980062024 >> 172.21.15.26:0/2980062024 conn(0x7f849c06ca90 msgr2=0x7f849c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.598+0000 7f84a0fff700 1 -- 172.21.15.26:0/2980062024 shutdown_connections 2023-07-13T22:18:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.598+0000 7f84a0fff700 1 -- 172.21.15.26:0/2980062024 wait complete. 2023-07-13T22:18:33.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.600+0000 7f84a0fff700 1 Processor -- start 2023-07-13T22:18:33.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.600+0000 7f84a0fff700 1 -- start start 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849c070c70 0x7f849c1acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 0x7f849c1b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1b4b30 con 0x7f849c071620 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1b4c70 con 0x7f849c070c70 2023-07-13T22:18:33.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.601+0000 7f84a0fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1b4e00 con 0x7f849c107a20 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56338/0 (socket says 172.21.15.26:56338) 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 -- 172.21.15.26:0/2683602125 learned_addr learned my addr 172.21.15.26:0/2683602125 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 -- 172.21.15.26:0/2683602125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 msgr2=0x7f849c1b4560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 0x7f849c1b4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 -- 172.21.15.26:0/2683602125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849c070c70 msgr2=0x7f849c1acfb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:18:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849c070c70 0x7f849c1acfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.602+0000 7f8491d9b700 1 -- 172.21.15.26:0/2683602125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009c40 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.603+0000 7f8491d9b700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f848400d950 tx=0x7f848400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.603+0000 7f8493fff700 1 -- 172.21.15.26:0/2683602125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484009b40 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.604+0000 7f8493fff700 1 -- 172.21.15.26:0/2683602125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8484009ca0 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.604+0000 7f8493fff700 1 -- 172.21.15.26:0/2683602125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400f620 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.605+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c0727a0 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.605+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c072c30 con 0x7f849c071620 2023-07-13T22:18:33.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.612+0000 7f84b4956b80 1 Processor -- start 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.619+0000 7f8493fff700 1 -- 172.21.15.26:0/2683602125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8484010460 con 0x7f849c071620 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.621+0000 7f8493fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 0x7f84880c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.622+0000 7f8493fff700 1 -- 172.21.15.26:0/2683602125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84840cc680 con 0x7f849c071620 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 -- start start 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6c4d0 0x55a9bff6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 0x55a9bff6d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 0x55a9bff6e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfe71cb0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfec4a40 con 0x55a9bff6dc70 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.623+0000 7f84b4956b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfebded0 con 0x55a9bff6ce20 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.624+0000 7f849ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 0x55a9bff6e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.624+0000 7f849ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 0x55a9bff6e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60408/0 (socket says 172.21.15.26:60408) 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.625+0000 7f849ad9d700 1 -- 172.21.15.26:0/3929151168 learned_addr learned my addr 172.21.15.26:0/3929151168 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.625+0000 7f849a59c700 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6c4d0 0x55a9bff6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.625+0000 7f8491d9b700 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 0x55a9bff6d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.625+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 0x7f84880c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.626+0000 7f8491d9b700 1 -- 172.21.15.26:0/3929151168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 msgr2=0x55a9bff6e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.626+0000 7f8491d9b700 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 0x55a9bff6e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.626+0000 7f8491d9b700 1 -- 172.21.15.26:0/3929151168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6c4d0 msgr2=0x55a9bff6c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.626+0000 7f8491d9b700 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6c4d0 0x55a9bff6c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.626+0000 7f8491d9b700 1 -- 172.21.15.26:0/3929151168 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bfcebde0 con 0x55a9bff6ce20 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.627+0000 7f8491d9b700 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 0x55a9bff6d6e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84840ca130 tx=0x7f84840994d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d55f86178f4106d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.628+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 0x7f84880c5810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8494005b40 tx=0x7f8494005dc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.630+0000 7f8490d99700 1 -- 172.21.15.26:0/3929151168 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848409aea0 con 0x55a9bff6ce20 2023-07-13T22:18:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.630+0000 7f8490d99700 1 -- 172.21.15.26:0/3929151168 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84840b2210 con 0x55a9bff6ce20 2023-07-13T22:18:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.630+0000 7f8490d99700 1 -- 172.21.15.26:0/3929151168 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484098f30 con 0x55a9bff6ce20 2023-07-13T22:18:33.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 -- 172.21.15.26:0/3929151168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 msgr2=0x55a9bff6d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 0x55a9bff6d6e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84840ca130 tx=0x7f84840994d0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 -- 172.21.15.26:0/3929151168 shutdown_connections 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6c4d0 0x55a9bff6c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6dc70 0x55a9bff6e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 --2- 172.21.15.26:0/3929151168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6ce20 0x55a9bff6d6e0 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f84840ca130 tx=0x7f84840994d0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.632+0000 7f84b4956b80 1 -- 172.21.15.26:0/3929151168 >> 172.21.15.26:0/3929151168 conn(0x55a9bff61da0 msgr2=0x55a9bff671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.638+0000 7f84b4956b80 1 -- 172.21.15.26:0/3929151168 shutdown_connections 2023-07-13T22:18:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.638+0000 7f84b4956b80 1 -- 172.21.15.26:0/3929151168 wait complete. 2023-07-13T22:18:33.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.639+0000 7f84b4956b80 1 -- start start 2023-07-13T22:18:33.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.639+0000 7f84b4956b80 1 Processor -- start 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6dc70 0x55a9bff76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff7b740 0x55a9bff7db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfe5dac0 con 0x55a9bff7b740 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfe5d7c0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfd41920 con 0x55a9bff6dc70 2023-07-13T22:18:33.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.640+0000 7f84b4956b80 10 client.0 _ll_register_callbacks cb 0x55a9bfe5b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:33.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.641+0000 7f849a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.641+0000 7f849a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60412/0 (socket says 172.21.15.26:60412) 2023-07-13T22:18:33.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.641+0000 7f849a59c700 1 -- 172.21.15.26:0/1585745927 learned_addr learned my addr 172.21.15.26:0/1585745927 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:33.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f8491d9b700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6dc70 0x55a9bff76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f849a59c700 1 -- 172.21.15.26:0/1585745927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6dc70 msgr2=0x55a9bff76710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f849a59c700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6dc70 0x55a9bff76710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f849a59c700 1 -- 172.21.15.26:0/1585745927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff7b740 msgr2=0x55a9bff7db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f849a59c700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff7b740 0x55a9bff7db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.642+0000 7f849a59c700 1 -- 172.21.15.26:0/1585745927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c072c30 con 0x55a9bff6c4d0 2023-07-13T22:18:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.643+0000 7f849a59c700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f848c01c000 tx=0x7f848c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d20cdce94cd7ee7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.643+0000 7f8481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.643+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c016040 con 0x55a9bff6c4d0 2023-07-13T22:18:33.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.643+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f848c003c10 con 0x55a9bff6c4d0 2023-07-13T22:18:33.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.643+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9bfe174f0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.644+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f848c027030 con 0x55a9bff6c4d0 2023-07-13T22:18:33.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.644+0000 7f84b4956b80 10 client.26123 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:33.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.644+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9bfcd4c60 con 0x55a9bff6c4d0 2023-07-13T22:18:33.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9bfe174f0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f848c005540 con 0x55a9bff6c4d0 2023-07-13T22:18:33.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 10 client.26123 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:33.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 10 client.26123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:33.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 20 client.26123 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bff82400 0x55a9bff84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 4 client.26123 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.645+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9bfeb6eb0 con 0x55a9bff82400 2023-07-13T22:18:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.646+0000 7f8491d9b700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bff82400 0x55a9bff84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.647+0000 7f8491d9b700 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bff82400 0x55a9bff84810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509c243433aa6c50 server_cookie=6bc262191e64dd01 in_seq=0 out_seq=0 2023-07-13T22:18:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.647+0000 7f8481ffb700 10 client.26123 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.648+0000 7f8481ffb700 1 -- 172.21.15.26:0/1585745927 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a9bfeb6eb0 con 0x55a9bff82400 2023-07-13T22:18:33.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.648+0000 7f8481ffb700 10 client.26123 handle_command_reply: tid=0 2023-07-13T22:18:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.648+0000 7f84b4956b80 1 client.26123 shutdown 2023-07-13T22:18:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 msgr2=0x55a9bff761d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f848c01c000 tx=0x7f848c009ba0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 shutdown_connections 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff7b740 0x55a9bff7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bff82400 0x55a9bff84810 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff761d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 --2- 172.21.15.26:0/1585745927 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6dc70 0x55a9bff76710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 >> 172.21.15.26:0/1585745927 conn(0x55a9bff61da0 msgr2=0x55a9bff668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f8481ffb700 0 client.26123 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:33.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.649+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 shutdown_connections 2023-07-13T22:18:33.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.650+0000 7f84b4956b80 1 -- 172.21.15.26:0/1585745927 wait complete. 2023-07-13T22:18:33.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.652+0000 7f84b4956b80 20 client.26123 trim_cache size 0 max 16384 2023-07-13T22:18:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.688+0000 7f84b4956b80 1 Processor -- start 2023-07-13T22:18:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 -- start start 2023-07-13T22:18:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff74250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 0x55a9bff67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfeb6eb0 con 0x55a9bff6dc70 2023-07-13T22:18:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bfe174f0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.689+0000 7f84b4956b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bff773c0 con 0x55a9bff7b740 2023-07-13T22:18:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 0x55a9bff67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 0x55a9bff67080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54056/0 (socket says 172.21.15.26:54056) 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849ad9d700 1 -- 172.21.15.26:0/574296080 learned_addr learned my addr 172.21.15.26:0/574296080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f8491d9b700 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849a59c700 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff74250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849a59c700 1 -- 172.21.15.26:0/574296080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 msgr2=0x55a9bff67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849a59c700 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 0x55a9bff67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849a59c700 1 -- 172.21.15.26:0/574296080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 msgr2=0x55a9bff92050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.690+0000 7f849a59c700 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff92050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.691+0000 7f849a59c700 1 -- 172.21.15.26:0/574296080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9bff6e560 con 0x55a9bff6c4d0 2023-07-13T22:18:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.691+0000 7f849a59c700 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff74250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f848c009f30 tx=0x7f848c00f7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62aaa644d86acac0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.691+0000 7f849159a700 1 -- 172.21.15.26:0/574296080 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c027070 con 0x55a9bff6c4d0 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.691+0000 7f849159a700 1 -- 172.21.15.26:0/574296080 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f848c005aa0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.691+0000 7f849159a700 1 -- 172.21.15.26:0/574296080 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c016040 con 0x55a9bff6c4d0 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.692+0000 7f84b4956b80 1 -- 172.21.15.26:0/574296080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 msgr2=0x55a9bff74250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.692+0000 7f84b4956b80 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff74250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f848c009f30 tx=0x7f848c00f7a0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 -- 172.21.15.26:0/574296080 shutdown_connections 2023-07-13T22:18:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff7b740 0x55a9bff67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff6c4d0 0x55a9bff74250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 --2- 172.21.15.26:0/574296080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 -- 172.21.15.26:0/574296080 >> 172.21.15.26:0/574296080 conn(0x55a9bff61da0 msgr2=0x55a9bff6d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 -- 172.21.15.26:0/574296080 shutdown_connections 2023-07-13T22:18:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.693+0000 7f84b4956b80 1 -- 172.21.15.26:0/574296080 wait complete. 2023-07-13T22:18:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.694+0000 7f84b4956b80 1 -- start start 2023-07-13T22:18:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.694+0000 7f84b4956b80 1 Processor -- start 2023-07-13T22:18:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 0x55a9bff8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff8fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 0x55a9bff90c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bff911b0 con 0x55a9bff6dc70 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bff91320 con 0x55a9bff7b740 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9bff914e0 con 0x55a9bff6c4d0 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f84b4956b80 10 client.0 _ll_register_callbacks cb 0x7f848c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f849a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 0x55a9bff8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f849a59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 0x55a9bff8f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54060/0 (socket says 172.21.15.26:54060) 2023-07-13T22:18:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.695+0000 7f849a59c700 1 -- 172.21.15.26:0/2002965844 learned_addr learned my addr 172.21.15.26:0/2002965844 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f8491d9b700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff8fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 0x55a9bff90c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 -- 172.21.15.26:0/2002965844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 msgr2=0x55a9bff8f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 0x55a9bff8f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 -- 172.21.15.26:0/2002965844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 msgr2=0x55a9bff8fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff8fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.696+0000 7f849ad9d700 1 -- 172.21.15.26:0/2002965844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9bfcd4c60 con 0x55a9bff7b740 2023-07-13T22:18:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.697+0000 7f849ad9d700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 0x55a9bff90c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8494005e30 tx=0x7f849400d5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed1dc32b457d30c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.697+0000 7f8481ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.698+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849401b820 con 0x55a9bff7b740 2023-07-13T22:18:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.698+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f849401b980 con 0x55a9bff7b740 2023-07-13T22:18:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.699+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84840b2210 con 0x55a9bff7b740 2023-07-13T22:18:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.699+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f849401bb40 con 0x55a9bff7b740 2023-07-13T22:18:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.699+0000 7f84b4956b80 10 client.26129 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.699+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a9bfd12250 con 0x55a9bff7b740 2023-07-13T22:18:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.700+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84840b2210 con 0x55a9bff7b740 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f849402e3d0 con 0x55a9bff7b740 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 10 client.26129 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 10 client.26129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 20 client.26129 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bffd70d0 0x55a9bffd94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 4 client.26129 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:18:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9bfe79050 con 0x55a9bffd70d0 2023-07-13T22:18:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.701+0000 7f849a59c700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bffd70d0 0x55a9bffd94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.702+0000 7f849a59c700 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bffd70d0 0x55a9bffd94e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ad381a3fc81af2 server_cookie=34c48e4f55a48ae8 in_seq=0 out_seq=0 2023-07-13T22:18:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.702+0000 7f8481ffb700 10 client.26129 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f8481ffb700 1 -- 172.21.15.26:0/2002965844 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9bfe79050 con 0x55a9bffd70d0 2023-07-13T22:18:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f8481ffb700 10 client.26129 handle_command_reply: tid=0 2023-07-13T22:18:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 client.26129 shutdown 2023-07-13T22:18:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 msgr2=0x55a9bff90c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 0x55a9bff90c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8494005e30 tx=0x7f849400d5c0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 shutdown_connections 2023-07-13T22:18:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a9bffd70d0 0x55a9bffd94e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a9bff7b740 0x55a9bff90c70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a9bff6c4d0 0x55a9bff8f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f8481ffb700 0 client.26129 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 --2- 172.21.15.26:0/2002965844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a9bff6dc70 0x55a9bff8fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.703+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 >> 172.21.15.26:0/2002965844 conn(0x55a9bff61da0 msgr2=0x55a9bff6d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.704+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 shutdown_connections 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.704+0000 7f84b4956b80 1 -- 172.21.15.26:0/2002965844 wait complete. 2023-07-13T22:18:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.704+0000 7f84b4956b80 20 client.26129 trim_cache size 0 max 16384 2023-07-13T22:18:33.693 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:18:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:18:33.693 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:18:33.693 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:18:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.705+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 msgr2=0x7f84880c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.705+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 0x7f84880c5810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8494005b40 tx=0x7f8494005dc0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.705+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 msgr2=0x7f849c1ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.705+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f848400d950 tx=0x7f848400dd10 comp rx=0 tx=0).stop 2023-07-13T22:18:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 shutdown_connections 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f849c107a20 0x7f849c1b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849c070c70 0x7f849c1acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f84880c3360 0x7f84880c5810 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 --2- 172.21.15.26:0/2683602125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f849c071620 0x7f849c1ad4f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 >> 172.21.15.26:0/2683602125 conn(0x7f849c06ca90 msgr2=0x7f849c10cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 shutdown_connections 2023-07-13T22:18:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:33.706+0000 7f84a0fff700 1 -- 172.21.15.26:0/2683602125 wait complete. 2023-07-13T22:18:33.728 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:18:33.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:18:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 -- 172.21.15.26:0/1381220267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6460100d10 msgr2=0x7f6460101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:34.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 --2- 172.21.15.26:0/1381220267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6460100d10 0x7f6460101160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6454009990 tx=0x7f6454009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 -- 172.21.15.26:0/1381220267 shutdown_connections 2023-07-13T22:18:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 --2- 172.21.15.26:0/1381220267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64601016a0 0x7f6460107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 --2- 172.21.15.26:0/1381220267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f64600fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 --2- 172.21.15.26:0/1381220267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6460100d10 0x7f6460101160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 -- 172.21.15.26:0/1381220267 >> 172.21.15.26:0/1381220267 conn(0x7f64600fb160 msgr2=0x7f64600fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 -- 172.21.15.26:0/1381220267 shutdown_connections 2023-07-13T22:18:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.327+0000 7f64677ff700 1 -- 172.21.15.26:0/1381220267 wait complete. 2023-07-13T22:18:34.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.329+0000 7f64677ff700 1 Processor -- start 2023-07-13T22:18:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.329+0000 7f64677ff700 1 -- start start 2023-07-13T22:18:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f646010d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6460100d10 0x7f646010dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:34.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 0x7f646010a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646010aac0 con 0x7f6460100d10 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646010ac30 con 0x7f64601016a0 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f64677ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646010adf0 con 0x7f64600ffb20 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f646559b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f646010d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f646559b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f646010d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54072/0 (socket says 172.21.15.26:54072) 2023-07-13T22:18:34.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.330+0000 7f646559b700 1 -- 172.21.15.26:0/3295882723 learned_addr learned my addr 172.21.15.26:0/3295882723 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.331+0000 7f6465d9c700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 0x7f646010a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.331+0000 7f6464d9a700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6460100d10 0x7f646010dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.332+0000 7f6465d9c700 1 -- 172.21.15.26:0/3295882723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 msgr2=0x7f646010d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:34.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.332+0000 7f6465d9c700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f646010d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.332+0000 7f6465d9c700 1 -- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6460100d10 msgr2=0x7f646010dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.332+0000 7f6465d9c700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6460100d10 0x7f646010dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.332+0000 7f6465d9c700 1 -- 172.21.15.26:0/3295882723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540095f0 con 0x7f64601016a0 2023-07-13T22:18:34.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.333+0000 7f6465d9c700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 0x7f646010a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f645000e970 tx=0x7f645000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:34.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.334+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450004220 con 0x7f64601016a0 2023-07-13T22:18:34.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.334+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6450004cf0 con 0x7f64601016a0 2023-07-13T22:18:34.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.334+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450010700 con 0x7f64601016a0 2023-07-13T22:18:34.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.335+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6460197730 con 0x7f64601016a0 2023-07-13T22:18:34.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.335+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460197bf0 con 0x7f64601016a0 2023-07-13T22:18:34.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.338+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c005320 con 0x7f64601016a0 2023-07-13T22:18:34.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.346+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6450004380 con 0x7f64601016a0 2023-07-13T22:18:34.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.349+0000 7f645e7fc700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 0x7f64480a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:34.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.349+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6450014070 con 0x7f64601016a0 2023-07-13T22:18:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.350+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f64500f8830 con 0x7f64601016a0 2023-07-13T22:18:34.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.350+0000 7f646559b700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 0x7f64480a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:34.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:34.351+0000 7f646559b700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 0x7f64480a3800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f645800f5e0 tx=0x7f645800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:35.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.062+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f644c005190 con 0x7f64601016a0 2023-07-13T22:18:35.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.064+0000 7f645e7fc700 1 -- 172.21.15.26:0/3295882723 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f64500963d0 con 0x7f64601016a0 2023-07-13T22:18:35.057 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:18:35.057 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:18:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.081+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 msgr2=0x7f64480a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.081+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 0x7f64480a3800 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f645800f5e0 tx=0x7f645800b040 comp rx=0 tx=0).stop 2023-07-13T22:18:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.081+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 msgr2=0x7f646010a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.081+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 0x7f646010a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f645000e970 tx=0x7f645000ec80 comp rx=0 tx=0).stop 2023-07-13T22:18:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.085+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 shutdown_connections 2023-07-13T22:18:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.086+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64601016a0 0x7f646010a320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.086+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64600ffb20 0x7f646010d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.086+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f64480a1350 0x7f64480a3800 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.086+0000 7f64677ff700 1 --2- 172.21.15.26:0/3295882723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6460100d10 0x7f646010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.086+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 >> 172.21.15.26:0/3295882723 conn(0x7f64600fb160 msgr2=0x7f64600fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.093+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 shutdown_connections 2023-07-13T22:18:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.095+0000 7f64677ff700 1 -- 172.21.15.26:0/3295882723 wait complete. 2023-07-13T22:18:35.096 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:18:35.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:18:35.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:35 smithi117 ceph-mon[129074]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:18:35.441 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:35 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3295882723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:18:35.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:35 smithi026 ceph-mon[95647]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:18:35.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:35 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3295882723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:18:35.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:35 smithi121 ceph-mon[129187]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 326 MiB/s wr, 104 op/s 2023-07-13T22:18:35.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:35 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3295882723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:18:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.728+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1018698693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4071830 msgr2=0x7f0af410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.728+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1018698693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4071830 0x7f0af410b7d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009990 tx=0x7f0ae4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.738+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1018698693 shutdown_connections 2023-07-13T22:18:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.738+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1018698693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af410bd60 0x7f0af410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.738+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1018698693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0af4070ee0 0x7f0af40712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.738+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1018698693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4071830 0x7f0af410b7d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.738+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1018698693 >> 172.21.15.26:0/1018698693 conn(0x7f0af406ca90 msgr2=0x7f0af406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.739+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1018698693 shutdown_connections 2023-07-13T22:18:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.740+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1018698693 wait complete. 2023-07-13T22:18:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.742+0000 7f0afa9ff700 1 Processor -- start 2023-07-13T22:18:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 -- start start 2023-07-13T22:18:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4070ee0 0x7f0af4114ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0af410bd60 0x7f0af4116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41166f0 con 0x7f0af4071830 2023-07-13T22:18:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4116860 con 0x7f0af4070ee0 2023-07-13T22:18:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.743+0000 7f0afa9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4116a20 con 0x7f0af410bd60 2023-07-13T22:18:35.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.746+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.747+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56400/0 (socket says 172.21.15.26:56400) 2023-07-13T22:18:35.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.747+0000 7f0af37fe700 1 -- 172.21.15.26:0/1739127634 learned_addr learned my addr 172.21.15.26:0/1739127634 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.748+0000 7f0af37fe700 1 -- 172.21.15.26:0/1739127634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0af410bd60 msgr2=0x7f0af4116120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.748+0000 7f0af37fe700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0af410bd60 0x7f0af4116120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.748+0000 7f0af37fe700 1 -- 172.21.15.26:0/1739127634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4070ee0 msgr2=0x7f0af4114ba0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.748+0000 7f0af37fe700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4070ee0 0x7f0af4114ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.748+0000 7f0af37fe700 1 -- 172.21.15.26:0/1739127634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40095f0 con 0x7f0af4071830 2023-07-13T22:18:35.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.749+0000 7f0af37fe700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.753+0000 7f0af17fa700 1 -- 172.21.15.26:0/1739127634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae401d070 con 0x7f0af4071830 2023-07-13T22:18:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.753+0000 7f0af17fa700 1 -- 172.21.15.26:0/1739127634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae400f460 con 0x7f0af4071830 2023-07-13T22:18:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.753+0000 7f0af17fa700 1 -- 172.21.15.26:0/1739127634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4017650 con 0x7f0af4071830 2023-07-13T22:18:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.753+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af4116c40 con 0x7f0af4071830 2023-07-13T22:18:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.754+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af4117120 con 0x7f0af4071830 2023-07-13T22:18:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.761+0000 7f0b0e393b80 1 Processor -- start 2023-07-13T22:18:35.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.762+0000 7f0af17fa700 1 -- 172.21.15.26:0/1739127634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0ae40052d0 con 0x7f0af4071830 2023-07-13T22:18:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.768+0000 7f0af17fa700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 0x7f0adc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.769+0000 7f0af17fa700 1 -- 172.21.15.26:0/1739127634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0ae40cdac0 con 0x7f0af4071830 2023-07-13T22:18:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.810+0000 7f0b0e393b80 1 -- start start 2023-07-13T22:18:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93ca46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93ca50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93ba8cb0 con 0x563c93ca4c70 2023-07-13T22:18:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93bfba40 con 0x563c93ca34d0 2023-07-13T22:18:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.811+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93bf4ed0 con 0x563c93ca3e20 2023-07-13T22:18:35.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.819+0000 7f0af3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.819+0000 7f0af3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60462/0 (socket says 172.21.15.26:60462) 2023-07-13T22:18:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.819+0000 7f0af3fff700 1 -- 172.21.15.26:0/2742711053 learned_addr learned my addr 172.21.15.26:0/2742711053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.819+0000 7f0af37fe700 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93ca46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 -- 172.21.15.26:0/2742711053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 msgr2=0x563c93ca46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93ca46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 -- 172.21.15.26:0/2742711053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 msgr2=0x563c93ca50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93ca50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 -- 172.21.15.26:0/2742711053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c93a22de0 con 0x563c93ca34d0 2023-07-13T22:18:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.820+0000 7f0af3fff700 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009810 tx=0x7f0ae8009b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7e378b68ebd20c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.830+0000 7f0ada7fc700 1 -- 172.21.15.26:0/2742711053 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8011830 con 0x563c93ca34d0 2023-07-13T22:18:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.830+0000 7f0ada7fc700 1 -- 172.21.15.26:0/2742711053 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8011e50 con 0x563c93ca34d0 2023-07-13T22:18:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.830+0000 7f0ada7fc700 1 -- 172.21.15.26:0/2742711053 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8010380 con 0x563c93ca34d0 2023-07-13T22:18:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.831+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2742711053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 msgr2=0x563c93ca38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.831+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009810 tx=0x7f0ae8009b20 comp rx=0 tx=0).stop 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.832+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2742711053 shutdown_connections 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.832+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93ca50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.832+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93ca38e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.832+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2742711053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93ca46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.832+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2742711053 >> 172.21.15.26:0/2742711053 conn(0x563c93c98da0 msgr2=0x563c93c9e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.833+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 0x7f0adc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.833+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2742711053 shutdown_connections 2023-07-13T22:18:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.833+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2742711053 wait complete. 2023-07-13T22:18:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.843+0000 7f0b0e393b80 1 -- start start 2023-07-13T22:18:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.843+0000 7f0b0e393b80 1 Processor -- start 2023-07-13T22:18:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca34d0 0x563c93cad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93cb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93b94ac0 con 0x563c93ca4c70 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93b947c0 con 0x563c93ca3e20 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93a78920 con 0x563c93ca34d0 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.844+0000 7f0b0e393b80 10 client.0 _ll_register_callbacks cb 0x563c93b92760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.845+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.845+0000 7f0af37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60474/0 (socket says 172.21.15.26:60474) 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.845+0000 7f0af37fe700 1 -- 172.21.15.26:0/3866026131 learned_addr learned my addr 172.21.15.26:0/3866026131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.845+0000 7f0af3fff700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca34d0 0x563c93cad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 -- 172.21.15.26:0/3866026131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca34d0 msgr2=0x563c93cad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca34d0 0x563c93cad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 -- 172.21.15.26:0/3866026131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 msgr2=0x563c93cb4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93cb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 -- 172.21.15.26:0/3866026131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae8009440 con 0x563c93ca3e20 2023-07-13T22:18:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.846+0000 7f0af37fe700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4003790 tx=0x7f0ae409b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7326927a861cc545 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.852+0000 7f0ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.852+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4020e10 con 0x563c93ca3e20 2023-07-13T22:18:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.852+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0ae409b170 con 0x563c93ca3e20 2023-07-13T22:18:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.889+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c93b4e4f0 con 0x563c93ca3e20 2023-07-13T22:18:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.901+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 0x7f0adc0a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007f20 tx=0x7f0aec00c3c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.904+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae409b330 con 0x563c93ca3e20 2023-07-13T22:18:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.912+0000 7f0b0e393b80 10 client.26144 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.913+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563c93a0bc60 con 0x563c93ca3e20 2023-07-13T22:18:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.953+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c93b4e4f0 con 0x563c93ca3e20 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.953+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0ae40a2e40 con 0x563c93ca3e20 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 10 client.26144 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 20 client.26144 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93cb8c30 0x563c93cbb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 4 client.26144 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:35.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.954+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c93bedeb0 con 0x563c93cb8c30 2023-07-13T22:18:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.973+0000 7f0af3fff700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93cb8c30 0x563c93cbb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.975+0000 7f0af3fff700 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93cb8c30 0x563c93cbb040 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=301b10ac415bcc02 server_cookie=f6032b5abc1de7ca in_seq=0 out_seq=0 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.975+0000 7f0ad37fe700 10 client.26144 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.979+0000 7f0ad37fe700 1 -- 172.21.15.26:0/3866026131 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c93bedeb0 con 0x563c93cb8c30 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.979+0000 7f0ad37fe700 10 client.26144 handle_command_reply: tid=0 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.979+0000 7f0b0e393b80 1 client.26144 shutdown 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.980+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 msgr2=0x563c93cad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.980+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4003790 tx=0x7f0ae409b000 comp rx=0 tx=0).stop 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 shutdown_connections 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93cb8c30 0x563c93cbb040 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca4c70 0x563c93cb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca34d0 0x563c93cad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/3866026131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca3e20 0x563c93cad730 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 >> 172.21.15.26:0/3866026131 conn(0x563c93c98da0 msgr2=0x563c93c9d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:35.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0ad37fe700 0 client.26144 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:18:35.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.988+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 shutdown_connections 2023-07-13T22:18:35.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:35.989+0000 7f0b0e393b80 1 -- 172.21.15.26:0/3866026131 wait complete. 2023-07-13T22:18:35.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.005+0000 7f0b0e393b80 20 client.26144 trim_cache size 0 max 16384 2023-07-13T22:18:36.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.034+0000 7f0b0e393b80 1 Processor -- start 2023-07-13T22:18:36.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.034+0000 7f0b0e393b80 1 -- start start 2023-07-13T22:18:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca34d0 0x563c93ca5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93cb4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93bedeb0 con 0x563c93ca34d0 2023-07-13T22:18:36.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93b4e4f0 con 0x563c93ca4c70 2023-07-13T22:18:36.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.035+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93ca4570 con 0x563c93ca3e20 2023-07-13T22:18:36.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.037+0000 7f0af8f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.037+0000 7f0af8f9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60488/0 (socket says 172.21.15.26:60488) 2023-07-13T22:18:36.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.037+0000 7f0af8f9c700 1 -- 172.21.15.26:0/2135872007 learned_addr learned my addr 172.21.15.26:0/2135872007 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:36.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.038+0000 7f0af8f9c700 1 -- 172.21.15.26:0/2135872007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 msgr2=0x563c93cb4730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:18:36.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.038+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93cb4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.038+0000 7f0af8f9c700 1 -- 172.21.15.26:0/2135872007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca34d0 msgr2=0x563c93ca5040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:18:36.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.038+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca34d0 0x563c93ca5040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.038+0000 7f0af8f9c700 1 -- 172.21.15.26:0/2135872007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c93a49250 con 0x563c93ca4c70 2023-07-13T22:18:36.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.045+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0078d0 tx=0x7f0aec011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5502d0af039b3d62 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.046+0000 7f0adaffd700 1 -- 172.21.15.26:0/2135872007 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00caf0 con 0x563c93ca4c70 2023-07-13T22:18:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.046+0000 7f0adaffd700 1 -- 172.21.15.26:0/2135872007 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0aec018df0 con 0x563c93ca4c70 2023-07-13T22:18:36.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.046+0000 7f0adaffd700 1 -- 172.21.15.26:0/2135872007 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0167b0 con 0x563c93ca4c70 2023-07-13T22:18:36.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.047+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2135872007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 msgr2=0x563c93c9d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:36.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.047+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0078d0 tx=0x7f0aec011000 comp rx=0 tx=0).stop 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2135872007 shutdown_connections 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca34d0 0x563c93ca5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca4c70 0x563c93c9d8b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/2135872007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca3e20 0x563c93cb4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2135872007 >> 172.21.15.26:0/2135872007 conn(0x563c93c98da0 msgr2=0x563c93cbf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:36.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2135872007 shutdown_connections 2023-07-13T22:18:36.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.048+0000 7f0b0e393b80 1 -- 172.21.15.26:0/2135872007 wait complete. 2023-07-13T22:18:36.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.049+0000 7f0b0e393b80 1 -- start start 2023-07-13T22:18:36.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.050+0000 7f0b0e393b80 1 Processor -- start 2023-07-13T22:18:36.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93cb4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca3e20 0x563c93cb4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 0x563c93cb0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93cb1060 con 0x563c93ca3e20 2023-07-13T22:18:36.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93cb11d0 con 0x563c93ca34d0 2023-07-13T22:18:36.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c93cb1390 con 0x563c93ca4c70 2023-07-13T22:18:36.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.051+0000 7f0b0e393b80 10 client.0 _ll_register_callbacks cb 0x7f0ae409ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.052+0000 7f0af3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93cb4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.052+0000 7f0af3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93cb4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60502/0 (socket says 172.21.15.26:60502) 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.052+0000 7f0af3fff700 1 -- 172.21.15.26:0/1465673558 learned_addr learned my addr 172.21.15.26:0/1465673558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.052+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 0x563c93cb0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.052+0000 7f0af37fe700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca3e20 0x563c93cb4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.053+0000 7f0af8f9c700 1 -- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 msgr2=0x563c93cb4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.053+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93cb4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.053+0000 7f0af8f9c700 1 -- 172.21.15.26:0/1465673558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca3e20 msgr2=0x563c93cb4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.053+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca3e20 0x563c93cb4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.053+0000 7f0af8f9c700 1 -- 172.21.15.26:0/1465673558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aec00cc50 con 0x563c93ca4c70 2023-07-13T22:18:36.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.054+0000 7f0af8f9c700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 0x563c93cb0ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007c70 tx=0x7f0aec003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=102c4a80d6edde88 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:36.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.054+0000 7f0ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:18:36.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.055+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec019640 con 0x563c93ca4c70 2023-07-13T22:18:36.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.056+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0aec0197a0 con 0x563c93ca4c70 2023-07-13T22:18:36.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.056+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ae8011e50 con 0x563c93ca4c70 2023-07-13T22:18:36.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.056+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aec019980 con 0x563c93ca4c70 2023-07-13T22:18:36.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.056+0000 7f0b0e393b80 10 client.26122 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.056+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563c93a212b0 con 0x563c93ca4c70 2023-07-13T22:18:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8011e50 con 0x563c93ca4c70 2023-07-13T22:18:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0aec029040 con 0x563c93ca4c70 2023-07-13T22:18:36.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 10 client.26122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:18:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 20 client.26122 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93d0de80 0x563c93d10290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 4 client.26122 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:18:36.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.057+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c93bb0050 con 0x563c93d0de80 2023-07-13T22:18:36.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.058+0000 7f0af3fff700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93d0de80 0x563c93d10290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.066+0000 7f0af3fff700 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93d0de80 0x563c93d10290 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34c69068dd4f46a server_cookie=fc317517353ab41e in_seq=0 out_seq=0 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.066+0000 7f0ad37fe700 10 client.26122 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.067+0000 7f0ad37fe700 1 -- 172.21.15.26:0/1465673558 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c93bb0050 con 0x563c93d0de80 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.067+0000 7f0ad37fe700 10 client.26122 handle_command_reply: tid=0 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.067+0000 7f0b0e393b80 1 client.26122 shutdown 2023-07-13T22:18:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 msgr2=0x563c93cb0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:36.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 0x563c93cb0ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007c70 tx=0x7f0aec003c30 comp rx=0 tx=0).stop 2023-07-13T22:18:36.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 shutdown_connections 2023-07-13T22:18:36.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563c93d0de80 0x563c93d10290 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563c93ca4c70 0x563c93cb0ae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563c93ca34d0 0x563c93cb4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 --2- 172.21.15.26:0/1465673558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563c93ca3e20 0x563c93cb4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.072+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 >> 172.21.15.26:0/1465673558 conn(0x563c93c98da0 msgr2=0x563c93c9d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.073+0000 7f0ad37fe700 0 client.26122 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:18:36.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.082+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 shutdown_connections 2023-07-13T22:18:36.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.082+0000 7f0b0e393b80 1 -- 172.21.15.26:0/1465673558 wait complete. 2023-07-13T22:18:36.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.137+0000 7f0b0e393b80 20 client.26122 trim_cache size 0 max 16384 2023-07-13T22:18:36.125 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:18:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.139+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 msgr2=0x7f0adc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.139+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 0x7f0adc0a3800 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007f20 tx=0x7f0aec00c3c0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.139+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 msgr2=0x7f0af41150e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:36.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.139+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae4003d80 comp rx=0 tx=0).stop 2023-07-13T22:18:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 shutdown_connections 2023-07-13T22:18:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0af410bd60 0x7f0af4116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0af4070ee0 0x7f0af4114ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0adc0a1350 0x7f0adc0a3800 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 --2- 172.21.15.26:0/1739127634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0af4071830 0x7f0af41150e0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4006010 tx=0x7f0ae4003d80 comp rx=0 tx=0).stop 2023-07-13T22:18:36.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.141+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 >> 172.21.15.26:0/1739127634 conn(0x7f0af406ca90 msgr2=0x7f0af406dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:36.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.154+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 shutdown_connections 2023-07-13T22:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.154+0000 7f0afa9ff700 1 -- 172.21.15.26:0/1739127634 wait complete. 2023-07-13T22:18:36.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:18:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.995+0000 7f326c1ff700 1 -- 172.21.15.26:0/2387249087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264071830 msgr2=0x7f326410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:36.995+0000 7f326c1ff700 1 --2- 172.21.15.26:0/2387249087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264071830 0x7f326410b7d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3254009990 tx=0x7f3254009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 -- 172.21.15.26:0/2387249087 shutdown_connections 2023-07-13T22:18:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 --2- 172.21.15.26:0/2387249087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264070ee0 0x7f32640712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 --2- 172.21.15.26:0/2387249087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f326410bd60 0x7f326410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 --2- 172.21.15.26:0/2387249087 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264071830 0x7f326410b7d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 -- 172.21.15.26:0/2387249087 >> 172.21.15.26:0/2387249087 conn(0x7f326406ca90 msgr2=0x7f326406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 -- 172.21.15.26:0/2387249087 shutdown_connections 2023-07-13T22:18:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.001+0000 7f326c1ff700 1 -- 172.21.15.26:0/2387249087 wait complete. 2023-07-13T22:18:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.003+0000 7f326c1ff700 1 Processor -- start 2023-07-13T22:18:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.003+0000 7f326c1ff700 1 -- start start 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264070ee0 0x7f3264113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f326410bd60 0x7f32641b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32641170b0 con 0x7f3264071830 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264117220 con 0x7f326410bd60 2023-07-13T22:18:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.004+0000 7f326c1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32641b4e50 con 0x7f3264070ee0 2023-07-13T22:18:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.006+0000 7f326979a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.006+0000 7f326979a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56434/0 (socket says 172.21.15.26:56434) 2023-07-13T22:18:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.006+0000 7f326979a700 1 -- 172.21.15.26:0/4265720047 learned_addr learned my addr 172.21.15.26:0/4265720047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 -- 172.21.15.26:0/4265720047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264070ee0 msgr2=0x7f3264113ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264070ee0 0x7f3264113ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 -- 172.21.15.26:0/4265720047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f326410bd60 msgr2=0x7f32641b4910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f326410bd60 0x7f32641b4910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 -- 172.21.15.26:0/4265720047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540095f0 con 0x7f3264071830 2023-07-13T22:18:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.007+0000 7f326979a700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3254000c00 tx=0x7f325400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.023+0000 7f325affd700 1 -- 172.21.15.26:0/4265720047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325401d070 con 0x7f3264071830 2023-07-13T22:18:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.024+0000 7f325affd700 1 -- 172.21.15.26:0/4265720047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325400fbb0 con 0x7f3264071830 2023-07-13T22:18:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.024+0000 7f325affd700 1 -- 172.21.15.26:0/4265720047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254017650 con 0x7f3264071830 2023-07-13T22:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.024+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32641b5050 con 0x7f3264071830 2023-07-13T22:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.024+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32641b53c0 con 0x7f3264071830 2023-07-13T22:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.031+0000 7f327fb1db80 1 Processor -- start 2023-07-13T22:18:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.032+0000 7f327fb1db80 1 -- start start 2023-07-13T22:18:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d2f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45c33cb0 con 0x556b45d2e4d0 2023-07-13T22:18:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45c86a40 con 0x556b45d2fc70 2023-07-13T22:18:37.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.036+0000 7f327fb1db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45c7fed0 con 0x556b45d2ee20 2023-07-13T22:18:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.042+0000 7f326a79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.042+0000 7f326a79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56436/0 (socket says 172.21.15.26:56436) 2023-07-13T22:18:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.042+0000 7f326a79c700 1 -- 172.21.15.26:0/1999524534 learned_addr learned my addr 172.21.15.26:0/1999524534 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.042+0000 7f325affd700 1 -- 172.21.15.26:0/4265720047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f325400fd20 con 0x7f3264071830 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 -- 172.21.15.26:0/1999524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 msgr2=0x556b45d2f6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d2f6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 -- 172.21.15.26:0/1999524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 msgr2=0x556b45d300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 -- 172.21.15.26:0/1999524534 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b45aadde0 con 0x556b45d2e4d0 2023-07-13T22:18:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.043+0000 7f326a79c700 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f325c009be0 tx=0x7f325c009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d0c8a35d07b8d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.044+0000 7f324ffff700 1 -- 172.21.15.26:0/1999524534 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c004bb0 con 0x556b45d2e4d0 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.044+0000 7f324ffff700 1 -- 172.21.15.26:0/1999524534 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325c00f040 con 0x556b45d2e4d0 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.044+0000 7f324ffff700 1 -- 172.21.15.26:0/1999524534 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c013650 con 0x556b45d2e4d0 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.045+0000 7f325affd700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 0x7f32500a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.045+0000 7f327fb1db80 1 -- 172.21.15.26:0/1999524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 msgr2=0x556b45d2e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.045+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f325c009be0 tx=0x7f325c009ef0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f325affd700 1 -- 172.21.15.26:0/4265720047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32540cdf50 con 0x7f3264071830 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f327fb1db80 1 -- 172.21.15.26:0/1999524534 shutdown_connections 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d2e8e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1999524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d2f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.046+0000 7f327fb1db80 1 -- 172.21.15.26:0/1999524534 >> 172.21.15.26:0/1999524534 conn(0x556b45d23da0 msgr2=0x556b45d291d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.047+0000 7f327fb1db80 1 -- 172.21.15.26:0/1999524534 shutdown_connections 2023-07-13T22:18:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.047+0000 7f327fb1db80 1 -- 172.21.15.26:0/1999524534 wait complete. 2023-07-13T22:18:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.048+0000 7f327fb1db80 1 -- start start 2023-07-13T22:18:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.048+0000 7f3269f9b700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 0x7f32500a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.048+0000 7f327fb1db80 1 Processor -- start 2023-07-13T22:18:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 0x556b45d38240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d38780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2fc70 0x556b45d3f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45c1fac0 con 0x556b45d2fc70 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45c1f7c0 con 0x556b45d2e4d0 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45b03920 con 0x556b45d2ee20 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f3269f9b700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 0x7f32500a3800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3260009610 tx=0x7f3260006c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f327fb1db80 10 client.0 _ll_register_callbacks cb 0x556b45c1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 0x556b45d38240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 0x556b45d38240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60532/0 (socket says 172.21.15.26:60532) 2023-07-13T22:18:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.049+0000 7f3269f9b700 1 -- 172.21.15.26:0/893060801 learned_addr learned my addr 172.21.15.26:0/893060801 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d38780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 -- 172.21.15.26:0/893060801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 msgr2=0x556b45d38240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 0x556b45d38240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 -- 172.21.15.26:0/893060801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2fc70 msgr2=0x556b45d3f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2fc70 0x556b45d3f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.050+0000 7f326979a700 1 -- 172.21.15.26:0/893060801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c0098c0 con 0x556b45d2ee20 2023-07-13T22:18:37.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.051+0000 7f326979a700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d38780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325409ae30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882e359e82cdf28c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.051+0000 7f324d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.051+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325409f3d0 con 0x556b45d2ee20 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.051+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f32540a3760 con 0x556b45d2ee20 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.051+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b45bd94f0 con 0x556b45d2ee20 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.052+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32540a10e0 con 0x556b45d2ee20 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.052+0000 7f327fb1db80 10 client.26128 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:37.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.052+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556b45a96c60 con 0x556b45d2ee20 2023-07-13T22:18:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b45bd94f0 con 0x556b45d2ee20 2023-07-13T22:18:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f325409a410 con 0x556b45d2ee20 2023-07-13T22:18:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 10 client.26128 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:37.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 10 client.26128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:18:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 20 client.26128 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d43ce0 0x556b45d460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 4 client.26128 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b45c78eb0 con 0x556b45d43ce0 2023-07-13T22:18:37.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.053+0000 7f3269f9b700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d43ce0 0x556b45d460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.054+0000 7f3269f9b700 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d43ce0 0x556b45d460f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61e5832e3e7a21f5 server_cookie=9f4378bf904d9669 in_seq=0 out_seq=0 2023-07-13T22:18:37.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.055+0000 7f324d7fa700 10 client.26128 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:18:37.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.057+0000 7f324d7fa700 1 -- 172.21.15.26:0/893060801 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556b45c78eb0 con 0x556b45d43ce0 2023-07-13T22:18:37.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.057+0000 7f324d7fa700 10 client.26128 handle_command_reply: tid=0 2023-07-13T22:18:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.057+0000 7f327fb1db80 1 client.26128 shutdown 2023-07-13T22:18:37.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 msgr2=0x556b45d38780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d38780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3254009fd0 tx=0x7f325409ae30 comp rx=0 tx=0).stop 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 shutdown_connections 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d43ce0 0x556b45d460f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2fc70 0x556b45d3f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2e4d0 0x556b45d38240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 --2- 172.21.15.26:0/893060801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d38780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 >> 172.21.15.26:0/893060801 conn(0x556b45d23da0 msgr2=0x556b45d291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f324d7fa700 0 client.26128 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:18:37.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 shutdown_connections 2023-07-13T22:18:37.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.058+0000 7f327fb1db80 1 -- 172.21.15.26:0/893060801 wait complete. 2023-07-13T22:18:37.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.059+0000 7f327fb1db80 20 client.26128 trim_cache size 0 max 16384 2023-07-13T22:18:37.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.087+0000 7f327fb1db80 1 Processor -- start 2023-07-13T22:18:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.087+0000 7f327fb1db80 1 -- start start 2023-07-13T22:18:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.087+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.087+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2ee20 0x556b45d3f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.088+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 0x556b45d3fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.088+0000 7f327fb1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d2ac50 con 0x556b45d2e4d0 2023-07-13T22:18:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.088+0000 7f327fb1db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d23970 con 0x556b45d2ee20 2023-07-13T22:18:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.088+0000 7f327fb1db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d393c0 con 0x556b45d2fc70 2023-07-13T22:18:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f326979a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2ee20 0x556b45d3f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56454/0 (socket says 172.21.15.26:56454) 2023-07-13T22:18:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f326a79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 0x556b45d3fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 -- 172.21.15.26:0/1553288307 learned_addr learned my addr 172.21.15.26:0/1553288307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 -- 172.21.15.26:0/1553288307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 msgr2=0x556b45d3fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 0x556b45d3fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 -- 172.21.15.26:0/1553288307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2ee20 msgr2=0x556b45d3f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2ee20 0x556b45d3f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.089+0000 7f3269f9b700 1 -- 172.21.15.26:0/1553288307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b45ad4250 con 0x556b45d2e4d0 2023-07-13T22:18:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.090+0000 7f326a79c700 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 0x556b45d3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.090+0000 7f3269f9b700 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3260022c30 tx=0x7f326000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2cbf0040aee25c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.091+0000 7f3258ff9700 1 -- 172.21.15.26:0/1553288307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260013e90 con 0x556b45d2e4d0 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.091+0000 7f3258ff9700 1 -- 172.21.15.26:0/1553288307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3260019040 con 0x556b45d2e4d0 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.091+0000 7f3258ff9700 1 -- 172.21.15.26:0/1553288307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260020ce0 con 0x556b45d2e4d0 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 -- 172.21.15.26:0/1553288307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 msgr2=0x556b45d361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3260022c30 tx=0x7f326000e000 comp rx=0 tx=0).stop 2023-07-13T22:18:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 -- 172.21.15.26:0/1553288307 shutdown_connections 2023-07-13T22:18:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d361a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2fc70 0x556b45d3fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 --2- 172.21.15.26:0/1553288307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2ee20 0x556b45d3f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.092+0000 7f327fb1db80 1 -- 172.21.15.26:0/1553288307 >> 172.21.15.26:0/1553288307 conn(0x556b45d23da0 msgr2=0x556b45d4aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.093+0000 7f327fb1db80 1 -- 172.21.15.26:0/1553288307 shutdown_connections 2023-07-13T22:18:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.093+0000 7f327fb1db80 1 -- 172.21.15.26:0/1553288307 wait complete. 2023-07-13T22:18:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.093+0000 7f327fb1db80 1 -- start start 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 Processor -- start 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d3f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d3d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d3dd90 con 0x556b45d2e4d0 2023-07-13T22:18:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d3df00 con 0x556b45d2fc70 2023-07-13T22:18:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.094+0000 7f327fb1db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b45d3e0c0 con 0x556b45d2ee20 2023-07-13T22:18:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.095+0000 7f327fb1db80 10 client.0 _ll_register_callbacks cb 0x556b45c1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.095+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.095+0000 7f3269f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56466/0 (socket says 172.21.15.26:56466) 2023-07-13T22:18:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.095+0000 7f3269f9b700 1 -- 172.21.15.26:0/2445727802 learned_addr learned my addr 172.21.15.26:0/2445727802 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.096+0000 7f3269f9b700 1 -- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 msgr2=0x556b45d3f590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.096+0000 7f3269f9b700 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d3f590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.096+0000 7f3269f9b700 1 -- 172.21.15.26:0/2445727802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 msgr2=0x556b45d3d740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.096+0000 7f3269f9b700 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d3d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.096+0000 7f3269f9b700 1 -- 172.21.15.26:0/2445727802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b45aadde0 con 0x556b45d2e4d0 2023-07-13T22:18:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.097+0000 7f3269f9b700 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f32600092f0 tx=0x7f32600206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a908647527d7a365 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.097+0000 7f324d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:18:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.098+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260005ce0 con 0x556b45d2e4d0 2023-07-13T22:18:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.098+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3260005e40 con 0x556b45d2e4d0 2023-07-13T22:18:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.098+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325c00f040 con 0x556b45d2e4d0 2023-07-13T22:18:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.098+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3260020a60 con 0x556b45d2e4d0 2023-07-13T22:18:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.099+0000 7f327fb1db80 10 client.15915 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.099+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f32600137a0 con 0x556b45d2e4d0 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.099+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f325c00f040 con 0x556b45d2e4d0 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3260006350 con 0x556b45d2e4d0 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 10 client.15915 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 20 client.15915 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d98df0 0x556b45d9b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 4 client.15915 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:18:37.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.100+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556b45c3b050 con 0x556b45d98df0 2023-07-13T22:18:37.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.101+0000 7f326979a700 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d98df0 0x556b45d9b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.102+0000 7f326979a700 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d98df0 0x556b45d9b200 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8e8607a5c8e645d server_cookie=3bdc481686911ab4 in_seq=0 out_seq=0 2023-07-13T22:18:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.102+0000 7f324d7fa700 10 client.15915 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:18:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.102+0000 7f324d7fa700 1 -- 172.21.15.26:0/2445727802 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556b45c3b050 con 0x556b45d98df0 2023-07-13T22:18:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.102+0000 7f324d7fa700 10 client.15915 handle_command_reply: tid=0 2023-07-13T22:18:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.102+0000 7f327fb1db80 1 client.15915 shutdown 2023-07-13T22:18:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 msgr2=0x556b45d3f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f32600092f0 tx=0x7f32600206e0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 shutdown_connections 2023-07-13T22:18:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x556b45d98df0 0x556b45d9b200 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556b45d2e4d0 0x556b45d3f050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556b45d2fc70 0x556b45d3d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 --2- 172.21.15.26:0/2445727802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b45d2ee20 0x556b45d3f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 >> 172.21.15.26:0/2445727802 conn(0x556b45d23da0 msgr2=0x556b45d361a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.103+0000 7f324d7fa700 0 client.15915 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.104+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 shutdown_connections 2023-07-13T22:18:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.104+0000 7f327fb1db80 1 -- 172.21.15.26:0/2445727802 wait complete. 2023-07-13T22:18:37.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.105+0000 7f327fb1db80 20 client.15915 trim_cache size 0 max 16384 2023-07-13T22:18:37.093 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:18:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 msgr2=0x7f32500a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 0x7f32500a3800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3260009610 tx=0x7f3260006c50 comp rx=0 tx=0).stop 2023-07-13T22:18:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 msgr2=0x7f3264114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3254000c00 tx=0x7f325400f740 comp rx=0 tx=0).stop 2023-07-13T22:18:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 shutdown_connections 2023-07-13T22:18:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f326410bd60 0x7f32641b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3264070ee0 0x7f3264113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f32500a1350 0x7f32500a3800 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 --2- 172.21.15.26:0/4265720047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3264071830 0x7f3264114020 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 >> 172.21.15.26:0/4265720047 conn(0x7f326406ca90 msgr2=0x7f326406e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 shutdown_connections 2023-07-13T22:18:37.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.127+0000 7f326c1ff700 1 -- 172.21.15.26:0/4265720047 wait complete. 2023-07-13T22:18:37.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:18:37.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:37 smithi026 ceph-mon[95647]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:18:37.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:37 smithi121 ceph-mon[129187]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:18:37.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:37 smithi117 ceph-mon[129074]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:18:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.747+0000 7ff2f49ff700 1 -- 172.21.15.26:0/2243746470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 msgr2=0x7ff2f00712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.747+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/2243746470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f00712f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009990 tx=0x7ff2e0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.763+0000 7ff2f49ff700 1 -- 172.21.15.26:0/2243746470 shutdown_connections 2023-07-13T22:18:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.763+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/2243746470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f010bd60 0x7ff2f010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.763+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/2243746470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f00712f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.763+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/2243746470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f0071830 0x7ff2f010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.763+0000 7ff2f49ff700 1 -- 172.21.15.26:0/2243746470 >> 172.21.15.26:0/2243746470 conn(0x7ff2f006ca90 msgr2=0x7ff2f006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.767+0000 7ff2f49ff700 1 -- 172.21.15.26:0/2243746470 shutdown_connections 2023-07-13T22:18:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.767+0000 7ff2f49ff700 1 -- 172.21.15.26:0/2243746470 wait complete. 2023-07-13T22:18:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.768+0000 7ff2f49ff700 1 Processor -- start 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.769+0000 7ff2f49ff700 1 -- start start 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f01a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0071830 0x7ff2f01a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 0x7ff2f01abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01ac230 con 0x7ff2f0071830 2023-07-13T22:18:37.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f010ec40 con 0x7ff2f0070ee0 2023-07-13T22:18:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2f49ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f010edd0 con 0x7ff2f010bd60 2023-07-13T22:18:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.770+0000 7ff2ee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f01a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2ee59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f01a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60566/0 (socket says 172.21.15.26:60566) 2023-07-13T22:18:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2ee59c700 1 -- 172.21.15.26:0/164625912 learned_addr learned my addr 172.21.15.26:0/164625912 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 0x7ff2f01abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 -- 172.21.15.26:0/164625912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 msgr2=0x7ff2f01a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f01a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 -- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0071830 msgr2=0x7ff2f01a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0071830 0x7ff2f01a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.771+0000 7ff2eed9d700 1 -- 172.21.15.26:0/164625912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00095f0 con 0x7ff2f010bd60 2023-07-13T22:18:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.772+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 0x7ff2f01abc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800be20 tx=0x7ff2e8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.772+0000 7ff2df7fe700 1 -- 172.21.15.26:0/164625912 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e800cc10 con 0x7ff2f010bd60 2023-07-13T22:18:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.773+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f010f0b0 con 0x7ff2f010bd60 2023-07-13T22:18:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.773+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f010f5c0 con 0x7ff2f010bd60 2023-07-13T22:18:37.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.773+0000 7ff2df7fe700 1 -- 172.21.15.26:0/164625912 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e800cd70 con 0x7ff2f010bd60 2023-07-13T22:18:37.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.773+0000 7ff2df7fe700 1 -- 172.21.15.26:0/164625912 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80126c0 con 0x7ff2f010bd60 2023-07-13T22:18:37.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.788+0000 7ff2df7fe700 1 -- 172.21.15.26:0/164625912 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff2e8012920 con 0x7ff2f010bd60 2023-07-13T22:18:37.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.789+0000 7ff3082a3b80 1 Processor -- start 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 -- start start 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edb76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edb80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56ecbbcb0 con 0x55a56edb6e20 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56ed0ea40 con 0x55a56edb64d0 2023-07-13T22:18:37.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.790+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56ed07ed0 con 0x55a56edb7c70 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60574/0 (socket says 172.21.15.26:60574) 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 -- 172.21.15.26:0/2706263023 learned_addr learned my addr 172.21.15.26:0/2706263023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edb80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 -- 172.21.15.26:0/2706263023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 msgr2=0x55a56edb80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edb80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 -- 172.21.15.26:0/2706263023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 msgr2=0x55a56edb76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edb76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.791+0000 7ff2edd9b700 1 -- 172.21.15.26:0/2706263023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56eb35de0 con 0x55a56edb64d0 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.792+0000 7ff2df7fe700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 0x7ff2d80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.792+0000 7ff2df7fe700 1 -- 172.21.15.26:0/164625912 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2e8006e40 con 0x7ff2f010bd60 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.793+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40368e0 tx=0x7ff2e40093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b924f151ec2d82ce server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.793+0000 7ff2dcff9700 1 -- 172.21.15.26:0/2706263023 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4031680 con 0x55a56edb64d0 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.793+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 0x7ff2d80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.793+0000 7ff2dcff9700 1 -- 172.21.15.26:0/2706263023 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e4009b40 con 0x55a56edb64d0 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.793+0000 7ff2dcff9700 1 -- 172.21.15.26:0/2706263023 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4033b70 con 0x55a56edb64d0 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.794+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 0x7ff2d80a3750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b5c0 tx=0x7ff2e0009ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.794+0000 7ff3082a3b80 1 -- 172.21.15.26:0/2706263023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 msgr2=0x55a56edb68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.794+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40368e0 tx=0x7ff2e40093d0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 -- 172.21.15.26:0/2706263023 shutdown_connections 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edb80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb64d0 0x55a56edb68e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/2706263023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edb76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 -- 172.21.15.26:0/2706263023 >> 172.21.15.26:0/2706263023 conn(0x55a56edabda0 msgr2=0x55a56edb11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 -- 172.21.15.26:0/2706263023 shutdown_connections 2023-07-13T22:18:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.795+0000 7ff3082a3b80 1 -- 172.21.15.26:0/2706263023 wait complete. 2023-07-13T22:18:37.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.796+0000 7ff3082a3b80 1 -- start start 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.796+0000 7ff3082a3b80 1 Processor -- start 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edc06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56eca7ac0 con 0x55a56edb6e20 2023-07-13T22:18:37.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56eca77c0 con 0x55a56edb7c70 2023-07-13T22:18:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56eb8b920 con 0x55a56edb64d0 2023-07-13T22:18:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.797+0000 7ff3082a3b80 10 client.0 _ll_register_callbacks cb 0x55a56eca5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.799+0000 7ff2ee59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.799+0000 7ff2eed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.799+0000 7ff2eed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60582/0 (socket says 172.21.15.26:60582) 2023-07-13T22:18:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.799+0000 7ff2eed9d700 1 -- 172.21.15.26:0/4138649756 learned_addr learned my addr 172.21.15.26:0/4138649756 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.800+0000 7ff2eed9d700 1 -- 172.21.15.26:0/4138649756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 msgr2=0x55a56edc0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.800+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.800+0000 7ff2eed9d700 1 -- 172.21.15.26:0/4138649756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 msgr2=0x55a56edc06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.800+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edc06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.800+0000 7ff2eed9d700 1 -- 172.21.15.26:0/4138649756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e4036590 con 0x55a56edb7c70 2023-07-13T22:18:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.801+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800cf90 tx=0x7ff2e8099f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ae4661c6653abe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.803+0000 7ff2d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.803+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80a1b70 con 0x55a56edb7c70 2023-07-13T22:18:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.803+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2e80999b0 con 0x55a56edb7c70 2023-07-13T22:18:37.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.803+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a56ec614f0 con 0x55a56edb7c70 2023-07-13T22:18:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.804+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e809d9e0 con 0x55a56edb7c70 2023-07-13T22:18:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.804+0000 7ff3082a3b80 10 client.26168 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.804+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a56eb1ec60 con 0x55a56edb7c70 2023-07-13T22:18:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a56ec614f0 con 0x55a56edb7c70 2023-07-13T22:18:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2e80cc290 con 0x55a56edb7c70 2023-07-13T22:18:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 10 client.26168 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:18:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 20 client.26168 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56edcbf50 0x55a56edce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 4 client.26168 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.805+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a56ed00eb0 con 0x55a56edcbf50 2023-07-13T22:18:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.806+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56edcbf50 0x55a56edce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.808+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56edcbf50 0x55a56edce360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ced778d40d30b43 server_cookie=3c164f2748f4d8da in_seq=0 out_seq=0 2023-07-13T22:18:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.808+0000 7ff2d5ffb700 10 client.26168 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:18:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.811+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4138649756 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a56ed00eb0 con 0x55a56edcbf50 2023-07-13T22:18:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.811+0000 7ff2d5ffb700 10 client.26168 handle_command_reply: tid=0 2023-07-13T22:18:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.811+0000 7ff3082a3b80 1 client.26168 shutdown 2023-07-13T22:18:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 msgr2=0x55a56edc7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800cf90 tx=0x7ff2e8099f10 comp rx=0 tx=0).stop 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 shutdown_connections 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56edcbf50 0x55a56edce360 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb7c70 0x55a56edc7710 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4138649756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb6e20 0x55a56edc06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 >> 172.21.15.26:0/4138649756 conn(0x55a56edabda0 msgr2=0x55a56edb08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff2d5ffb700 0 client.26168 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:18:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 shutdown_connections 2023-07-13T22:18:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.812+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4138649756 wait complete. 2023-07-13T22:18:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.813+0000 7ff3082a3b80 20 client.26168 trim_cache size 0 max 16384 2023-07-13T22:18:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.829+0000 7ff3082a3b80 1 Processor -- start 2023-07-13T22:18:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.829+0000 7ff3082a3b80 1 -- start start 2023-07-13T22:18:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 0x55a56edbab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edbb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edb7570 con 0x55a56edb64d0 2023-07-13T22:18:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edb24d0 con 0x55a56edb6e20 2023-07-13T22:18:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.830+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edab9c0 con 0x55a56edb7c70 2023-07-13T22:18:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edbb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2eed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2eed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54218/0 (socket says 172.21.15.26:54218) 2023-07-13T22:18:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2eed9d700 1 -- 172.21.15.26:0/1121773535 learned_addr learned my addr 172.21.15.26:0/1121773535 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 0x55a56edbab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 -- 172.21.15.26:0/1121773535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 msgr2=0x55a56edc7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 -- 172.21.15.26:0/1121773535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 msgr2=0x55a56edbab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 0x55a56edbab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2edd9b700 1 -- 172.21.15.26:0/1121773535 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56edb8310 con 0x55a56edb6e20 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.831+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 0x55a56edbab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.832+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edbb0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4036ed0 tx=0x7ff2e4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52edc62825231b39 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.832+0000 7ff2dd7fa700 1 -- 172.21.15.26:0/1121773535 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4005ee0 con 0x55a56edb6e20 2023-07-13T22:18:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.833+0000 7ff2dd7fa700 1 -- 172.21.15.26:0/1121773535 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e404ab40 con 0x55a56edb6e20 2023-07-13T22:18:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.833+0000 7ff2dd7fa700 1 -- 172.21.15.26:0/1121773535 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4041ba0 con 0x55a56edb6e20 2023-07-13T22:18:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 -- 172.21.15.26:0/1121773535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 msgr2=0x55a56edbb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edbb0d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4036ed0 tx=0x7ff2e4004710 comp rx=0 tx=0).stop 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 -- 172.21.15.26:0/1121773535 shutdown_connections 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb64d0 0x55a56edbab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb7c70 0x55a56edc7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/1121773535 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edbb0d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 -- 172.21.15.26:0/1121773535 >> 172.21.15.26:0/1121773535 conn(0x55a56edabda0 msgr2=0x55a56edbe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 -- 172.21.15.26:0/1121773535 shutdown_connections 2023-07-13T22:18:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.834+0000 7ff3082a3b80 1 -- 172.21.15.26:0/1121773535 wait complete. 2023-07-13T22:18:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.835+0000 7ff3082a3b80 1 -- start start 2023-07-13T22:18:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.835+0000 7ff3082a3b80 1 Processor -- start 2023-07-13T22:18:37.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 0x55a56edc6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edc7270 con 0x55a56edb7c70 2023-07-13T22:18:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edc73e0 con 0x55a56edb6e20 2023-07-13T22:18:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56edc75a0 con 0x55a56edb64d0 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff3082a3b80 10 client.0 _ll_register_callbacks cb 0x7ff2e80993c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff2edd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff2edd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60596/0 (socket says 172.21.15.26:60596) 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.836+0000 7ff2edd9b700 1 -- 172.21.15.26:0/4115353806 learned_addr learned my addr 172.21.15.26:0/4115353806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 0x55a56edc6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2edd9b700 1 -- 172.21.15.26:0/4115353806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 msgr2=0x55a56edc37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2edd9b700 1 -- 172.21.15.26:0/4115353806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 msgr2=0x55a56edc6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 0x55a56edc6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.837+0000 7ff2edd9b700 1 -- 172.21.15.26:0/4115353806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a56eb35de0 con 0x55a56edb6e20 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2edd9b700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40417b0 tx=0x7ff2e404a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a7090aa7ea58f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2eed9d700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 0x55a56edc6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4041ae0 con 0x55a56edb6e20 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff2e4045070 con 0x55a56edb6e20 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.838+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2e4009b40 con 0x55a56edb6e20 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e4041c40 con 0x55a56edb6e20 2023-07-13T22:18:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 10 client.26174 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a56eb5c250 con 0x55a56edb6e20 2023-07-13T22:18:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2e4009b40 con 0x55a56edb6e20 2023-07-13T22:18:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2e4054430 con 0x55a56edb6e20 2023-07-13T22:18:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 10 client.26174 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 10 client.26174 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:18:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 20 client.26174 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56ee20be0 0x55a56ee22ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 4 client.26174 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.839+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a56ecc3050 con 0x55a56ee20be0 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56ee20be0 0x55a56ee22ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff2ee59c700 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56ee20be0 0x55a56ee22ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92f7e9405807d071 server_cookie=9f72c3b75155e8a4 in_seq=0 out_seq=0 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff2d5ffb700 10 client.26174 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:18:37.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff2d5ffb700 1 -- 172.21.15.26:0/4115353806 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a56ecc3050 con 0x55a56ee20be0 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff2d5ffb700 10 client.26174 handle_command_reply: tid=0 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.840+0000 7ff3082a3b80 1 client.26174 shutdown 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 msgr2=0x55a56edc3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40417b0 tx=0x7ff2e404a480 comp rx=0 tx=0).stop 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 shutdown_connections 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a56edb7c70 0x55a56edc6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55a56ee20be0 0x55a56ee22ff0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a56edb64d0 0x55a56edc37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 --2- 172.21.15.26:0/4115353806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a56edb6e20 0x55a56edc3d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 >> 172.21.15.26:0/4115353806 conn(0x55a56edabda0 msgr2=0x55a56edbab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff2d5ffb700 0 client.26174 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 shutdown_connections 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.841+0000 7ff3082a3b80 1 -- 172.21.15.26:0/4115353806 wait complete. 2023-07-13T22:18:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.842+0000 7ff3082a3b80 20 client.26174 trim_cache size 0 max 16384 2023-07-13T22:18:37.835 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:18:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.842+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 msgr2=0x7ff2d80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.842+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 0x7ff2d80a3750 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000b5c0 tx=0x7ff2e0009ee0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.842+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 msgr2=0x7ff2f01abc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.842+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 0x7ff2f01abc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800be20 tx=0x7ff2e8009ed0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 shutdown_connections 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff2f010bd60 0x7ff2f01abc60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2f0070ee0 0x7ff2f01a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff2d80a12a0 0x7ff2d80a3750 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 --2- 172.21.15.26:0/164625912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2f0071830 0x7ff2f01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 >> 172.21.15.26:0/164625912 conn(0x7ff2f006ca90 msgr2=0x7ff2f006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 shutdown_connections 2023-07-13T22:18:37.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:37.843+0000 7ff2f49ff700 1 -- 172.21.15.26:0/164625912 wait complete. 2023-07-13T22:18:37.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:18:38.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.680+0000 7f6ce57ff700 1 -- 172.21.15.26:0/331015835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0071830 msgr2=0x7f6ce010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.680+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/331015835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0071830 0x7f6ce010b7d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009990 tx=0x7f6cd4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.690+0000 7f6ce57ff700 1 -- 172.21.15.26:0/331015835 shutdown_connections 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.690+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/331015835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ce0070ee0 0x7f6ce00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.690+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/331015835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.690+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/331015835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0071830 0x7f6ce010b7d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.690+0000 7f6ce57ff700 1 -- 172.21.15.26:0/331015835 >> 172.21.15.26:0/331015835 conn(0x7f6ce006ca90 msgr2=0x7f6ce006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.691+0000 7f6ce57ff700 1 -- 172.21.15.26:0/331015835 shutdown_connections 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.691+0000 7f6ce57ff700 1 -- 172.21.15.26:0/331015835 wait complete. 2023-07-13T22:18:38.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.692+0000 7f6ce57ff700 1 Processor -- start 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.693+0000 7f6ce57ff700 1 -- start start 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.693+0000 7f6ce57ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 0x7f6ce0114be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.693+0000 7f6ce57ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ce0071830 0x7f6ce0115120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6ce57ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce0116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6ce57ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0116730 con 0x7f6ce0071830 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6ce57ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce01168a0 con 0x7f6ce0070ee0 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6ce57ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0116a60 con 0x7f6ce010bd60 2023-07-13T22:18:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 0x7f6ce0114be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 0x7f6ce0114be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60628/0 (socket says 172.21.15.26:60628) 2023-07-13T22:18:38.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6cdeffd700 1 -- 172.21.15.26:0/1790923568 learned_addr learned my addr 172.21.15.26:0/1790923568 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:38.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.694+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce0116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.695+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1790923568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 msgr2=0x7f6ce0114be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.695+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 0x7f6ce0114be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.695+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ce0071830 msgr2=0x7f6ce0115120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.695+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ce0071830 0x7f6ce0115120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.695+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1790923568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd40095f0 con 0x7f6ce010bd60 2023-07-13T22:18:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.696+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce0116160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8007f10 tx=0x7f6cd800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.708+0000 7f6cc7fff700 1 -- 172.21.15.26:0/1790923568 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd800ddd0 con 0x7f6ce010bd60 2023-07-13T22:18:38.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.708+0000 7f6cc7fff700 1 -- 172.21.15.26:0/1790923568 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd800f040 con 0x7f6ce010bd60 2023-07-13T22:18:38.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.708+0000 7f6cc7fff700 1 -- 172.21.15.26:0/1790923568 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8005a90 con 0x7f6ce010bd60 2023-07-13T22:18:38.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.708+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce0116ce0 con 0x7f6ce010bd60 2023-07-13T22:18:38.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.708+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce01171a0 con 0x7f6ce010bd60 2023-07-13T22:18:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.715+0000 7f6cf91e1b80 1 Processor -- start 2023-07-13T22:18:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 -- start start 2023-07-13T22:18:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce714d0 0x55dc4ce718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cd76cb0 con 0x55dc4ce714d0 2023-07-13T22:18:38.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cdc9a40 con 0x55dc4ce71e20 2023-07-13T22:18:38.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.716+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cdc2ed0 con 0x55dc4ce72c70 2023-07-13T22:18:38.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.724+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.724+0000 7f6cde7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.724+0000 7f6cde7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54248/0 (socket says 172.21.15.26:54248) 2023-07-13T22:18:38.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.724+0000 7f6cde7fc700 1 -- 172.21.15.26:0/111157861 learned_addr learned my addr 172.21.15.26:0/111157861 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cc7fff700 1 -- 172.21.15.26:0/1790923568 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6cd8004ad0 con 0x7f6ce010bd60 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cde7fc700 1 -- 172.21.15.26:0/111157861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 msgr2=0x55dc4ce726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cde7fc700 1 -- 172.21.15.26:0/111157861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce714d0 msgr2=0x55dc4ce718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce714d0 0x55dc4ce718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.732+0000 7f6cde7fc700 1 -- 172.21.15.26:0/111157861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc4cbf0de0 con 0x55dc4ce72c70 2023-07-13T22:18:38.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.733+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005c80 tx=0x7f6cd4009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d576f0db91a55d66 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.734+0000 7f6cc7fff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 0x7f6cc80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.734+0000 7f6cc7fff700 1 -- 172.21.15.26:0/1790923568 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cd80cc0b0 con 0x7f6ce010bd60 2023-07-13T22:18:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.737+0000 7f6cc57fa700 1 -- 172.21.15.26:0/111157861 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400b9d0 con 0x55dc4ce72c70 2023-07-13T22:18:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.738+0000 7f6cc57fa700 1 -- 172.21.15.26:0/111157861 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd400be80 con 0x55dc4ce72c70 2023-07-13T22:18:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.738+0000 7f6cc57fa700 1 -- 172.21.15.26:0/111157861 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40056c0 con 0x55dc4ce72c70 2023-07-13T22:18:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.746+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/111157861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 msgr2=0x55dc4ce730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.746+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4005c80 tx=0x7f6cd4009ee0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.746+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 0x7f6cc80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/111157861 shutdown_connections 2023-07-13T22:18:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce714d0 0x55dc4ce718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce730e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/111157861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/111157861 >> 172.21.15.26:0/111157861 conn(0x55dc4ce66da0 msgr2=0x55dc4ce6c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/111157861 shutdown_connections 2023-07-13T22:18:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.747+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/111157861 wait complete. 2023-07-13T22:18:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.748+0000 7f6cf91e1b80 1 -- start start 2023-07-13T22:18:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.749+0000 7f6cf91e1b80 1 Processor -- start 2023-07-13T22:18:38.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce71e20 0x55dc4ce7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cd62ac0 con 0x55dc4ce71e20 2023-07-13T22:18:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cd627c0 con 0x55dc4ce714d0 2023-07-13T22:18:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cc46920 con 0x55dc4ce72c70 2023-07-13T22:18:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.761+0000 7f6cf91e1b80 10 client.0 _ll_register_callbacks cb 0x55dc4cd60760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:38.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.762+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.762+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60642/0 (socket says 172.21.15.26:60642) 2023-07-13T22:18:38.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.762+0000 7f6cdeffd700 1 -- 172.21.15.26:0/3382217650 learned_addr learned my addr 172.21.15.26:0/3382217650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:38.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.762+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.763+0000 7f6cdeffd700 1 -- 172.21.15.26:0/3382217650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 msgr2=0x55dc4ce82690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.763+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.763+0000 7f6cdeffd700 1 -- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce71e20 msgr2=0x55dc4ce7b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.763+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce71e20 0x55dc4ce7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.763+0000 7f6cdeffd700 1 -- 172.21.15.26:0/3382217650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce0117640 con 0x55dc4ce714d0 2023-07-13T22:18:38.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.764+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00104e0 tx=0x7f6cd00108a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15506a464fa12365 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.765+0000 7f6cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:38.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.765+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd000b570 con 0x55dc4ce714d0 2023-07-13T22:18:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.765+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cd0004830 con 0x55dc4ce714d0 2023-07-13T22:18:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.765+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc4cd1c4f0 con 0x55dc4ce714d0 2023-07-13T22:18:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.771+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd0020420 con 0x55dc4ce714d0 2023-07-13T22:18:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.771+0000 7f6cf91e1b80 10 client.26186 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:38.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.771+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc4cbd9c60 con 0x55dc4ce714d0 2023-07-13T22:18:38.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.776+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 0x7f6cc80a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c010 tx=0x7f6cd0017000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.783+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc4cd1c4f0 con 0x55dc4ce714d0 2023-07-13T22:18:38.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.783+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6cd00294f0 con 0x55dc4ce714d0 2023-07-13T22:18:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.784+0000 7f6cf91e1b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.784+0000 7f6cf91e1b80 10 client.26186 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:18:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.785+0000 7f6cf91e1b80 20 client.26186 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:38.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.785+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4ce86ec0 0x55dc4ce892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.785+0000 7f6cf91e1b80 4 client.26186 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:38.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.785+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc4cdbbeb0 con 0x55dc4ce86ec0 2023-07-13T22:18:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.790+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4ce86ec0 0x55dc4ce892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.791+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4ce86ec0 0x55dc4ce892d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd70093d572097e8 server_cookie=b2dfd1e63d1e64cd in_seq=0 out_seq=0 2023-07-13T22:18:38.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.792+0000 7f6cc2ff5700 10 client.26186 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:18:38.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.793+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/3382217650 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc4cdbbeb0 con 0x55dc4ce86ec0 2023-07-13T22:18:38.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.793+0000 7f6cc2ff5700 10 client.26186 handle_command_reply: tid=0 2023-07-13T22:18:38.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.794+0000 7f6cf91e1b80 1 client.26186 shutdown 2023-07-13T22:18:38.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 msgr2=0x55dc4ce7b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00104e0 tx=0x7f6cd00108a0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 shutdown_connections 2023-07-13T22:18:38.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce714d0 0x55dc4ce7b110 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00104e0 tx=0x7f6cd00108a0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4ce86ec0 0x55dc4ce892d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.804+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/3382217650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce71e20 0x55dc4ce7b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.805+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 >> 172.21.15.26:0/3382217650 conn(0x55dc4ce66da0 msgr2=0x55dc4ce6b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.805+0000 7f6cc2ff5700 0 client.26186 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:18:38.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.813+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 shutdown_connections 2023-07-13T22:18:38.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.826+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/3382217650 wait complete. 2023-07-13T22:18:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.827+0000 7f6cf91e1b80 20 client.26186 trim_cache size 0 max 16384 2023-07-13T22:18:38.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.855+0000 7f6cf91e1b80 1 Processor -- start 2023-07-13T22:18:38.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 -- start start 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce71e20 0x55dc4ce73080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ce669c0 con 0x55dc4ce72c70 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ce6d4c0 con 0x55dc4ce79000 2023-07-13T22:18:38.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.856+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ce7c2e0 con 0x55dc4ce71e20 2023-07-13T22:18:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.857+0000 7f6cdf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.857+0000 7f6cdf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60658/0 (socket says 172.21.15.26:60658) 2023-07-13T22:18:38.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.857+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1777898043 learned_addr learned my addr 172.21.15.26:0/1777898043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1777898043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce71e20 msgr2=0x55dc4ce73080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce71e20 0x55dc4ce73080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1777898043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce72c70 msgr2=0x55dc4ce82620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.858+0000 7f6cdf7fe700 1 -- 172.21.15.26:0/1777898043 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc4cc17250 con 0x55dc4ce79000 2023-07-13T22:18:38.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.859+0000 7f6cdf7fe700 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8095ee0 tx=0x7f6cd80992c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5c3af908d6b6f16 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.871+0000 7f6cc5ffb700 1 -- 172.21.15.26:0/1777898043 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8009e20 con 0x55dc4ce79000 2023-07-13T22:18:38.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.871+0000 7f6cc5ffb700 1 -- 172.21.15.26:0/1777898043 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd8099470 con 0x55dc4ce79000 2023-07-13T22:18:38.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.871+0000 7f6cc5ffb700 1 -- 172.21.15.26:0/1777898043 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80996d0 con 0x55dc4ce79000 2023-07-13T22:18:38.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.872+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/1777898043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 msgr2=0x55dc4ce79470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.872+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8095ee0 tx=0x7f6cd80992c0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.873+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/1777898043 shutdown_connections 2023-07-13T22:18:38.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.873+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce79000 0x55dc4ce79470 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.873+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce71e20 0x55dc4ce73080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.873+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/1777898043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce72c70 0x55dc4ce82620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.873+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/1777898043 >> 172.21.15.26:0/1777898043 conn(0x55dc4ce66da0 msgr2=0x55dc4ce79e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.882+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/1777898043 shutdown_connections 2023-07-13T22:18:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.882+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/1777898043 wait complete. 2023-07-13T22:18:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.883+0000 7f6cf91e1b80 1 -- start start 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.883+0000 7f6cf91e1b80 1 Processor -- start 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce7efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce79000 0x55dc4ce7f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ce7fbe0 con 0x55dc4ce79000 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cedad90 con 0x55dc4ce71e20 2023-07-13T22:18:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4cedaf00 con 0x55dc4ce72c70 2023-07-13T22:18:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cf91e1b80 10 client.0 _ll_register_callbacks cb 0x7f6cd0011aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.884+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60672/0 (socket says 172.21.15.26:60672) 2023-07-13T22:18:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 -- 172.21.15.26:0/738987936 learned_addr learned my addr 172.21.15.26:0/738987936 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 -- 172.21.15.26:0/738987936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 msgr2=0x55dc4ce7efc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce7efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 -- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce79000 msgr2=0x55dc4ce7f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce79000 0x55dc4ce7f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.885+0000 7f6cdeffd700 1 -- 172.21.15.26:0/738987936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce01171a0 con 0x55dc4ce71e20 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.886+0000 7f6cdeffd700 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0010ca0 tx=0x7f6cd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67a838e78bc09bfd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.886+0000 7f6cc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.886+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0005230 con 0x55dc4ce71e20 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.886+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6cd0005390 con 0x55dc4ce71e20 2023-07-13T22:18:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.886+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cd400be80 con 0x55dc4ce71e20 2023-07-13T22:18:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.887+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd0005910 con 0x55dc4ce71e20 2023-07-13T22:18:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.888+0000 7f6cf91e1b80 10 client.26192 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.888+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc4cbef2b0 con 0x55dc4ce71e20 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cd400be80 con 0x55dc4ce71e20 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6cd000d590 con 0x55dc4ce71e20 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 10 client.26192 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 10 client.26192 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 20 client.26192 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4cedc990 0x55dc4cededa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 4 client.26192 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:18:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.889+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc4cd7e050 con 0x55dc4cedc990 2023-07-13T22:18:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.893+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4cedc990 0x55dc4cededa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.895+0000 7f6cde7fc700 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4cedc990 0x55dc4cededa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b00bcf7f6e9afb20 server_cookie=468c88bf4e9a932c in_seq=0 out_seq=0 2023-07-13T22:18:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.895+0000 7f6cc2ff5700 10 client.26192 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:18:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.896+0000 7f6cc2ff5700 1 -- 172.21.15.26:0/738987936 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc4cd7e050 con 0x55dc4cedc990 2023-07-13T22:18:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cc2ff5700 10 client.26192 handle_command_reply: tid=0 2023-07-13T22:18:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cf91e1b80 1 client.26192 shutdown 2023-07-13T22:18:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 msgr2=0x55dc4ce82520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0010ca0 tx=0x7f6cd000b330 comp rx=0 tx=0).stop 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 shutdown_connections 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.897+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55dc4cedc990 0x55dc4cededa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55dc4ce79000 0x55dc4ce7f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dc4ce71e20 0x55dc4ce82520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cc2ff5700 0 client.26192 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:18:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 --2- 172.21.15.26:0/738987936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dc4ce72c70 0x55dc4ce7efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 >> 172.21.15.26:0/738987936 conn(0x55dc4ce66da0 msgr2=0x55dc4ce75b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 shutdown_connections 2023-07-13T22:18:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.898+0000 7f6cf91e1b80 1 -- 172.21.15.26:0/738987936 wait complete. 2023-07-13T22:18:38.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.905+0000 7f6cf91e1b80 20 client.26192 trim_cache size 0 max 16384 2023-07-13T22:18:38.892 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:18:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.906+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 msgr2=0x7f6cc80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.906+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 0x7f6cc80a3800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c010 tx=0x7f6cd0017000 comp rx=0 tx=0).stop 2023-07-13T22:18:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.906+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 msgr2=0x7f6ce0116160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:38.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.906+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce0116160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8007f10 tx=0x7f6cd800d3b0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 shutdown_connections 2023-07-13T22:18:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ce010bd60 0x7f6ce0116160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ce0070ee0 0x7f6ce0114be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6cc80a1350 0x7f6cc80a3800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 --2- 172.21.15.26:0/1790923568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ce0071830 0x7f6ce0115120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.911+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 >> 172.21.15.26:0/1790923568 conn(0x7f6ce006ca90 msgr2=0x7f6ce006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.922+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 shutdown_connections 2023-07-13T22:18:38.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:38.922+0000 7f6ce57ff700 1 -- 172.21.15.26:0/1790923568 wait complete. 2023-07-13T22:18:38.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:18:39.288 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:39 smithi026 ceph-mon[95647]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 95 op/s 2023-07-13T22:18:39.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:39 smithi117 ceph-mon[129074]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 95 op/s 2023-07-13T22:18:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.673+0000 7f21053ff700 1 -- 172.21.15.26:0/3321171678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2100070ee0 msgr2=0x7f21000712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.673+0000 7f21053ff700 1 --2- 172.21.15.26:0/3321171678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2100070ee0 0x7f21000712f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009990 tx=0x7f20f0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 -- 172.21.15.26:0/3321171678 shutdown_connections 2023-07-13T22:18:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 --2- 172.21.15.26:0/3321171678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f210010bd60 0x7f210010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 --2- 172.21.15.26:0/3321171678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2100070ee0 0x7f21000712f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 --2- 172.21.15.26:0/3321171678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100071830 0x7f210010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 -- 172.21.15.26:0/3321171678 >> 172.21.15.26:0/3321171678 conn(0x7f210006ca90 msgr2=0x7f210006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 -- 172.21.15.26:0/3321171678 shutdown_connections 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.674+0000 7f21053ff700 1 -- 172.21.15.26:0/3321171678 wait complete. 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 Processor -- start 2023-07-13T22:18:39.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 -- start start 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2100071830 0x7f2100077a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f210010bd60 0x7f210007eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210007f0c0 con 0x7f2100071830 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210007f230 con 0x7f2100070ee0 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.676+0000 7f21053ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210007f3f0 con 0x7f210010bd60 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.677+0000 7f20feffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.677+0000 7f20feffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60678/0 (socket says 172.21.15.26:60678) 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.677+0000 7f20feffd700 1 -- 172.21.15.26:0/1788418044 learned_addr learned my addr 172.21.15.26:0/1788418044 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f210010bd60 0x7f210007eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20feffd700 1 -- 172.21.15.26:0/1788418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f210010bd60 msgr2=0x7f210007eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20feffd700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f210010bd60 0x7f210007eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20feffd700 1 -- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2100071830 msgr2=0x7f2100077a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20feffd700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2100071830 0x7f2100077a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.678+0000 7f20feffd700 1 -- 172.21.15.26:0/1788418044 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00095f0 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.679+0000 7f20feffd700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20f000b680 tx=0x7f20f00057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.679+0000 7f20effff700 1 -- 172.21.15.26:0/1788418044 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0061070 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.679+0000 7f20effff700 1 -- 172.21.15.26:0/1788418044 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f000fbc0 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.679+0000 7f20effff700 1 -- 172.21.15.26:0/1788418044 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f000e640 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.679+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210007f610 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.680+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210007faf0 con 0x7f2100070ee0 2023-07-13T22:18:39.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.687+0000 7f2118ca8b80 1 Processor -- start 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.689+0000 7f2118ca8b80 1 -- start start 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617df4d0 0x557d617df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617dfe20 0x557d617e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d616e4cb0 con 0x557d617dfe20 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d61737a40 con 0x557d617df4d0 2023-07-13T22:18:39.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d61730ed0 con 0x557d617e0c70 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54274/0 (socket says 172.21.15.26:54274) 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.690+0000 7f20fe7fc700 1 -- 172.21.15.26:0/3740071684 learned_addr learned my addr 172.21.15.26:0/3740071684 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.692+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617df4d0 0x557d617df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.697+0000 7f20fe7fc700 1 -- 172.21.15.26:0/3740071684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617df4d0 msgr2=0x557d617df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.697+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617df4d0 0x557d617df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.697+0000 7f20fe7fc700 1 -- 172.21.15.26:0/3740071684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617dfe20 msgr2=0x557d617e06e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.697+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617dfe20 0x557d617e06e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.707+0000 7f20fe7fc700 1 -- 172.21.15.26:0/3740071684 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d6155ede0 con 0x557d617e0c70 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.707+0000 7f20effff700 1 -- 172.21.15.26:0/1788418044 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f20f005b400 con 0x7f2100070ee0 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.710+0000 7f20effff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 0x7f20e80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.710+0000 7f20effff700 1 -- 172.21.15.26:0/1788418044 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f0111e90 con 0x7f2100070ee0 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.713+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 0x7f20e80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.713+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009c80 tx=0x7f20f4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b521b8f4a12b50 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.715+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 0x7f20e80a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20f800a510 tx=0x7f20f800d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.715+0000 7f20ed7fa700 1 -- 172.21.15.26:0/3740071684 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4009800 con 0x557d617e0c70 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.715+0000 7f20ed7fa700 1 -- 172.21.15.26:0/3740071684 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f4004cf0 con 0x557d617e0c70 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.715+0000 7f20ed7fa700 1 -- 172.21.15.26:0/3740071684 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4098440 con 0x557d617e0c70 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3740071684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 msgr2=0x557d617e10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009c80 tx=0x7f20f4009f90 comp rx=0 tx=0).stop 2023-07-13T22:18:39.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3740071684 shutdown_connections 2023-07-13T22:18:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617e0c70 0x557d617e10e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617df4d0 0x557d617df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3740071684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617dfe20 0x557d617e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.716+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3740071684 >> 172.21.15.26:0/3740071684 conn(0x557d617d4da0 msgr2=0x557d617da1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.717+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3740071684 shutdown_connections 2023-07-13T22:18:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.717+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3740071684 wait complete. 2023-07-13T22:18:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.718+0000 7f2118ca8b80 1 -- start start 2023-07-13T22:18:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.718+0000 7f2118ca8b80 1 Processor -- start 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617e9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d616d0ac0 con 0x557d617df4d0 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d616d07c0 con 0x557d617e0c70 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d615b4920 con 0x557d617dfe20 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.719+0000 7f2118ca8b80 10 client.0 _ll_register_callbacks cb 0x557d616ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:39.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.722+0000 7f20ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.722+0000 7f20ff7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60698/0 (socket says 172.21.15.26:60698) 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.722+0000 7f20ff7fe700 1 -- 172.21.15.26:0/1571217896 learned_addr learned my addr 172.21.15.26:0/1571217896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.722+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.723+0000 7f20ff7fe700 1 -- 172.21.15.26:0/1571217896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 msgr2=0x557d617e9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.723+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617e9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.723+0000 7f20ff7fe700 1 -- 172.21.15.26:0/1571217896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 msgr2=0x557d617e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.723+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.723+0000 7f20ff7fe700 1 -- 172.21.15.26:0/1571217896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f4009960 con 0x557d617e0c70 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.724+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f20f800afd0 tx=0x7f20f8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89469985cefa0971 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.725+0000 7f20e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.725+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f80167e0 con 0x557d617e0c70 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.725+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f8006e40 con 0x557d617e0c70 2023-07-13T22:18:39.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.725+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d6168a4f0 con 0x557d617e0c70 2023-07-13T22:18:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f8016940 con 0x557d617e0c70 2023-07-13T22:18:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 10 client.26204 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d61547c60 con 0x557d617e0c70 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d6168a4f0 con 0x557d617e0c70 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20f802b070 con 0x557d617e0c70 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 10 client.26204 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 20 client.26204 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.735+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d617f4f90 0x557d617f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.736+0000 7f2118ca8b80 4 client.26204 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.736+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d61729eb0 con 0x557d617f4f90 2023-07-13T22:18:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.737+0000 7f20feffd700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d617f4f90 0x557d617f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.738+0000 7f20feffd700 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d617f4f90 0x557d617f73a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a50efa08605c922 server_cookie=49113ade701c315d in_seq=0 out_seq=0 2023-07-13T22:18:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.739+0000 7f20e67fc700 10 client.26204 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.740+0000 7f20e67fc700 1 -- 172.21.15.26:0/1571217896 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d61729eb0 con 0x557d617f4f90 2023-07-13T22:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.740+0000 7f20e67fc700 10 client.26204 handle_command_reply: tid=0 2023-07-13T22:18:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.740+0000 7f2118ca8b80 1 client.26204 shutdown 2023-07-13T22:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.741+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 msgr2=0x557d617f0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.741+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f20f800afd0 tx=0x7f20f8004ad0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 shutdown_connections 2023-07-13T22:18:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d617f4f90 0x557d617f73a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617f0780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/1571217896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617e9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 >> 172.21.15.26:0/1571217896 conn(0x557d617d4da0 msgr2=0x557d617d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.744+0000 7f20e67fc700 0 client.26204 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:39.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.745+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 shutdown_connections 2023-07-13T22:18:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.745+0000 7f2118ca8b80 1 -- 172.21.15.26:0/1571217896 wait complete. 2023-07-13T22:18:39.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.746+0000 7f2118ca8b80 20 client.26204 trim_cache size 0 max 16384 2023-07-13T22:18:39.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:39 smithi121 ceph-mon[129187]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 292 MiB/s wr, 95 op/s 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.774+0000 7f2118ca8b80 1 Processor -- start 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 -- start start 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617df4d0 0x557d617f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617e0c70 0x557d617e7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617f4ad0 con 0x557d617e0c70 2023-07-13T22:18:39.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617f4cb0 con 0x557d617dfe20 2023-07-13T22:18:39.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.775+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617dbc60 con 0x557d617df4d0 2023-07-13T22:18:39.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.777+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.777+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60702/0 (socket says 172.21.15.26:60702) 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.777+0000 7f20fe7fc700 1 -- 172.21.15.26:0/502562002 learned_addr learned my addr 172.21.15.26:0/502562002 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.777+0000 7f20feffd700 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617df4d0 0x557d617f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.778+0000 7f20fe7fc700 1 -- 172.21.15.26:0/502562002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617df4d0 msgr2=0x557d617f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.778+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617df4d0 0x557d617f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.778+0000 7f20fe7fc700 1 -- 172.21.15.26:0/502562002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617e0c70 msgr2=0x557d617e7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.778+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617e0c70 0x557d617e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.778+0000 7f20fe7fc700 1 -- 172.21.15.26:0/502562002 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d61585250 con 0x557d617dfe20 2023-07-13T22:18:39.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.779+0000 7f20fe7fc700 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009c80 tx=0x7f20f4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d811ed07a18efde server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.796+0000 7f20edffb700 1 -- 172.21.15.26:0/502562002 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4004a30 con 0x557d617dfe20 2023-07-13T22:18:39.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.796+0000 7f20edffb700 1 -- 172.21.15.26:0/502562002 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f4004e60 con 0x557d617dfe20 2023-07-13T22:18:39.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.796+0000 7f20edffb700 1 -- 172.21.15.26:0/502562002 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40ab480 con 0x557d617dfe20 2023-07-13T22:18:39.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.798+0000 7f2118ca8b80 1 -- 172.21.15.26:0/502562002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 msgr2=0x557d617f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.798+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009c80 tx=0x7f20f4005dc0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.799+0000 7f2118ca8b80 1 -- 172.21.15.26:0/502562002 shutdown_connections 2023-07-13T22:18:39.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.799+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617e0c70 0x557d617e7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.799+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617df4d0 0x557d617f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.799+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/502562002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617dfe20 0x557d617f06a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.799+0000 7f2118ca8b80 1 -- 172.21.15.26:0/502562002 >> 172.21.15.26:0/502562002 conn(0x557d617d4da0 msgr2=0x557d617e7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.800+0000 7f2118ca8b80 1 -- 172.21.15.26:0/502562002 shutdown_connections 2023-07-13T22:18:39.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.800+0000 7f2118ca8b80 1 -- 172.21.15.26:0/502562002 wait complete. 2023-07-13T22:18:39.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.801+0000 7f2118ca8b80 1 -- start start 2023-07-13T22:18:39.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.801+0000 7f2118ca8b80 1 Processor -- start 2023-07-13T22:18:39.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617efa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617effc0 con 0x557d617df4d0 2023-07-13T22:18:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617f0130 con 0x557d617e0c70 2023-07-13T22:18:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d617f02f0 con 0x557d617dfe20 2023-07-13T22:18:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.802+0000 7f2118ca8b80 10 client.0 _ll_register_callbacks cb 0x7f20f800acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.803+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.803+0000 7f20fe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617ece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54294/0 (socket says 172.21.15.26:54294) 2023-07-13T22:18:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.803+0000 7f20fe7fc700 1 -- 172.21.15.26:0/3529911299 learned_addr learned my addr 172.21.15.26:0/3529911299 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.803+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 -- 172.21.15.26:0/3529911299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 msgr2=0x557d617ece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617ece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 -- 172.21.15.26:0/3529911299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 msgr2=0x557d617ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 -- 172.21.15.26:0/3529911299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d6155ede0 con 0x557d617e0c70 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.804+0000 7f20ff7fe700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617efa80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009b60 tx=0x7f20f8018f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb5dc035a2817c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.805+0000 7f20e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.805+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f802cdb0 con 0x557d617e0c70 2023-07-13T22:18:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.805+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20f802c3f0 con 0x557d617e0c70 2023-07-13T22:18:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.805+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20f4004cf0 con 0x557d617e0c70 2023-07-13T22:18:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.809+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20f8023400 con 0x557d617e0c70 2023-07-13T22:18:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.809+0000 7f2118ca8b80 10 client.26210 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.809+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557d6155d2b0 con 0x557d617e0c70 2023-07-13T22:18:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.816+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f4004cf0 con 0x557d617e0c70 2023-07-13T22:18:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.816+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f20f8018770 con 0x557d617e0c70 2023-07-13T22:18:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 10 client.26210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 20 client.26210 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d61849de0 0x557d6184c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 4 client.26210 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:18:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d616ec050 con 0x557d61849de0 2023-07-13T22:18:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.817+0000 7f20feffd700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d61849de0 0x557d6184c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.825+0000 7f20feffd700 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d61849de0 0x557d6184c1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e1228269c08c7e server_cookie=f9bc6a1a1ac60d77 in_seq=0 out_seq=0 2023-07-13T22:18:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.825+0000 7f20e67fc700 10 client.26210 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.829+0000 7f20e67fc700 1 -- 172.21.15.26:0/3529911299 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d616ec050 con 0x557d61849de0 2023-07-13T22:18:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.829+0000 7f20e67fc700 10 client.26210 handle_command_reply: tid=0 2023-07-13T22:18:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.829+0000 7f2118ca8b80 1 client.26210 shutdown 2023-07-13T22:18:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 msgr2=0x557d617efa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617efa80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009b60 tx=0x7f20f8018f10 comp rx=0 tx=0).stop 2023-07-13T22:18:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 shutdown_connections 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557d617df4d0 0x557d617ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557d61849de0 0x557d6184c1f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557d617e0c70 0x557d617efa80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 --2- 172.21.15.26:0/3529911299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557d617dfe20 0x557d617ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 >> 172.21.15.26:0/3529911299 conn(0x557d617d4da0 msgr2=0x557d617f1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.830+0000 7f20e67fc700 0 client.26210 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.831+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 shutdown_connections 2023-07-13T22:18:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.831+0000 7f2118ca8b80 1 -- 172.21.15.26:0/3529911299 wait complete. 2023-07-13T22:18:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.832+0000 7f2118ca8b80 20 client.26210 trim_cache size 0 max 16384 2023-07-13T22:18:39.819 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:18:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 msgr2=0x7f20e80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 0x7f20e80a3800 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20f800a510 tx=0x7f20f800d040 comp rx=0 tx=0).stop 2023-07-13T22:18:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 msgr2=0x7f2100077540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20f000b680 tx=0x7f20f00057b0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 shutdown_connections 2023-07-13T22:18:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f210010bd60 0x7f210007eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2100070ee0 0x7f2100077540 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20f000b680 tx=0x7f20f00057b0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f20e80a1350 0x7f20e80a3800 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 --2- 172.21.15.26:0/1788418044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2100071830 0x7f2100077a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.833+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 >> 172.21.15.26:0/1788418044 conn(0x7f210006ca90 msgr2=0x7f210006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.834+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 shutdown_connections 2023-07-13T22:18:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:39.835+0000 7f21053ff700 1 -- 172.21.15.26:0/1788418044 wait complete. 2023-07-13T22:18:40.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:40 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:18:40.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:40 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:18:40.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:40 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:18:40.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:18:40.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.598+0000 7fbae3dff700 1 -- 172.21.15.26:0/129006575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 msgr2=0x7fbadc10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.598+0000 7fbae3dff700 1 --2- 172.21.15.26:0/129006575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc10e1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b0a0 tx=0x7fbad400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.599+0000 7fbae3dff700 1 -- 172.21.15.26:0/129006575 shutdown_connections 2023-07-13T22:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.599+0000 7fbae3dff700 1 --2- 172.21.15.26:0/129006575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbadc070ee0 0x7fbadc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.599+0000 7fbae3dff700 1 --2- 172.21.15.26:0/129006575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc10e1f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.599+0000 7fbae3dff700 1 --2- 172.21.15.26:0/129006575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 0x7fbadc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.599+0000 7fbae3dff700 1 -- 172.21.15.26:0/129006575 >> 172.21.15.26:0/129006575 conn(0x7fbadc06ca90 msgr2=0x7fbadc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.602+0000 7fbae3dff700 1 -- 172.21.15.26:0/129006575 shutdown_connections 2023-07-13T22:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.602+0000 7fbae3dff700 1 -- 172.21.15.26:0/129006575 wait complete. 2023-07-13T22:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.604+0000 7fbae3dff700 1 Processor -- start 2023-07-13T22:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.604+0000 7fbae3dff700 1 -- start start 2023-07-13T22:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbadc070ee0 0x7fbadc114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 0x7fbadc115170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc116780 con 0x7fbadc070ee0 2023-07-13T22:18:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc1168f0 con 0x7fbadc071830 2023-07-13T22:18:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.605+0000 7fbae3dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc116ab0 con 0x7fbadc10bd60 2023-07-13T22:18:41.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.606+0000 7fbae239c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.606+0000 7fbae239c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54320/0 (socket says 172.21.15.26:54320) 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.606+0000 7fbae239c700 1 -- 172.21.15.26:0/240731837 learned_addr learned my addr 172.21.15.26:0/240731837 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.606+0000 7fbae139a700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 0x7fbadc115170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.607+0000 7fbae239c700 1 -- 172.21.15.26:0/240731837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 msgr2=0x7fbadc115170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.607+0000 7fbae239c700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 0x7fbadc115170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.607+0000 7fbae239c700 1 -- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbadc070ee0 msgr2=0x7fbadc114c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.607+0000 7fbae239c700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbadc070ee0 0x7fbadc114c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.607+0000 7fbae239c700 1 -- 172.21.15.26:0/240731837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4009d00 con 0x7fbadc10bd60 2023-07-13T22:18:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.608+0000 7fbae239c700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b070 tx=0x7fbad40124a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.609+0000 7fbad2ffd700 1 -- 172.21.15.26:0/240731837 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad400e040 con 0x7fbadc10bd60 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.609+0000 7fbad2ffd700 1 -- 172.21.15.26:0/240731837 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbad4004e60 con 0x7fbadc10bd60 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.609+0000 7fbad2ffd700 1 -- 172.21.15.26:0/240731837 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40057b0 con 0x7fbadc10bd60 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.611+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc116cd0 con 0x7fbadc10bd60 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.611+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc117180 con 0x7fbadc10bd60 2023-07-13T22:18:41.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 Processor -- start 2023-07-13T22:18:41.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 -- start start 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a42cf0 0x557e29a43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.619+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29946d30 con 0x557e29a41ea0 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.620+0000 7fbae139a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.620+0000 7fbae139a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60736/0 (socket says 172.21.15.26:60736) 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.620+0000 7fbae139a700 1 -- 172.21.15.26:0/2862031398 learned_addr learned my addr 172.21.15.26:0/2862031398 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.621+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29999ac0 con 0x557e29a41550 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.621+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29992f50 con 0x557e29a42cf0 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.624+0000 7fbad2ffd700 1 -- 172.21.15.26:0/240731837 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbad4012700 con 0x7fbadc10bd60 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.624+0000 7fbae139a700 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a42cf0 0x557e29a43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.625+0000 7fbae139a700 1 -- 172.21.15.26:0/2862031398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a42cf0 msgr2=0x557e29a43160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.625+0000 7fbae139a700 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a42cf0 0x557e29a43160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.625+0000 7fbae139a700 1 -- 172.21.15.26:0/2862031398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 msgr2=0x557e29a42760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.625+0000 7fbae139a700 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.625+0000 7fbae139a700 1 -- 172.21.15.26:0/2862031398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e297c0e60 con 0x557e29a41550 2023-07-13T22:18:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.626+0000 7fbae139a700 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00afc0 tx=0x7fbacc00a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d459f6c26b4a4608 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.627+0000 7fbac7fff700 1 -- 172.21.15.26:0/2862031398 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc016b80 con 0x557e29a41550 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.627+0000 7fbac7fff700 1 -- 172.21.15.26:0/2862031398 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc00c030 con 0x557e29a41550 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.627+0000 7fbac7fff700 1 -- 172.21.15.26:0/2862031398 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc012370 con 0x557e29a41550 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.628+0000 7fbad2ffd700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 0x7fbac80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.628+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 msgr2=0x557e29a41960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.628+0000 7fbad2ffd700 1 -- 172.21.15.26:0/240731837 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbad40cd580 con 0x7fbadc10bd60 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.628+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00afc0 tx=0x7fbacc00a330 comp rx=0 tx=0).stop 2023-07-13T22:18:41.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.629+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 shutdown_connections 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.629+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a42cf0 0x557e29a43160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.629+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41550 0x557e29a41960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.629+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2862031398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a42760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.629+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 >> 172.21.15.26:0/2862031398 conn(0x557e29a36e20 msgr2=0x557e29a3c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.630+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 0x7fbac80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.631+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 0x7fbac80a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009670 tx=0x7fbad8006ca0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.636+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 shutdown_connections 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.636+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2862031398 wait complete. 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.637+0000 7fbaf778cb80 1 -- start start 2023-07-13T22:18:41.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.637+0000 7fbaf778cb80 1 Processor -- start 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a52810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29932b40 con 0x557e29a41ea0 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29932840 con 0x557e29a42cf0 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e298169a0 con 0x557e29a41550 2023-07-13T22:18:41.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.638+0000 7fbaf778cb80 10 client.0 _ll_register_callbacks cb 0x557e299307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.641+0000 7fbae139a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.641+0000 7fbae139a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56652/0 (socket says 172.21.15.26:56652) 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.641+0000 7fbae139a700 1 -- 172.21.15.26:0/2805543149 learned_addr learned my addr 172.21.15.26:0/2805543149 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.642+0000 7fbae139a700 1 -- 172.21.15.26:0/2805543149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 msgr2=0x557e29a4b290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:41.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.642+0000 7fbae139a700 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.642+0000 7fbae139a700 1 -- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 msgr2=0x557e29a52810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.642+0000 7fbae139a700 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a52810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.642+0000 7fbae139a700 1 -- 172.21.15.26:0/2805543149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc00e040 con 0x557e29a41ea0 2023-07-13T22:18:41.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.643+0000 7fbae139a700 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbacc039f60 tx=0x7fbacc006e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9abc0208f0b2ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.643+0000 7fbac57fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.646+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0393f0 con 0x557e29a41ea0 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.646+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbacc039550 con 0x557e29a41ea0 2023-07-13T22:18:41.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.646+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e298ec570 con 0x557e29a41ea0 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.647+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbacc039710 con 0x557e29a41ea0 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.647+0000 7fbaf778cb80 10 client.15924 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:41.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.647+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557e297a9ce0 con 0x557e29a41ea0 2023-07-13T22:18:41.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e298ec570 con 0x557e29a41ea0 2023-07-13T22:18:41.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbacc04b3c0 con 0x557e29a41ea0 2023-07-13T22:18:41.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:41.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:41.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 20 client.15924 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:41.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29a56dd0 0x557e29a591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 4 client.15924 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:41.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.648+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e2998bf30 con 0x557e29a56dd0 2023-07-13T22:18:41.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.650+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29a56dd0 0x557e29a591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.651+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29a56dd0 0x557e29a591e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3662a30131ed9e05 server_cookie=a19721508bbe85b6 in_seq=0 out_seq=0 2023-07-13T22:18:41.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.651+0000 7fbac57fa700 10 client.15924 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:41.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.651+0000 7fbac57fa700 1 -- 172.21.15.26:0/2805543149 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e2998bf30 con 0x557e29a56dd0 2023-07-13T22:18:41.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.652+0000 7fbac57fa700 10 client.15924 handle_command_reply: tid=0 2023-07-13T22:18:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.652+0000 7fbaf778cb80 1 client.15924 shutdown 2023-07-13T22:18:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.652+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 msgr2=0x557e29a4b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.652+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbacc039f60 tx=0x7fbacc006e30 comp rx=0 tx=0).stop 2023-07-13T22:18:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 shutdown_connections 2023-07-13T22:18:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29a56dd0 0x557e29a591e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a52810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/2805543149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a4b7d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 >> 172.21.15.26:0/2805543149 conn(0x557e29a36e20 msgr2=0x557e29a3c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbac57fa700 0 client.15924 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 shutdown_connections 2023-07-13T22:18:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.653+0000 7fbaf778cb80 1 -- 172.21.15.26:0/2805543149 wait complete. 2023-07-13T22:18:41.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.654+0000 7fbaf778cb80 20 client.15924 trim_cache size 0 max 16384 2023-07-13T22:18:41.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.691+0000 7fbaf778cb80 1 Processor -- start 2023-07-13T22:18:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 -- start start 2023-07-13T22:18:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a53100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41ea0 0x557e29a527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a42290 con 0x557e29a42cf0 2023-07-13T22:18:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a56bb0 con 0x557e29a41ea0 2023-07-13T22:18:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.692+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a3dce0 con 0x557e29a41550 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.693+0000 7fbae239c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.693+0000 7fbae239c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56668/0 (socket says 172.21.15.26:56668) 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.693+0000 7fbae239c700 1 -- 172.21.15.26:0/438080623 learned_addr learned my addr 172.21.15.26:0/438080623 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.693+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a53100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae139a700 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41ea0 0x557e29a527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 -- 172.21.15.26:0/438080623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 msgr2=0x557e29a53100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a53100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 -- 172.21.15.26:0/438080623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41ea0 msgr2=0x557e29a527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41ea0 0x557e29a527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 -- 172.21.15.26:0/438080623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e29a3b160 con 0x557e29a42cf0 2023-07-13T22:18:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.694+0000 7fbae239c700 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009bb0 tx=0x7fbad4005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b3908f66e38fbf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.695+0000 7fbad0bff700 1 -- 172.21.15.26:0/438080623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad409a330 con 0x557e29a42cf0 2023-07-13T22:18:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.695+0000 7fbad0bff700 1 -- 172.21.15.26:0/438080623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbad4005030 con 0x557e29a42cf0 2023-07-13T22:18:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.695+0000 7fbad0bff700 1 -- 172.21.15.26:0/438080623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad40cb140 con 0x557e29a42cf0 2023-07-13T22:18:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.696+0000 7fbaf778cb80 1 -- 172.21.15.26:0/438080623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 msgr2=0x557e29a4b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.696+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009bb0 tx=0x7fbad4005210 comp rx=0 tx=0).stop 2023-07-13T22:18:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 -- 172.21.15.26:0/438080623 shutdown_connections 2023-07-13T22:18:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a42cf0 0x557e29a4b040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a53100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/438080623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a41ea0 0x557e29a527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 -- 172.21.15.26:0/438080623 >> 172.21.15.26:0/438080623 conn(0x557e29a36e20 msgr2=0x557e29a49f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 -- 172.21.15.26:0/438080623 shutdown_connections 2023-07-13T22:18:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.698+0000 7fbaf778cb80 1 -- 172.21.15.26:0/438080623 wait complete. 2023-07-13T22:18:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.699+0000 7fbaf778cb80 1 -- start start 2023-07-13T22:18:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.700+0000 7fbaf778cb80 1 Processor -- start 2023-07-13T22:18:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a51900 con 0x557e29a41ea0 2023-07-13T22:18:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a51a70 con 0x557e29a42cf0 2023-07-13T22:18:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e29a51c30 con 0x557e29a41550 2023-07-13T22:18:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.701+0000 7fbaf778cb80 10 client.0 _ll_register_callbacks cb 0x7fbacc00ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:41.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.702+0000 7fbae239c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.702+0000 7fbae239c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60776/0 (socket says 172.21.15.26:60776) 2023-07-13T22:18:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.702+0000 7fbae239c700 1 -- 172.21.15.26:0/1068953557 learned_addr learned my addr 172.21.15.26:0/1068953557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.702+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.702+0000 7fbae139a700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 -- 172.21.15.26:0/1068953557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 msgr2=0x557e29a4fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 -- 172.21.15.26:0/1068953557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 msgr2=0x557e29a503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 -- 172.21.15.26:0/1068953557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc117180 con 0x557e29a42cf0 2023-07-13T22:18:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbae239c700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbad409a960 tx=0x7fbad4012cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db6b9bd37de19d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.703+0000 7fbac57fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad409ae30 con 0x557e29a42cf0 2023-07-13T22:18:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbad409a9d0 con 0x557e29a42cf0 2023-07-13T22:18:41.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbacc00c030 con 0x557e29a42cf0 2023-07-13T22:18:41.692 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:41 smithi117 ceph-mon[129074]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:18:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbad40b3120 con 0x557e29a42cf0 2023-07-13T22:18:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbaf778cb80 10 client.26219 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.704+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x557e297e72d0 con 0x557e29a42cf0 2023-07-13T22:18:41.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.705+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbacc00c030 con 0x557e29a42cf0 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.705+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbad40cd0d0 con 0x557e29a42cf0 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.706+0000 7fbaf778cb80 10 client.26219 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.706+0000 7fbaf778cb80 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.706+0000 7fbaf778cb80 20 client.26219 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.707+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29aacfe0 0x557e29aaf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:41.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.707+0000 7fbaf778cb80 4 client.26219 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:18:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.707+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557e2994e0d0 con 0x557e29aacfe0 2023-07-13T22:18:41.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.708+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29aacfe0 0x557e29aaf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:41.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:41 smithi026 ceph-mon[95647]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:18:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.732+0000 7fbac57fa700 10 client.26219 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.732+0000 7fbae1b9b700 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29aacfe0 0x557e29aaf3f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b810ce37ce88f50 server_cookie=4100bfe52c6af078 in_seq=0 out_seq=0 2023-07-13T22:18:41.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:41 smithi121 ceph-mon[129187]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:18:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.746+0000 7fbac57fa700 1 -- 172.21.15.26:0/1068953557 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557e2994e0d0 con 0x557e29aacfe0 2023-07-13T22:18:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.746+0000 7fbac57fa700 10 client.26219 handle_command_reply: tid=0 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.748+0000 7fbaf778cb80 1 client.26219 shutdown 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.751+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 msgr2=0x557e29a513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.751+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbad409a960 tx=0x7fbad4012cc0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 shutdown_connections 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x557e29aacfe0 0x557e29aaf3f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557e29a42cf0 0x557e29a513c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557e29a41550 0x557e29a4fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 --2- 172.21.15.26:0/1068953557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557e29a41ea0 0x557e29a503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 >> 172.21.15.26:0/1068953557 conn(0x557e29a36e20 msgr2=0x557e29a45c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.752+0000 7fbac57fa700 0 client.26219 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.754+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 shutdown_connections 2023-07-13T22:18:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.754+0000 7fbaf778cb80 1 -- 172.21.15.26:0/1068953557 wait complete. 2023-07-13T22:18:41.746 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:18:41.746 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:18:41.747 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "25bc73e1-7d94-4f8b-a4fc-5ec71e7e3cb7", 2023-07-13T22:18:41.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:18:41.747 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:18:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.759+0000 7fbaf778cb80 20 client.26219 trim_cache size 0 max 16384 2023-07-13T22:18:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.760+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 msgr2=0x7fbac80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.760+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 0x7fbac80a3800 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009670 tx=0x7fbad8006ca0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.760+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 msgr2=0x7fbadc1161b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:41.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.760+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbad400b070 tx=0x7fbad40124a0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 shutdown_connections 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbac80a1350 0x7fbac80a3800 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbadc070ee0 0x7fbadc114c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbadc10bd60 0x7fbadc1161b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 --2- 172.21.15.26:0/240731837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbadc071830 0x7fbadc115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.761+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 >> 172.21.15.26:0/240731837 conn(0x7fbadc06ca90 msgr2=0x7fbadc06dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.762+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 shutdown_connections 2023-07-13T22:18:41.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:41.762+0000 7fbae3dff700 1 -- 172.21.15.26:0/240731837 wait complete. 2023-07-13T22:18:41.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:18:42.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.516+0000 7fdc011ff700 1 -- 172.21.15.26:0/3402289415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc10bd60 msgr2=0x7fdbfc10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.516+0000 7fdc011ff700 1 --2- 172.21.15.26:0/3402289415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc10bd60 0x7fdbfc10e1f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400b0a0 tx=0x7fdbf400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.517+0000 7fdc011ff700 1 -- 172.21.15.26:0/3402289415 shutdown_connections 2023-07-13T22:18:42.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.517+0000 7fdc011ff700 1 --2- 172.21.15.26:0/3402289415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc10bd60 0x7fdbfc10e1f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.517+0000 7fdc011ff700 1 --2- 172.21.15.26:0/3402289415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.517+0000 7fdc011ff700 1 --2- 172.21.15.26:0/3402289415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc071830 0x7fdbfc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.517+0000 7fdc011ff700 1 -- 172.21.15.26:0/3402289415 >> 172.21.15.26:0/3402289415 conn(0x7fdbfc06ca90 msgr2=0x7fdbfc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.519+0000 7fdc011ff700 1 -- 172.21.15.26:0/3402289415 shutdown_connections 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.519+0000 7fdc011ff700 1 -- 172.21.15.26:0/3402289415 wait complete. 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 Processor -- start 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 -- start start 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbfc071830 0x7fdbfc115570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc10bd60 0x7fdbfc116440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc116ad0 con 0x7fdbfc070ee0 2023-07-13T22:18:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc116c40 con 0x7fdbfc071830 2023-07-13T22:18:42.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.521+0000 7fdc011ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc116e00 con 0x7fdbfc10bd60 2023-07-13T22:18:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.524+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.524+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56706/0 (socket says 172.21.15.26:56706) 2023-07-13T22:18:42.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.524+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1239062712 learned_addr learned my addr 172.21.15.26:0/1239062712 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:42.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.525+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1239062712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc10bd60 msgr2=0x7fdbfc116440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:42.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.525+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc10bd60 0x7fdbfc116440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.525+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1239062712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbfc071830 msgr2=0x7fdbfc115570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:18:42.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.525+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbfc071830 0x7fdbfc115570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.525+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1239062712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4009d00 con 0x7fdbfc070ee0 2023-07-13T22:18:42.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.526+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9a0 tx=0x7fdbec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.527+0000 7fdbe3fff700 1 -- 172.21.15.26:0/1239062712 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00cbd0 con 0x7fdbfc070ee0 2023-07-13T22:18:42.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.527+0000 7fdbe3fff700 1 -- 172.21.15.26:0/1239062712 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbec00cd30 con 0x7fdbfc070ee0 2023-07-13T22:18:42.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.527+0000 7fdbe3fff700 1 -- 172.21.15.26:0/1239062712 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec018680 con 0x7fdbfc070ee0 2023-07-13T22:18:42.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.527+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1170e0 con 0x7fdbfc070ee0 2023-07-13T22:18:42.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.527+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc078950 con 0x7fdbfc070ee0 2023-07-13T22:18:42.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.535+0000 7fdc14bbab80 1 Processor -- start 2023-07-13T22:18:42.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 -- start start 2023-07-13T22:18:42.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01be20 0x564f1a01c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a01d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f20cb0 con 0x564f1a01be20 2023-07-13T22:18:42.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f73a40 con 0x564f1a01cc70 2023-07-13T22:18:42.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.536+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f6ced0 con 0x564f1a01b4d0 2023-07-13T22:18:42.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.537+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a01d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.537+0000 7fdbfb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.537+0000 7fdbfb59e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54378/0 (socket says 172.21.15.26:54378) 2023-07-13T22:18:42.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.537+0000 7fdbfb59e700 1 -- 172.21.15.26:0/4144758052 learned_addr learned my addr 172.21.15.26:0/4144758052 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:42.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.543+0000 7fdbfb59e700 1 -- 172.21.15.26:0/4144758052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 msgr2=0x564f1a01d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.543+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a01d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.543+0000 7fdbfb59e700 1 -- 172.21.15.26:0/4144758052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01be20 msgr2=0x564f1a01c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.543+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01be20 0x564f1a01c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.543+0000 7fdbfb59e700 1 -- 172.21.15.26:0/4144758052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f19d9ade0 con 0x564f1a01b4d0 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.545+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009bd0 tx=0x7fdbf4009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49e4eae9808bffd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.545+0000 7fdbe17fa700 1 -- 172.21.15.26:0/4144758052 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf401f070 con 0x564f1a01b4d0 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.545+0000 7fdbe17fa700 1 -- 172.21.15.26:0/4144758052 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf400bb70 con 0x564f1a01b4d0 2023-07-13T22:18:42.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.545+0000 7fdbe17fa700 1 -- 172.21.15.26:0/4144758052 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400e040 con 0x564f1a01b4d0 2023-07-13T22:18:42.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.546+0000 7fdc14bbab80 1 -- 172.21.15.26:0/4144758052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 msgr2=0x564f1a01b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.546+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009bd0 tx=0x7fdbf4009450 comp rx=0 tx=0).stop 2023-07-13T22:18:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.549+0000 7fdbe3fff700 1 -- 172.21.15.26:0/1239062712 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdbec0187e0 con 0x7fdbfc070ee0 2023-07-13T22:18:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.552+0000 7fdbe3fff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 0x7fdbe40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.553+0000 7fdbe3fff700 1 -- 172.21.15.26:0/1239062712 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdbec014070 con 0x7fdbfc070ee0 2023-07-13T22:18:42.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 0x7fdbe40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdc14bbab80 1 -- 172.21.15.26:0/4144758052 shutdown_connections 2023-07-13T22:18:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a01d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a01b8e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/4144758052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01be20 0x564f1a01c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.561+0000 7fdc14bbab80 1 -- 172.21.15.26:0/4144758052 >> 172.21.15.26:0/4144758052 conn(0x564f1a010da0 msgr2=0x564f1a0161d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.564+0000 7fdc14bbab80 1 -- 172.21.15.26:0/4144758052 shutdown_connections 2023-07-13T22:18:42.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.564+0000 7fdc14bbab80 1 -- 172.21.15.26:0/4144758052 wait complete. 2023-07-13T22:18:42.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.565+0000 7fdc14bbab80 1 -- start start 2023-07-13T22:18:42.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.565+0000 7fdc14bbab80 1 Processor -- start 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.566+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 0x7fdbe40a3800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007680 tx=0x7fdbf400b3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a0256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01cc70 0x564f1a02c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f0cac0 con 0x564f1a01cc70 2023-07-13T22:18:42.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f0c7c0 con 0x564f1a01be20 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19df0920 con 0x564f1a01b4d0 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.568+0000 7fdc14bbab80 10 client.0 _ll_register_callbacks cb 0x564f19f0a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.569+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.570+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54380/0 (socket says 172.21.15.26:54380) 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.570+0000 7fdbfad9d700 1 -- 172.21.15.26:0/2747698873 learned_addr learned my addr 172.21.15.26:0/2747698873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:42.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.571+0000 7fdbfa59c700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a0256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.571+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01cc70 0x564f1a02c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.572+0000 7fdbfad9d700 1 -- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 msgr2=0x564f1a0256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.572+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a0256c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.573+0000 7fdbfad9d700 1 -- 172.21.15.26:0/2747698873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01cc70 msgr2=0x564f1a02c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.573+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01cc70 0x564f1a02c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.573+0000 7fdbfad9d700 1 -- 172.21.15.26:0/2747698873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc117640 con 0x564f1a01b4d0 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.573+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0a40a0 tx=0x7fdbec09bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a96c03c93f44cfa server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.573+0000 7fdbdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:18:42.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.574+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec09b870 con 0x564f1a01b4d0 2023-07-13T22:18:42.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.574+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdbec0a01b0 con 0x564f1a01b4d0 2023-07-13T22:18:42.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.574+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f19ec64f0 con 0x564f1a01b4d0 2023-07-13T22:18:42.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.576+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbec0a45d0 con 0x564f1a01b4d0 2023-07-13T22:18:42.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.577+0000 7fdc14bbab80 10 client.26197 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:42.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.577+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f19d83c60 con 0x564f1a01b4d0 2023-07-13T22:18:42.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.577+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f19ec64f0 con 0x564f1a01b4d0 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdbec09b2c0 con 0x564f1a01b4d0 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 10 client.26197 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 10 client.26197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 20 client.26197 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a030ef0 0x564f1a033300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 4 client.26197 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:18:42.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f19f65eb0 con 0x564f1a030ef0 2023-07-13T22:18:42.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.578+0000 7fdbfa59c700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a030ef0 0x564f1a033300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.580+0000 7fdbfa59c700 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a030ef0 0x564f1a033300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f24bdab2daeac35 server_cookie=19d0b595434059d9 in_seq=0 out_seq=0 2023-07-13T22:18:42.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.580+0000 7fdbdeff5700 10 client.26197 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:42.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2747698873 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f19f65eb0 con 0x564f1a030ef0 2023-07-13T22:18:42.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdbdeff5700 10 client.26197 handle_command_reply: tid=0 2023-07-13T22:18:42.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdc14bbab80 1 client.26197 shutdown 2023-07-13T22:18:42.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 msgr2=0x564f1a025180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdbec0a40a0 tx=0x7fdbec09bae0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.581+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 shutdown_connections 2023-07-13T22:18:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01cc70 0x564f1a02c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a030ef0 0x564f1a033300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01b4d0 0x564f1a025180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2747698873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a0256c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 >> 172.21.15.26:0/2747698873 conn(0x564f1a010da0 msgr2=0x564f1a0158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdbdeff5700 0 client.26197 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 shutdown_connections 2023-07-13T22:18:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.582+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2747698873 wait complete. 2023-07-13T22:18:42.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.583+0000 7fdc14bbab80 20 client.26197 trim_cache size 0 max 16384 2023-07-13T22:18:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.620+0000 7fdc14bbab80 1 Processor -- start 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.620+0000 7fdc14bbab80 1 -- start start 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a023010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a02c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01cc70 0x564f1a01fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a0109c0 con 0x564f1a01b4d0 2023-07-13T22:18:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a02e3f0 con 0x564f1a01be20 2023-07-13T22:18:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a0174c0 con 0x564f1a01cc70 2023-07-13T22:18:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a02c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a02c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60796/0 (socket says 172.21.15.26:60796) 2023-07-13T22:18:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.621+0000 7fdbfa59c700 1 -- 172.21.15.26:0/1596023441 learned_addr learned my addr 172.21.15.26:0/1596023441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a023010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01cc70 0x564f1a01fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1596023441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01cc70 msgr2=0x564f1a01fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01cc70 0x564f1a01fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1596023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 msgr2=0x564f1a02c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a02c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.622+0000 7fdbfad9d700 1 -- 172.21.15.26:0/1596023441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f1a01d310 con 0x564f1a01b4d0 2023-07-13T22:18:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.623+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a023010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdbec09c110 tx=0x7fdbec09c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd61dc06738e43 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.623+0000 7fdbe1ffb700 1 -- 172.21.15.26:0/1596023441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec018b30 con 0x564f1a01b4d0 2023-07-13T22:18:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.623+0000 7fdbe1ffb700 1 -- 172.21.15.26:0/1596023441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbec0a01b0 con 0x564f1a01b4d0 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.623+0000 7fdbe1ffb700 1 -- 172.21.15.26:0/1596023441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0b4b60 con 0x564f1a01b4d0 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 -- 172.21.15.26:0/1596023441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 msgr2=0x564f1a023010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a023010 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdbec09c110 tx=0x7fdbec09c300 comp rx=0 tx=0).stop 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 -- 172.21.15.26:0/1596023441 shutdown_connections 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a023010 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01cc70 0x564f1a01fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/1596023441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01be20 0x564f1a02c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.624+0000 7fdc14bbab80 1 -- 172.21.15.26:0/1596023441 >> 172.21.15.26:0/1596023441 conn(0x564f1a010da0 msgr2=0x564f1a023e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.625+0000 7fdc14bbab80 1 -- 172.21.15.26:0/1596023441 shutdown_connections 2023-07-13T22:18:42.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.625+0000 7fdc14bbab80 1 -- 172.21.15.26:0/1596023441 wait complete. 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.625+0000 7fdc14bbab80 1 -- start start 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 Processor -- start 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a028050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01be20 0x564f1a028590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a02baf0 con 0x564f1a01b4d0 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a02bc60 con 0x564f1a01cc70 2023-07-13T22:18:42.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a02be20 con 0x564f1a01be20 2023-07-13T22:18:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.626+0000 7fdc14bbab80 10 client.0 _ll_register_callbacks cb 0x7fdbec09bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:18:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60802/0 (socket says 172.21.15.26:60802) 2023-07-13T22:18:42.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 -- 172.21.15.26:0/2363981444 learned_addr learned my addr 172.21.15.26:0/2363981444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a028050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 -- 172.21.15.26:0/2363981444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01be20 msgr2=0x564f1a028590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01be20 0x564f1a028590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 -- 172.21.15.26:0/2363981444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 msgr2=0x564f1a028050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.627+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a028050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.628+0000 7fdbfb59e700 1 -- 172.21.15.26:0/2363981444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f19d83c60 con 0x564f1a01cc70 2023-07-13T22:18:42.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.629+0000 7fdbfb59e700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007840 tx=0x7fdbf4006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf95ec571984a30 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:18:42.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.629+0000 7fdbdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:18:42.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.634+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf401f040 con 0x564f1a01cc70 2023-07-13T22:18:42.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.634+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdbf4026480 con 0x564f1a01cc70 2023-07-13T22:18:42.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.634+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbf400bb70 con 0x564f1a01cc70 2023-07-13T22:18:42.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.635+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf400e040 con 0x564f1a01cc70 2023-07-13T22:18:42.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.635+0000 7fdc14bbab80 10 client.26234 fetch_fsmap learned FSMap version 29 2023-07-13T22:18:42.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.635+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f19dc1250 con 0x564f1a01cc70 2023-07-13T22:18:42.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.635+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbf400bb70 con 0x564f1a01cc70 2023-07-13T22:18:42.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdbf4026c00 con 0x564f1a01cc70 2023-07-13T22:18:42.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 10 client.26234 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:18:42.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:18:42.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 20 client.26234 populate_metadata read hostname 'smithi026' 2023-07-13T22:18:42.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a0855f0 0x564f1a087a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:18:42.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 4 client.26234 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:18:42.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.636+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f19f28050 con 0x564f1a0855f0 2023-07-13T22:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.638+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a0855f0 0x564f1a087a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:18:42.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.665+0000 7fdbfad9d700 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a0855f0 0x564f1a087a00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654d4fdb5837d675 server_cookie=3da64794e8b2b29b in_seq=0 out_seq=0 2023-07-13T22:18:42.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.665+0000 7fdbdeff5700 10 client.26234 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.666+0000 7fdbdeff5700 1 -- 172.21.15.26:0/2363981444 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f19f28050 con 0x564f1a0855f0 2023-07-13T22:18:42.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.666+0000 7fdbdeff5700 10 client.26234 handle_command_reply: tid=0 2023-07-13T22:18:42.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.667+0000 7fdc14bbab80 1 client.26234 shutdown 2023-07-13T22:18:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.668+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 msgr2=0x564f1a02b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.668+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007840 tx=0x7fdbf4006000 comp rx=0 tx=0).stop 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 shutdown_connections 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f1a01b4d0 0x564f1a028050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f1a0855f0 0x564f1a087a00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f1a01cc70 0x564f1a02b5b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 --2- 172.21.15.26:0/2363981444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f1a01be20 0x564f1a028590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 >> 172.21.15.26:0/2363981444 conn(0x564f1a010da0 msgr2=0x564f1a023e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.669+0000 7fdbdeff5700 0 client.26234 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:18:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.670+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 shutdown_connections 2023-07-13T22:18:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.670+0000 7fdc14bbab80 1 -- 172.21.15.26:0/2363981444 wait complete. 2023-07-13T22:18:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.670+0000 7fdc14bbab80 20 client.26234 trim_cache size 0 max 16384 2023-07-13T22:18:42.658 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:18:42.658 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:18:42.658 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:18:42.658 INFO:teuthology.orchestra.run.smithi026.stdout: "25bc73e1-7d94-4f8b-a4fc-5ec71e7e3cb7": { 2023-07-13T22:18:42.658 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:18:42.659 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "25bc73e1-7d94-4f8b-a4fc-5ec71e7e3cb7", 2023-07-13T22:18:42.659 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:18:42.659 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:18:42.659 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:18:42.659 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 msgr2=0x7fdbe40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 0x7fdbe40a3800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007680 tx=0x7fdbf400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 msgr2=0x7fdbfc114ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9a0 tx=0x7fdbec00ed60 comp rx=0 tx=0).stop 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 shutdown_connections 2023-07-13T22:18:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdbe40a1350 0x7fdbe40a3800 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbfc070ee0 0x7fdbfc114ff0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbfc10bd60 0x7fdbfc116440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 --2- 172.21.15.26:0/1239062712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdbfc071830 0x7fdbfc115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.672+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 >> 172.21.15.26:0/1239062712 conn(0x7fdbfc06ca90 msgr2=0x7fdbfc06dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.673+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 shutdown_connections 2023-07-13T22:18:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:18:42.673+0000 7fdc011ff700 1 -- 172.21.15.26:0/1239062712 wait complete. 2023-07-13T22:18:42.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:18:42.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:18:42.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:18:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:18:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:18:42.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:18:42.718 INFO:tasks.cephfs.filesystem:scrub status for tag:25bc73e1-7d94-4f8b-a4fc-5ec71e7e3cb7 - {'path': '/', 'tag': '25bc73e1-7d94-4f8b-a4fc-5ec71e7e3cb7', 'options': 'recursive,force'} 2023-07-13T22:18:42.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:18:42.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:18:42.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:18:43.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:18:43.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:18:43.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:18:43.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.691 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:43 smithi117 ceph-mon[129074]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:18:43.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:18:43.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:18:43.716 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:18:43.716 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.716 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.716 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:43 smithi026 ceph-mon[95647]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:18:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:18:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:18:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:18:43.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:18:43.742 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:43 smithi121 ceph-mon[129187]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:18:45.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:45 smithi026 ceph-mon[95647]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 98 op/s 2023-07-13T22:18:45.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:45 smithi121 ceph-mon[129187]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 98 op/s 2023-07-13T22:18:45.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:45 smithi117 ceph-mon[129074]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 98 op/s 2023-07-13T22:18:47.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:47 smithi117 ceph-mon[129074]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:18:47.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:47 smithi026 ceph-mon[95647]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:18:47.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:47 smithi121 ceph-mon[129187]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:18:49.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:49 smithi117 ceph-mon[129074]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 300 MiB/s wr, 97 op/s 2023-07-13T22:18:49.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:49 smithi026 ceph-mon[95647]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 300 MiB/s wr, 97 op/s 2023-07-13T22:18:49.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:49 smithi121 ceph-mon[129187]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 300 MiB/s wr, 97 op/s 2023-07-13T22:18:51.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:51 smithi117 ceph-mon[129074]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 97 op/s 2023-07-13T22:18:51.982 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:51 smithi026 ceph-mon[95647]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 97 op/s 2023-07-13T22:18:51.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:51 smithi121 ceph-mon[129187]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 97 op/s 2023-07-13T22:18:52.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:52 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:18:52.954 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:52 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:18:52.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:52 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:18:53.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:53 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:18:53.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:53 smithi117 ceph-mon[129074]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 96 op/s 2023-07-13T22:18:53.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:53 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:18:53.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:53 smithi026 ceph-mon[95647]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 96 op/s 2023-07-13T22:18:53.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:53 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:18:53.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:53 smithi121 ceph-mon[129187]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 96 op/s 2023-07-13T22:18:54.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:54 smithi121 ceph-mon[129187]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 328 MiB/s wr, 107 op/s 2023-07-13T22:18:55.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:54 smithi117 ceph-mon[129074]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 328 MiB/s wr, 107 op/s 2023-07-13T22:18:55.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:54 smithi026 ceph-mon[95647]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 328 MiB/s wr, 107 op/s 2023-07-13T22:18:57.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:56 smithi121 ceph-mon[129187]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 95 op/s 2023-07-13T22:18:57.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:56 smithi117 ceph-mon[129074]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 95 op/s 2023-07-13T22:18:57.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:56 smithi026 ceph-mon[95647]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 95 op/s 2023-07-13T22:18:59.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:18:58 smithi121 ceph-mon[129187]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 313 MiB/s wr, 105 op/s 2023-07-13T22:18:59.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:18:58 smithi117 ceph-mon[129074]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 313 MiB/s wr, 105 op/s 2023-07-13T22:18:59.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:18:58 smithi026 ceph-mon[95647]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 313 MiB/s wr, 105 op/s 2023-07-13T22:19:01.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:01 smithi117 ceph-mon[129074]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 MiB/s wr, 103 op/s 2023-07-13T22:19:01.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:01 smithi026 ceph-mon[95647]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 MiB/s wr, 103 op/s 2023-07-13T22:19:01.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:01 smithi121 ceph-mon[129187]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 320 MiB/s wr, 103 op/s 2023-07-13T22:19:03.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:03 smithi117 ceph-mon[129074]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 298 MiB/s wr, 98 op/s 2023-07-13T22:19:03.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:03 smithi026 ceph-mon[95647]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 298 MiB/s wr, 98 op/s 2023-07-13T22:19:03.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:03 smithi121 ceph-mon[129187]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 298 MiB/s wr, 98 op/s 2023-07-13T22:19:05.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:05 smithi117 ceph-mon[129074]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 339 MiB/s wr, 110 op/s 2023-07-13T22:19:05.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:05 smithi026 ceph-mon[95647]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 339 MiB/s wr, 110 op/s 2023-07-13T22:19:05.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:05 smithi121 ceph-mon[129187]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 339 MiB/s wr, 110 op/s 2023-07-13T22:19:07.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:07 smithi121 ceph-mon[129187]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 98 op/s 2023-07-13T22:19:07.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:07 smithi117 ceph-mon[129074]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 98 op/s 2023-07-13T22:19:07.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:07 smithi026 ceph-mon[95647]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 300 MiB/s wr, 98 op/s 2023-07-13T22:19:09.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:09 smithi117 ceph-mon[129074]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 311 MiB/s wr, 105 op/s 2023-07-13T22:19:09.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:09 smithi026 ceph-mon[95647]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 311 MiB/s wr, 105 op/s 2023-07-13T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:09 smithi121 ceph-mon[129187]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 311 MiB/s wr, 105 op/s 2023-07-13T22:19:11.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:11 smithi117 ceph-mon[129074]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 99 op/s 2023-07-13T22:19:11.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:11 smithi026 ceph-mon[95647]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 99 op/s 2023-07-13T22:19:11.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:11 smithi121 ceph-mon[129187]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 99 op/s 2023-07-13T22:19:12.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:19:13.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.299+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/2205366278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 msgr2=0x7f3a980712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.299+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/2205366278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a980712f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000b0a0 tx=0x7f3a9000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.321+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/2205366278 shutdown_connections 2023-07-13T22:19:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.321+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/2205366278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a9810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.321+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/2205366278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a980712f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.321+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/2205366278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 0x7f3a9810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.321+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/2205366278 >> 172.21.15.26:0/2205366278 conn(0x7f3a9806ca90 msgr2=0x7f3a9806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.331+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/2205366278 shutdown_connections 2023-07-13T22:19:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.331+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/2205366278 wait complete. 2023-07-13T22:19:13.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.333+0000 7f3a9c9ff700 1 Processor -- start 2023-07-13T22:19:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.333+0000 7f3a9c9ff700 1 -- start start 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.333+0000 7f3a9c9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9c9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 0x7f3a98115170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a981161b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9c9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98116780 con 0x7f3a98071830 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a981168f0 con 0x7f3a9810bd60 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9c9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98116ab0 con 0x7f3a98070ee0 2023-07-13T22:19:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a981161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a981161b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35714/0 (socket says 172.21.15.26:35714) 2023-07-13T22:19:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52262/0 (socket says 172.21.15.26:52262) 2023-07-13T22:19:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.334+0000 7f3a96d9d700 1 -- 172.21.15.26:0/903037120 learned_addr learned my addr 172.21.15.26:0/903037120 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.338+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 0x7f3a98115170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.339+0000 7f3a9659c700 1 -- 172.21.15.26:0/903037120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 msgr2=0x7f3a981161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.339+0000 7f3a9659c700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a981161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.339+0000 7f3a9659c700 1 -- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 msgr2=0x7f3a98115170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.339+0000 7f3a9659c700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 0x7f3a98115170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.339+0000 7f3a9659c700 1 -- 172.21.15.26:0/903037120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90009d00 con 0x7f3a98070ee0 2023-07-13T22:19:13.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.340+0000 7f3a9659c700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009fd0 tx=0x7f3a90009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.340+0000 7f3a877fe700 1 -- 172.21.15.26:0/903037120 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000e050 con 0x7f3a98070ee0 2023-07-13T22:19:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.340+0000 7f3a877fe700 1 -- 172.21.15.26:0/903037120 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a90005570 con 0x7f3a98070ee0 2023-07-13T22:19:13.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.340+0000 7f3a877fe700 1 -- 172.21.15.26:0/903037120 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9001b9f0 con 0x7f3a98070ee0 2023-07-13T22:19:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.340+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a98116cd0 con 0x7f3a98070ee0 2023-07-13T22:19:13.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.341+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a98117210 con 0x7f3a98070ee0 2023-07-13T22:19:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.349+0000 7f3ab0402b80 1 Processor -- start 2023-07-13T22:19:13.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.351+0000 7f3ab0402b80 1 -- start start 2023-07-13T22:19:13.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296988e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12959dcb0 con 0x55b129698e20 2023-07-13T22:19:13.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1295f0a40 con 0x55b1296984d0 2023-07-13T22:19:13.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1295e9ed0 con 0x55b129699c70 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.352+0000 7f3a95d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52270/0 (socket says 172.21.15.26:52270) 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 -- 172.21.15.26:0/1217708929 learned_addr learned my addr 172.21.15.26:0/1217708929 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 -- 172.21.15.26:0/1217708929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 msgr2=0x55b1296988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 -- 172.21.15.26:0/1217708929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 msgr2=0x55b1296996e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:13.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296996e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.353+0000 7f3a95d9b700 1 -- 172.21.15.26:0/1217708929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b129417de0 con 0x55b129699c70 2023-07-13T22:19:13.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.355+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0077f0 tx=0x7f3a8c007b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c2c7d1ae91900 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.359+0000 7f3a877fe700 1 -- 172.21.15.26:0/903037120 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3a900056e0 con 0x7f3a98070ee0 2023-07-13T22:19:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.361+0000 7f3a877fe700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 0x7f3a800a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.362+0000 7f3a877fe700 1 -- 172.21.15.26:0/903037120 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a900cda90 con 0x7f3a98070ee0 2023-07-13T22:19:13.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.364+0000 7f3a84ff9700 1 -- 172.21.15.26:0/1217708929 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0125f0 con 0x55b129699c70 2023-07-13T22:19:13.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.364+0000 7f3a84ff9700 1 -- 172.21.15.26:0/1217708929 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8c012c10 con 0x55b129699c70 2023-07-13T22:19:13.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.364+0000 7f3a84ff9700 1 -- 172.21.15.26:0/1217708929 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c013e10 con 0x55b129699c70 2023-07-13T22:19:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 -- 172.21.15.26:0/1217708929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 msgr2=0x55b12969a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0077f0 tx=0x7f3a8c007b00 comp rx=0 tx=0).stop 2023-07-13T22:19:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 0x7f3a800a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 -- 172.21.15.26:0/1217708929 shutdown_connections 2023-07-13T22:19:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b12969a0e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/1217708929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.366+0000 7f3ab0402b80 1 -- 172.21.15.26:0/1217708929 >> 172.21.15.26:0/1217708929 conn(0x55b12968dda0 msgr2=0x55b1296931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.367+0000 7f3ab0402b80 1 -- 172.21.15.26:0/1217708929 shutdown_connections 2023-07-13T22:19:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.369+0000 7f3ab0402b80 1 -- 172.21.15.26:0/1217708929 wait complete. 2023-07-13T22:19:13.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.370+0000 7f3ab0402b80 1 -- start start 2023-07-13T22:19:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.370+0000 7f3ab0402b80 1 Processor -- start 2023-07-13T22:19:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 0x7f3a800a3800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e40 tx=0x7f3a88009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1296984d0 0x55b1296a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b1296a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b129589ac0 con 0x55b1296984d0 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1295897c0 con 0x55b129698e20 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12946d920 con 0x55b129699c70 2023-07-13T22:19:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.371+0000 7f3ab0402b80 10 client.0 _ll_register_callbacks cb 0x55b129587760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.372+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b1296a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.372+0000 7f3a95d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35734/0 (socket says 172.21.15.26:35734) 2023-07-13T22:19:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 -- 172.21.15.26:0/2272766418 learned_addr learned my addr 172.21.15.26:0/2272766418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 -- 172.21.15.26:0/2272766418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 msgr2=0x55b1296a9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b1296a9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 -- 172.21.15.26:0/2272766418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1296984d0 msgr2=0x55b1296a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1296984d0 0x55b1296a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.373+0000 7f3a95d9b700 1 -- 172.21.15.26:0/2272766418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c00e040 con 0x55b129698e20 2023-07-13T22:19:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.375+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c013040 tx=0x7f3a8c013690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bd031c693c40db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.375+0000 7f3a7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.375+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c013d20 con 0x55b129698e20 2023-07-13T22:19:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.375+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a8c0125f0 con 0x55b129698e20 2023-07-13T22:19:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.376+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1295434f0 con 0x55b129698e20 2023-07-13T22:19:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.376+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a8c013e80 con 0x55b129698e20 2023-07-13T22:19:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.376+0000 7f3ab0402b80 10 client.26246 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.376+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b129400c60 con 0x55b129698e20 2023-07-13T22:19:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.377+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1295434f0 con 0x55b129698e20 2023-07-13T22:19:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a8c031b80 con 0x55b129698e20 2023-07-13T22:19:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 20 client.26246 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b1296aded0 0x55b1296b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 4 client.26246 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.378+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1295e2eb0 con 0x55b1296aded0 2023-07-13T22:19:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.379+0000 7f3a9659c700 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b1296aded0 0x55b1296b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.380+0000 7f3a9659c700 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b1296aded0 0x55b1296b02e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1443d3da948c98 server_cookie=982adc38eb173b1b in_seq=0 out_seq=0 2023-07-13T22:19:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.380+0000 7f3a7dffb700 10 client.26246 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.381+0000 7f3a7dffb700 1 -- 172.21.15.26:0/2272766418 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b1295e2eb0 con 0x55b1296aded0 2023-07-13T22:19:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.381+0000 7f3a7dffb700 10 client.26246 handle_command_reply: tid=0 2023-07-13T22:19:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.381+0000 7f3ab0402b80 1 client.26246 shutdown 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 msgr2=0x55b1296a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c013040 tx=0x7f3a8c013690 comp rx=0 tx=0).stop 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 shutdown_connections 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b1296984d0 0x55b1296a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b1296aded0 0x55b1296b02e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129699c70 0x55b1296a9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2272766418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129698e20 0x55b1296a2730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 >> 172.21.15.26:0/2272766418 conn(0x55b12968dda0 msgr2=0x55b1296928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.382+0000 7f3a7dffb700 0 client.26246 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:13.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.383+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 shutdown_connections 2023-07-13T22:19:13.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.383+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2272766418 wait complete. 2023-07-13T22:19:13.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.383+0000 7f3ab0402b80 20 client.26246 trim_cache size 0 max 16384 2023-07-13T22:19:13.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.424+0000 7f3ab0402b80 1 Processor -- start 2023-07-13T22:19:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.429+0000 7f3ab0402b80 1 -- start start 2023-07-13T22:19:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.429+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.429+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129698e20 0x55b1296a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.430+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129699c70 0x55b1296a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.430+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b129694330 con 0x55b129699c70 2023-07-13T22:19:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.430+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1296a16a0 con 0x55b1296984d0 2023-07-13T22:19:13.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.430+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b129694070 con 0x55b129698e20 2023-07-13T22:19:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35748/0 (socket says 172.21.15.26:35748) 2023-07-13T22:19:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 -- 172.21.15.26:0/2604600750 learned_addr learned my addr 172.21.15.26:0/2604600750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a95d9b700 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129698e20 0x55b1296a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 -- 172.21.15.26:0/2604600750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129698e20 msgr2=0x55b1296a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129698e20 0x55b1296a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 -- 172.21.15.26:0/2604600750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129699c70 msgr2=0x55b1296a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129699c70 0x55b1296a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.431+0000 7f3a9659c700 1 -- 172.21.15.26:0/2604600750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b12943e250 con 0x55b1296984d0 2023-07-13T22:19:13.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.432+0000 7f3a9659c700 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a900062a0 tx=0x7f3a9000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0bbb89125f243f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.433+0000 7f3a857fa700 1 -- 172.21.15.26:0/2604600750 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900a3320 con 0x55b1296984d0 2023-07-13T22:19:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.433+0000 7f3a857fa700 1 -- 172.21.15.26:0/2604600750 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a90005990 con 0x55b1296984d0 2023-07-13T22:19:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.433+0000 7f3a857fa700 1 -- 172.21.15.26:0/2604600750 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9009b1c0 con 0x55b1296984d0 2023-07-13T22:19:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.434+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2604600750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 msgr2=0x55b1296afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.434+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3a900062a0 tx=0x7f3a9000b040 comp rx=0 tx=0).stop 2023-07-13T22:19:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2604600750 shutdown_connections 2023-07-13T22:19:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129699c70 0x55b1296a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1296984d0 0x55b1296afee0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/2604600750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b129698e20 0x55b1296a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2604600750 >> 172.21.15.26:0/2604600750 conn(0x55b12968dda0 msgr2=0x55b1296a00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2604600750 shutdown_connections 2023-07-13T22:19:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.437+0000 7f3ab0402b80 1 -- 172.21.15.26:0/2604600750 wait complete. 2023-07-13T22:19:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.438+0000 7f3ab0402b80 1 -- start start 2023-07-13T22:19:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.438+0000 7f3ab0402b80 1 Processor -- start 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 0x55b1296a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 0x55b1296a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1296a90a0 con 0x55b129698e20 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1296a9210 con 0x55b129699c70 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1296a93d0 con 0x55b1296984d0 2023-07-13T22:19:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.439+0000 7f3ab0402b80 10 client.0 _ll_register_callbacks cb 0x7f3a8c007670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.440+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 0x55b1296a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.440+0000 7f3a9659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 0x55b1296a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52300/0 (socket says 172.21.15.26:52300) 2023-07-13T22:19:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.440+0000 7f3a9659c700 1 -- 172.21.15.26:0/675814575 learned_addr learned my addr 172.21.15.26:0/675814575 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.440+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 0x55b1296a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.441+0000 7f3a96d9d700 1 -- 172.21.15.26:0/675814575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 msgr2=0x55b1296a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.441+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 0x55b1296a5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.441+0000 7f3a96d9d700 1 -- 172.21.15.26:0/675814575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 msgr2=0x55b1296a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.441+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.441+0000 7f3a96d9d700 1 -- 172.21.15.26:0/675814575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a98117210 con 0x55b129699c70 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.442+0000 7f3a96d9d700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 0x55b1296a8b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009c20 tx=0x7f3a8801dc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d212f23e6b87288 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.442+0000 7f3a7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.442+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88023050 con 0x55b129699c70 2023-07-13T22:19:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.442+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3a8800f720 con 0x55b129699c70 2023-07-13T22:19:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.443+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a8c012c10 con 0x55b129699c70 2023-07-13T22:19:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.444+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a88017070 con 0x55b129699c70 2023-07-13T22:19:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.444+0000 7f3ab0402b80 10 client.26252 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.444+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b1294162b0 con 0x55b129699c70 2023-07-13T22:19:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.446+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a8c012c10 con 0x55b129699c70 2023-07-13T22:19:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.446+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a88029ae0 con 0x55b129699c70 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 10 client.26252 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 20 client.26252 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b129702ec0 0x55b1297052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 4 client.26252 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:19:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b1295a5050 con 0x55b129702ec0 2023-07-13T22:19:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.447+0000 7f3a9659c700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b129702ec0 0x55b1297052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.448+0000 7f3a9659c700 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b129702ec0 0x55b1297052d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9130e2fbfc4af151 server_cookie=da8755326bbae8be in_seq=0 out_seq=0 2023-07-13T22:19:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.448+0000 7f3a7dffb700 10 client.26252 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.448+0000 7f3a7dffb700 1 -- 172.21.15.26:0/675814575 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b1295a5050 con 0x55b129702ec0 2023-07-13T22:19:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.448+0000 7f3a7dffb700 10 client.26252 handle_command_reply: tid=0 2023-07-13T22:19:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.448+0000 7f3ab0402b80 1 client.26252 shutdown 2023-07-13T22:19:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 msgr2=0x55b1296a8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 0x55b1296a8b60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009c20 tx=0x7f3a8801dc80 comp rx=0 tx=0).stop 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 shutdown_connections 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b129702ec0 0x55b1297052d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b129699c70 0x55b1296a8b60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.449+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b1296984d0 0x55b1296a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.450+0000 7f3ab0402b80 1 --2- 172.21.15.26:0/675814575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b129698e20 0x55b1296a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.450+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 >> 172.21.15.26:0/675814575 conn(0x55b12968dda0 msgr2=0x55b1296a00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.450+0000 7f3a7dffb700 0 client.26252 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.450+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 shutdown_connections 2023-07-13T22:19:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.450+0000 7f3ab0402b80 1 -- 172.21.15.26:0/675814575 wait complete. 2023-07-13T22:19:13.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.451+0000 7f3ab0402b80 20 client.26252 trim_cache size 0 max 16384 2023-07-13T22:19:13.446 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:19:13.446 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:19:13.446 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:19:13.446 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:19:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.468+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 msgr2=0x7f3a800a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.468+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 0x7f3a800a3800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e40 tx=0x7f3a88009260 comp rx=0 tx=0).stop 2023-07-13T22:19:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.468+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 msgr2=0x7f3a98114c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.468+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009fd0 tx=0x7f3a90009450 comp rx=0 tx=0).stop 2023-07-13T22:19:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 shutdown_connections 2023-07-13T22:19:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a9810bd60 0x7f3a981161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a98070ee0 0x7f3a98114c30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3a800a1350 0x7f3a800a3800 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 --2- 172.21.15.26:0/903037120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a98071830 0x7f3a98115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 >> 172.21.15.26:0/903037120 conn(0x7f3a9806ca90 msgr2=0x7f3a9806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.470+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 shutdown_connections 2023-07-13T22:19:13.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.471+0000 7f3a9c9ff700 1 -- 172.21.15.26:0/903037120 wait complete. 2023-07-13T22:19:13.487 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:19:13.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:19:13.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:13 smithi117 ceph-mon[129074]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 94 op/s 2023-07-13T22:19:13.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:13 smithi026 ceph-mon[95647]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 94 op/s 2023-07-13T22:19:13.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:13 smithi121 ceph-mon[129187]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 94 op/s 2023-07-13T22:19:13.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.901+0000 7f3248b9d700 1 -- 172.21.15.26:0/1015857680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32440711d0 msgr2=0x7f32440715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.901+0000 7f3248b9d700 1 --2- 172.21.15.26:0/1015857680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32440711d0 0x7f32440715e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f323401a9a0 tx=0x7f323401acb0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 -- 172.21.15.26:0/1015857680 shutdown_connections 2023-07-13T22:19:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 --2- 172.21.15.26:0/1015857680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f324410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 --2- 172.21.15.26:0/1015857680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32440711d0 0x7f32440715e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 --2- 172.21.15.26:0/1015857680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3244071b20 0x7f324410b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 -- 172.21.15.26:0/1015857680 >> 172.21.15.26:0/1015857680 conn(0x7f324406c8e0 msgr2=0x7f324406ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 -- 172.21.15.26:0/1015857680 shutdown_connections 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.903+0000 7f3248b9d700 1 -- 172.21.15.26:0/1015857680 wait complete. 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 Processor -- start 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 -- start start 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32440711d0 0x7f324419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3244071b20 0x7f32441a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441a3740 con 0x7f32440711d0 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441a38b0 con 0x7f324410bba0 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.905+0000 7f3248b9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32441a3a70 con 0x7f3244071b20 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.906+0000 7f3242ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3244071b20 0x7f32441a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.907+0000 7f3243fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.907+0000 7f3243fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35792/0 (socket says 172.21.15.26:35792) 2023-07-13T22:19:13.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.907+0000 7f3243fff700 1 -- 172.21.15.26:0/3976197710 learned_addr learned my addr 172.21.15.26:0/3976197710 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.907+0000 7f32437fe700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32440711d0 0x7f324419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.908+0000 7f3243fff700 1 -- 172.21.15.26:0/3976197710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3244071b20 msgr2=0x7f32441a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.908+0000 7f3243fff700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3244071b20 0x7f32441a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.908+0000 7f3243fff700 1 -- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32440711d0 msgr2=0x7f324419e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.908+0000 7f3243fff700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32440711d0 0x7f324419e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.908+0000 7f3243fff700 1 -- 172.21.15.26:0/3976197710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323401a600 con 0x7f324410bba0 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3243fff700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f323c00ef30 tx=0x7f323c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:13.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c018070 con 0x7f324410bba0 2023-07-13T22:19:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f323c00f040 con 0x7f324410bba0 2023-07-13T22:19:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00cdb0 con 0x7f324410bba0 2023-07-13T22:19:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32441a3cf0 con 0x7f324410bba0 2023-07-13T22:19:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.909+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32441a41b0 con 0x7f324410bba0 2023-07-13T22:19:13.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.913+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32280052f0 con 0x7f324410bba0 2023-07-13T22:19:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.917+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f323c004ad0 con 0x7f324410bba0 2023-07-13T22:19:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.920+0000 7f3240ff9700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 0x7f322c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.921+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f323c0cc4d0 con 0x7f324410bba0 2023-07-13T22:19:13.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.922+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f323c095ce0 con 0x7f324410bba0 2023-07-13T22:19:13.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:13.928+0000 7f32437fe700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 0x7f322c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:13.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.003+0000 7f32437fe700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 0x7f322c0a3930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3234006010 tx=0x7f3234005de0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.536+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3228005160 con 0x7f324410bba0 2023-07-13T22:19:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.537+0000 7f3240ff9700 1 -- 172.21.15.26:0/3976197710 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f323c099300 con 0x7f324410bba0 2023-07-13T22:19:14.530 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:19:14.530 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:19:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 msgr2=0x7f322c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 0x7f322c0a3930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3234006010 tx=0x7f3234005de0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 msgr2=0x7f32441a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f323c00ef30 tx=0x7f323c00c3a0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 shutdown_connections 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f322c0a1480 0x7f322c0a3930 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32440711d0 0x7f324419e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f324410bba0 0x7f32441a2fa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 --2- 172.21.15.26:0/3976197710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3244071b20 0x7f32441a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 >> 172.21.15.26:0/3976197710 conn(0x7f324406c8e0 msgr2=0x7f324406da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.548+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 shutdown_connections 2023-07-13T22:19:14.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:14.549+0000 7f3248b9d700 1 -- 172.21.15.26:0/3976197710 wait complete. 2023-07-13T22:19:14.537 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:19:14.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:19:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.189+0000 7f2603dff700 1 -- 172.21.15.26:0/4268995198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc10bd60 msgr2=0x7f25fc10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.189+0000 7f2603dff700 1 --2- 172.21.15.26:0/4268995198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc10bd60 0x7f25fc10e1f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f25f80098f0 tx=0x7f25f8009c00 comp rx=0 tx=0).stop 2023-07-13T22:19:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.201+0000 7f2603dff700 1 -- 172.21.15.26:0/4268995198 shutdown_connections 2023-07-13T22:19:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.201+0000 7f2603dff700 1 --2- 172.21.15.26:0/4268995198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25fc070ee0 0x7f25fc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.201+0000 7f2603dff700 1 --2- 172.21.15.26:0/4268995198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc10bd60 0x7f25fc10e1f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.201+0000 7f2603dff700 1 --2- 172.21.15.26:0/4268995198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc071830 0x7f25fc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.201+0000 7f2603dff700 1 -- 172.21.15.26:0/4268995198 >> 172.21.15.26:0/4268995198 conn(0x7f25fc06ca90 msgr2=0x7f25fc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.203+0000 7f2603dff700 1 -- 172.21.15.26:0/4268995198 shutdown_connections 2023-07-13T22:19:15.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.203+0000 7f2603dff700 1 -- 172.21.15.26:0/4268995198 wait complete. 2023-07-13T22:19:15.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.205+0000 7f2603dff700 1 Processor -- start 2023-07-13T22:19:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 -- start start 2023-07-13T22:19:15.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc070ee0 0x7f25fc1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25fc10bd60 0x7f25fc1abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc1ac1a0 con 0x7f25fc10bd60 2023-07-13T22:19:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc10ec40 con 0x7f25fc070ee0 2023-07-13T22:19:15.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.206+0000 7f2603dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25fc10edd0 con 0x7f25fc071830 2023-07-13T22:19:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.207+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.207+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52342/0 (socket says 172.21.15.26:52342) 2023-07-13T22:19:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.207+0000 7f260139a700 1 -- 172.21.15.26:0/2172004436 learned_addr learned my addr 172.21.15.26:0/2172004436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.207+0000 7f2601b9b700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc070ee0 0x7f25fc1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.208+0000 7f260139a700 1 -- 172.21.15.26:0/2172004436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc070ee0 msgr2=0x7f25fc1a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.208+0000 7f260139a700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc070ee0 0x7f25fc1a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.208+0000 7f260139a700 1 -- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25fc10bd60 msgr2=0x7f25fc1abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.208+0000 7f260139a700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25fc10bd60 0x7f25fc1abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.208+0000 7f260139a700 1 -- 172.21.15.26:0/2172004436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8009520 con 0x7f25fc071830 2023-07-13T22:19:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.218+0000 7f260139a700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25f4007f10 tx=0x7f25f400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.227+0000 7f25f2ffd700 1 -- 172.21.15.26:0/2172004436 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4017070 con 0x7f25fc071830 2023-07-13T22:19:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.227+0000 7f25f2ffd700 1 -- 172.21.15.26:0/2172004436 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25f400f040 con 0x7f25fc071830 2023-07-13T22:19:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.227+0000 7f25f2ffd700 1 -- 172.21.15.26:0/2172004436 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4004680 con 0x7f25fc071830 2023-07-13T22:19:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.228+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25fc10f0b0 con 0x7f25fc071830 2023-07-13T22:19:15.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.229+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc10f540 con 0x7f25fc071830 2023-07-13T22:19:15.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.236+0000 7f26177dcb80 1 Processor -- start 2023-07-13T22:19:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.237+0000 7f25f2ffd700 1 -- 172.21.15.26:0/2172004436 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25f4003a00 con 0x7f25fc071830 2023-07-13T22:19:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.240+0000 7f25f2ffd700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 0x7f25e80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.240+0000 7f25f2ffd700 1 -- 172.21.15.26:0/2172004436 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25f40cb5f0 con 0x7f25fc071830 2023-07-13T22:19:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.256+0000 7f26177dcb80 1 -- start start 2023-07-13T22:19:15.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.256+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af22f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.256+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.256+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af230c70 0x5627af2310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f26177dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af134cb0 con 0x5627af230c70 2023-07-13T22:19:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f26177dcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af187a40 con 0x5627af22f4d0 2023-07-13T22:19:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f26177dcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af180ed0 con 0x5627af22fe20 2023-07-13T22:19:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52354/0 (socket says 172.21.15.26:52354) 2023-07-13T22:19:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.257+0000 7f260139a700 1 -- 172.21.15.26:0/1449087634 learned_addr learned my addr 172.21.15.26:0/1449087634 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:15.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 -- 172.21.15.26:0/1449087634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 msgr2=0x5627af22f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af22f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 -- 172.21.15.26:0/1449087634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af230c70 msgr2=0x5627af2310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af230c70 0x5627af2310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 -- 172.21.15.26:0/1449087634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627aefaede0 con 0x5627af22fe20 2023-07-13T22:19:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.258+0000 7f260139a700 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f80 tx=0x7f25f4004120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de215d2d486aee4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.266+0000 7f25e7fff700 1 -- 172.21.15.26:0/1449087634 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f409c9e0 con 0x5627af22fe20 2023-07-13T22:19:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.266+0000 7f25e7fff700 1 -- 172.21.15.26:0/1449087634 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25f40a0e00 con 0x5627af22fe20 2023-07-13T22:19:15.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.266+0000 7f25e7fff700 1 -- 172.21.15.26:0/1449087634 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4098e70 con 0x5627af22fe20 2023-07-13T22:19:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.268+0000 7f26177dcb80 1 -- 172.21.15.26:0/1449087634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 msgr2=0x5627af2306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.268+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000f80 tx=0x7f25f4004120 comp rx=0 tx=0).stop 2023-07-13T22:19:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f260239c700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 0x7f25e80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f26177dcb80 1 -- 172.21.15.26:0/1449087634 shutdown_connections 2023-07-13T22:19:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af230c70 0x5627af2310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af22f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1449087634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af2306e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.272+0000 7f26177dcb80 1 -- 172.21.15.26:0/1449087634 >> 172.21.15.26:0/1449087634 conn(0x5627af224da0 msgr2=0x5627af22a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.273+0000 7f26177dcb80 1 -- 172.21.15.26:0/1449087634 shutdown_connections 2023-07-13T22:19:15.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.273+0000 7f26177dcb80 1 -- 172.21.15.26:0/1449087634 wait complete. 2023-07-13T22:19:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.274+0000 7f26177dcb80 1 -- start start 2023-07-13T22:19:15.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.274+0000 7f26177dcb80 1 Processor -- start 2023-07-13T22:19:15.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af239740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af230c70 0x5627af240780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af120ac0 con 0x5627af22fe20 2023-07-13T22:19:15.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af1207c0 con 0x5627af22f4d0 2023-07-13T22:19:15.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af004920 con 0x5627af230c70 2023-07-13T22:19:15.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.275+0000 7f26177dcb80 10 client.0 _ll_register_callbacks cb 0x5627af11e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:15.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.276+0000 7f2601b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.276+0000 7f2601b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35836/0 (socket says 172.21.15.26:35836) 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.276+0000 7f2601b9b700 1 -- 172.21.15.26:0/45859866 learned_addr learned my addr 172.21.15.26:0/45859866 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f260139a700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af239740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f260239c700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af230c70 0x5627af240780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f2601b9b700 1 -- 172.21.15.26:0/45859866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af230c70 msgr2=0x5627af240780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f2601b9b700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af230c70 0x5627af240780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f2601b9b700 1 -- 172.21.15.26:0/45859866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 msgr2=0x5627af239740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f2601b9b700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af239740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.277+0000 7f2601b9b700 1 -- 172.21.15.26:0/45859866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25fc10f540 con 0x5627af22f4d0 2023-07-13T22:19:15.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.278+0000 7f260239c700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 0x7f25e80a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009230 tx=0x7f25f800bfb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.281+0000 7f2601b9b700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25ec011660 tx=0x7f25ec011a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69233b39eb7f80db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.281+0000 7f25e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.281+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec012980 con 0x5627af22f4d0 2023-07-13T22:19:15.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.281+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25ec012ae0 con 0x5627af22f4d0 2023-07-13T22:19:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.282+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627af0da4f0 con 0x5627af22f4d0 2023-07-13T22:19:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.282+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25ec012ca0 con 0x5627af22f4d0 2023-07-13T22:19:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.282+0000 7f26177dcb80 10 client.26267 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:15.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.282+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5627aef97c60 con 0x5627af22f4d0 2023-07-13T22:19:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627af0da4f0 con 0x5627af22f4d0 2023-07-13T22:19:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25ec01a3f0 con 0x5627af22f4d0 2023-07-13T22:19:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 10 client.26267 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:15.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 10 client.26267 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:19:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 20 client.26267 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af244ff0 0x5627af247400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 4 client.26267 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.283+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627af179eb0 con 0x5627af244ff0 2023-07-13T22:19:15.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.284+0000 7f260139a700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af244ff0 0x5627af247400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.288+0000 7f260139a700 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af244ff0 0x5627af247400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be33ebe28cba28e9 server_cookie=10be6e6f498f4d8f in_seq=0 out_seq=0 2023-07-13T22:19:15.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.289+0000 7f25e57fa700 10 client.26267 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:15.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.289+0000 7f25e57fa700 1 -- 172.21.15.26:0/45859866 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627af179eb0 con 0x5627af244ff0 2023-07-13T22:19:15.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.289+0000 7f25e57fa700 10 client.26267 handle_command_reply: tid=0 2023-07-13T22:19:15.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.289+0000 7f26177dcb80 1 client.26267 shutdown 2023-07-13T22:19:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 msgr2=0x5627af239200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25ec011660 tx=0x7f25ec011a20 comp rx=0 tx=0).stop 2023-07-13T22:19:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 shutdown_connections 2023-07-13T22:19:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af244ff0 0x5627af247400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af230c70 0x5627af240780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af22f4d0 0x5627af239200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 --2- 172.21.15.26:0/45859866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af239740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.290+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 >> 172.21.15.26:0/45859866 conn(0x5627af224da0 msgr2=0x5627af2298b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.291+0000 7f25e57fa700 0 client.26267 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:15.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.291+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 shutdown_connections 2023-07-13T22:19:15.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.291+0000 7f26177dcb80 1 -- 172.21.15.26:0/45859866 wait complete. 2023-07-13T22:19:15.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.292+0000 7f26177dcb80 20 client.26267 trim_cache size 0 max 16384 2023-07-13T22:19:15.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.335+0000 7f26177dcb80 1 Processor -- start 2023-07-13T22:19:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 -- start start 2023-07-13T22:19:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af240740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af233c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af238dd0 con 0x5627af22fe20 2023-07-13T22:19:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af2386b0 con 0x5627af230c70 2023-07-13T22:19:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.336+0000 7f26177dcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af2301f0 con 0x5627af22f4d0 2023-07-13T22:19:15.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.337+0000 7f2601b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.337+0000 7f2601b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52362/0 (socket says 172.21.15.26:52362) 2023-07-13T22:19:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.337+0000 7f2601b9b700 1 -- 172.21.15.26:0/1254441910 learned_addr learned my addr 172.21.15.26:0/1254441910 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.337+0000 7f260239c700 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af233c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f260139a700 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af240740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f2601b9b700 1 -- 172.21.15.26:0/1254441910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 msgr2=0x5627af233c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f2601b9b700 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af233c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f2601b9b700 1 -- 172.21.15.26:0/1254441910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 msgr2=0x5627af240740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f2601b9b700 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af240740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.340+0000 7f2601b9b700 1 -- 172.21.15.26:0/1254441910 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627af231310 con 0x5627af22f4d0 2023-07-13T22:19:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.363+0000 7f2601b9b700 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25ec001eb0 tx=0x7f25ec009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6588338e0bc24170 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.375+0000 7f25f0bff700 1 -- 172.21.15.26:0/1254441910 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec01a3f0 con 0x5627af22f4d0 2023-07-13T22:19:15.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.375+0000 7f25f0bff700 1 -- 172.21.15.26:0/1254441910 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec01a550 con 0x5627af22f4d0 2023-07-13T22:19:15.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.375+0000 7f25f0bff700 1 -- 172.21.15.26:0/1254441910 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec012c80 con 0x5627af22f4d0 2023-07-13T22:19:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.376+0000 7f26177dcb80 1 -- 172.21.15.26:0/1254441910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 msgr2=0x5627af247000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.376+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25ec001eb0 tx=0x7f25ec009930 comp rx=0 tx=0).stop 2023-07-13T22:19:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.381+0000 7f26177dcb80 1 -- 172.21.15.26:0/1254441910 shutdown_connections 2023-07-13T22:19:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.381+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af233c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.381+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22f4d0 0x5627af247000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.381+0000 7f26177dcb80 1 --2- 172.21.15.26:0/1254441910 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22fe20 0x5627af240740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.381+0000 7f26177dcb80 1 -- 172.21.15.26:0/1254441910 >> 172.21.15.26:0/1254441910 conn(0x5627af224da0 msgr2=0x5627af240fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.387+0000 7f26177dcb80 1 -- 172.21.15.26:0/1254441910 shutdown_connections 2023-07-13T22:19:15.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.387+0000 7f26177dcb80 1 -- 172.21.15.26:0/1254441910 wait complete. 2023-07-13T22:19:15.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.387+0000 7f26177dcb80 1 -- start start 2023-07-13T22:19:15.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 Processor -- start 2023-07-13T22:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22f4d0 0x5627af240050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af23e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af23ed40 con 0x5627af22f4d0 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af23eeb0 con 0x5627af230c70 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627af23f070 con 0x5627af22fe20 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.389+0000 7f26177dcb80 10 client.0 _ll_register_callbacks cb 0x7f25ec009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.390+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.390+0000 7f260239c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af23e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.391+0000 7f260139a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52366/0 (socket says 172.21.15.26:52366) 2023-07-13T22:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.391+0000 7f260139a700 1 -- 172.21.15.26:0/577338815 learned_addr learned my addr 172.21.15.26:0/577338815 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.399+0000 7f260139a700 1 -- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 msgr2=0x5627af23e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.399+0000 7f260139a700 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af23e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.399+0000 7f260139a700 1 -- 172.21.15.26:0/577338815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22f4d0 msgr2=0x5627af240050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.399+0000 7f260139a700 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22f4d0 0x5627af240050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.399+0000 7f260139a700 1 -- 172.21.15.26:0/577338815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627aef97c60 con 0x5627af22fe20 2023-07-13T22:19:15.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.405+0000 7f260139a700 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25f4007ee0 tx=0x7f25f400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84828d971c33861d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.410+0000 7f25e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.412+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f409ca60 con 0x5627af22fe20 2023-07-13T22:19:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.412+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25f40b6b40 con 0x5627af22fe20 2023-07-13T22:19:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.412+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25f40a0e00 con 0x5627af22fe20 2023-07-13T22:19:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.413+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25f40b6d00 con 0x5627af22fe20 2023-07-13T22:19:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.413+0000 7f26177dcb80 10 client.26242 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.413+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5627aefd5250 con 0x5627af22fe20 2023-07-13T22:19:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25f40a0e00 con 0x5627af22fe20 2023-07-13T22:19:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f25f40c6030 con 0x5627af22fe20 2023-07-13T22:19:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 10 client.26242 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 10 client.26242 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:19:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 20 client.26242 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af299ee0 0x5627af29c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 4 client.26242 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.414+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627af13c050 con 0x5627af299ee0 2023-07-13T22:19:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.416+0000 7f2601b9b700 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af299ee0 0x5627af29c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f2601b9b700 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af299ee0 0x5627af29c2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac1c23f9268d8eb5 server_cookie=ce01d06c7a3128aa in_seq=0 out_seq=0 2023-07-13T22:19:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f25e57fa700 10 client.26242 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f25e57fa700 1 -- 172.21.15.26:0/577338815 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627af13c050 con 0x5627af299ee0 2023-07-13T22:19:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f25e57fa700 10 client.26242 handle_command_reply: tid=0 2023-07-13T22:19:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f26177dcb80 1 client.26242 shutdown 2023-07-13T22:19:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 msgr2=0x5627af240590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.417+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f25f4007ee0 tx=0x7f25f400dc90 comp rx=0 tx=0).stop 2023-07-13T22:19:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 shutdown_connections 2023-07-13T22:19:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5627af299ee0 0x5627af29c2f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627af22f4d0 0x5627af240050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627af230c70 0x5627af23e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 --2- 172.21.15.26:0/577338815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627af22fe20 0x5627af240590 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 >> 172.21.15.26:0/577338815 conn(0x5627af224da0 msgr2=0x5627af240fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f25e57fa700 0 client.26242 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 shutdown_connections 2023-07-13T22:19:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.418+0000 7f26177dcb80 1 -- 172.21.15.26:0/577338815 wait complete. 2023-07-13T22:19:15.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.419+0000 7f26177dcb80 20 client.26242 trim_cache size 0 max 16384 2023-07-13T22:19:15.407 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 msgr2=0x7f25e80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 0x7f25e80a3800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009230 tx=0x7f25f800bfb0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 msgr2=0x7f25fc1a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25f4007f10 tx=0x7f25f400d3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 shutdown_connections 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f25e80a1350 0x7f25e80a3800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25fc10bd60 0x7f25fc1abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25fc070ee0 0x7f25fc1a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 --2- 172.21.15.26:0/2172004436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25fc071830 0x7f25fc1a4b60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.420+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 >> 172.21.15.26:0/2172004436 conn(0x7f25fc06ca90 msgr2=0x7f25fc06e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.421+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 shutdown_connections 2023-07-13T22:19:15.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:15.421+0000 7f2603dff700 1 -- 172.21.15.26:0/2172004436 wait complete. 2023-07-13T22:19:15.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:19:15.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:15 smithi117 ceph-mon[129074]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:19:15.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:15 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3976197710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:15.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:15 smithi026 ceph-mon[95647]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:19:15.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:15 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3976197710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:15.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:15 smithi121 ceph-mon[129187]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:19:15.741 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:15 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3976197710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:16.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.121+0000 7f993e1ff700 1 -- 172.21.15.26:0/2228069696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938070ee0 msgr2=0x7f99380712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.121+0000 7f993e1ff700 1 --2- 172.21.15.26:0/2228069696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938070ee0 0x7f99380712f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f992801a9a0 tx=0x7f992801acb0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.132+0000 7f993e1ff700 1 -- 172.21.15.26:0/2228069696 shutdown_connections 2023-07-13T22:19:16.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.132+0000 7f993e1ff700 1 --2- 172.21.15.26:0/2228069696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 0x7f993810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.132+0000 7f993e1ff700 1 --2- 172.21.15.26:0/2228069696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938070ee0 0x7f99380712f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.132+0000 7f993e1ff700 1 --2- 172.21.15.26:0/2228069696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938071830 0x7f993810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.132+0000 7f993e1ff700 1 -- 172.21.15.26:0/2228069696 >> 172.21.15.26:0/2228069696 conn(0x7f993806ca90 msgr2=0x7f993806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.141+0000 7f993e1ff700 1 -- 172.21.15.26:0/2228069696 shutdown_connections 2023-07-13T22:19:16.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.152+0000 7f993e1ff700 1 -- 172.21.15.26:0/2228069696 wait complete. 2023-07-13T22:19:16.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.153+0000 7f993e1ff700 1 Processor -- start 2023-07-13T22:19:16.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.156+0000 7f993e1ff700 1 -- start start 2023-07-13T22:19:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.157+0000 7f993e1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938070ee0 0x7f99381c07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.157+0000 7f993e1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.158+0000 7f993e1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 0x7f99381c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.158+0000 7f993e1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381c5b00 con 0x7f9938070ee0 2023-07-13T22:19:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.158+0000 7f993e1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381c5c70 con 0x7f993810bd60 2023-07-13T22:19:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.158+0000 7f993e1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99381c5e30 con 0x7f9938071830 2023-07-13T22:19:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.162+0000 7f9936ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.162+0000 7f9936ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52380/0 (socket says 172.21.15.26:52380) 2023-07-13T22:19:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.162+0000 7f9936ffd700 1 -- 172.21.15.26:0/4167396522 learned_addr learned my addr 172.21.15.26:0/4167396522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9937fff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 0x7f99381c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9936ffd700 1 -- 172.21.15.26:0/4167396522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 msgr2=0x7f99381c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9936ffd700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 0x7f99381c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9936ffd700 1 -- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938070ee0 msgr2=0x7f99381c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9936ffd700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938070ee0 0x7f99381c07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.169+0000 7f9936ffd700 1 -- 172.21.15.26:0/4167396522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992801a600 con 0x7f9938071830 2023-07-13T22:19:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.183+0000 7f9936ffd700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b500 tx=0x7f992c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.184+0000 7f9934ff9700 1 -- 172.21.15.26:0/4167396522 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c011830 con 0x7f9938071830 2023-07-13T22:19:16.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.184+0000 7f9934ff9700 1 -- 172.21.15.26:0/4167396522 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f992c011e50 con 0x7f9938071830 2023-07-13T22:19:16.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.184+0000 7f9934ff9700 1 -- 172.21.15.26:0/4167396522 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00f2e0 con 0x7f9938071830 2023-07-13T22:19:16.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.185+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381c6110 con 0x7f9938071830 2023-07-13T22:19:16.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.186+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99381c65a0 con 0x7f9938071830 2023-07-13T22:19:16.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.193+0000 7f9951ad0b80 1 Processor -- start 2023-07-13T22:19:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 -- start start 2023-07-13T22:19:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9621e20 0x55cbb96226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9622c70 0x55cbb96230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9526cb0 con 0x55cbb9622c70 2023-07-13T22:19:16.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9579a40 con 0x55cbb9621e20 2023-07-13T22:19:16.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.194+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9572ed0 con 0x55cbb96214d0 2023-07-13T22:19:16.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52386/0 (socket says 172.21.15.26:52386) 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 -- 172.21.15.26:0/2247548503 learned_addr learned my addr 172.21.15.26:0/2247548503 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 -- 172.21.15.26:0/2247548503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9621e20 msgr2=0x55cbb96226e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9621e20 0x55cbb96226e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 -- 172.21.15.26:0/2247548503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9622c70 msgr2=0x55cbb96230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9622c70 0x55cbb96230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.195+0000 7f9937fff700 1 -- 172.21.15.26:0/2247548503 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb93a0de0 con 0x55cbb96214d0 2023-07-13T22:19:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.196+0000 7f9937fff700 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9930008550 tx=0x7f9930008860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128a185786602476 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.196+0000 7f991dffb700 1 -- 172.21.15.26:0/2247548503 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930003c00 con 0x55cbb96214d0 2023-07-13T22:19:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.196+0000 7f991dffb700 1 -- 172.21.15.26:0/2247548503 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9930010040 con 0x55cbb96214d0 2023-07-13T22:19:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.196+0000 7f991dffb700 1 -- 172.21.15.26:0/2247548503 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930012740 con 0x55cbb96214d0 2023-07-13T22:19:16.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 -- 172.21.15.26:0/2247548503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 msgr2=0x55cbb96218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9930008550 tx=0x7f9930008860 comp rx=0 tx=0).stop 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 -- 172.21.15.26:0/2247548503 shutdown_connections 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9622c70 0x55cbb96230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb96218e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/2247548503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9621e20 0x55cbb96226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.197+0000 7f9951ad0b80 1 -- 172.21.15.26:0/2247548503 >> 172.21.15.26:0/2247548503 conn(0x55cbb9616da0 msgr2=0x55cbb961c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.200+0000 7f9934ff9700 1 -- 172.21.15.26:0/4167396522 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f992c010580 con 0x7f9938071830 2023-07-13T22:19:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.200+0000 7f9951ad0b80 1 -- 172.21.15.26:0/2247548503 shutdown_connections 2023-07-13T22:19:16.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.200+0000 7f9951ad0b80 1 -- 172.21.15.26:0/2247548503 wait complete. 2023-07-13T22:19:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.201+0000 7f9951ad0b80 1 -- start start 2023-07-13T22:19:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.202+0000 7f9951ad0b80 1 Processor -- start 2023-07-13T22:19:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9632c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9512ac0 con 0x55cbb9621e20 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb95127c0 con 0x55cbb9630820 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.222+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb93f6920 con 0x55cbb96214d0 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.223+0000 7f9951ad0b80 10 client.0 _ll_register_callbacks cb 0x55cbb9510760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.223+0000 7f9934ff9700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 0x7f99200a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.224+0000 7f9934ff9700 1 -- 172.21.15.26:0/4167396522 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f992c0ce490 con 0x7f9938071830 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.224+0000 7f9937fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.225+0000 7f9937fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52388/0 (socket says 172.21.15.26:52388) 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.225+0000 7f9937fff700 1 -- 172.21.15.26:0/1495857880 learned_addr learned my addr 172.21.15.26:0/1495857880 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f99377fe700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 -- 172.21.15.26:0/1495857880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 msgr2=0x55cbb9632c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9632c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 -- 172.21.15.26:0/1495857880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 msgr2=0x55cbb962b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 -- 172.21.15.26:0/1495857880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930007d90 con 0x55cbb96214d0 2023-07-13T22:19:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 0x7f99200a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.226+0000 7f9937fff700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9930007890 tx=0x7f9930012d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232d342c4cc44019 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.248+0000 7f9916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:16.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.248+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300076f0 con 0x55cbb96214d0 2023-07-13T22:19:16.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.248+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9930014600 con 0x55cbb96214d0 2023-07-13T22:19:16.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.248+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbb94cc4f0 con 0x55cbb96214d0 2023-07-13T22:19:16.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.259+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9930030070 con 0x55cbb96214d0 2023-07-13T22:19:16.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.259+0000 7f9951ad0b80 10 client.26254 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:16.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.259+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cbb9389c60 con 0x55cbb96214d0 2023-07-13T22:19:16.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cbb94cc4f0 con 0x55cbb96214d0 2023-07-13T22:19:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99300353f0 con 0x55cbb96214d0 2023-07-13T22:19:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 10 client.26254 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:19:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 20 client.26254 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb9637360 0x55cbb9639770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 4 client.26254 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.260+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbb956beb0 con 0x55cbb9637360 2023-07-13T22:19:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.261+0000 7f99377fe700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb9637360 0x55cbb9639770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.271+0000 7f9937fff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 0x7f99200a3710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9930007400 tx=0x7f9930029000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.290+0000 7f99377fe700 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb9637360 0x55cbb9639770 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93b1e14ab7b6e3b9 server_cookie=43081c1886bcc62 in_seq=0 out_seq=0 2023-07-13T22:19:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.291+0000 7f9916ffd700 10 client.26254 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:16.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.302+0000 7f9916ffd700 1 -- 172.21.15.26:0/1495857880 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cbb956beb0 con 0x55cbb9637360 2023-07-13T22:19:16.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.302+0000 7f9916ffd700 10 client.26254 handle_command_reply: tid=0 2023-07-13T22:19:16.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.302+0000 7f9951ad0b80 1 client.26254 shutdown 2023-07-13T22:19:16.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 msgr2=0x55cbb962b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9930007890 tx=0x7f9930012d40 comp rx=0 tx=0).stop 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 shutdown_connections 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb9637360 0x55cbb9639770 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9632c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb962b2b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/1495857880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.318+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 >> 172.21.15.26:0/1495857880 conn(0x55cbb9616da0 msgr2=0x55cbb961b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.319+0000 7f9916ffd700 0 client.26254 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:16.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.319+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 shutdown_connections 2023-07-13T22:19:16.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.319+0000 7f9951ad0b80 1 -- 172.21.15.26:0/1495857880 wait complete. 2023-07-13T22:19:16.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.320+0000 7f9951ad0b80 20 client.26254 trim_cache size 0 max 16384 2023-07-13T22:19:16.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.367+0000 7f9951ad0b80 1 Processor -- start 2023-07-13T22:19:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.367+0000 7f9951ad0b80 1 -- start start 2023-07-13T22:19:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9623040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9646ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb962c430 con 0x55cbb9621e20 2023-07-13T22:19:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9647530 con 0x55cbb9630820 2023-07-13T22:19:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.368+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9639740 con 0x55cbb96214d0 2023-07-13T22:19:16.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.369+0000 7f9937fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9646ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.369+0000 7f9937fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9646ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35900/0 (socket says 172.21.15.26:35900) 2023-07-13T22:19:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.369+0000 7f9937fff700 1 -- 172.21.15.26:0/3067432423 learned_addr learned my addr 172.21.15.26:0/3067432423 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.369+0000 7f99377fe700 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9623040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.370+0000 7f9936ffd700 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.371+0000 7f99377fe700 1 -- 172.21.15.26:0/3067432423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 msgr2=0x55cbb9646ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.371+0000 7f99377fe700 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9646ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.371+0000 7f99377fe700 1 -- 172.21.15.26:0/3067432423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 msgr2=0x55cbb962a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.371+0000 7f99377fe700 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.371+0000 7f99377fe700 1 -- 172.21.15.26:0/3067432423 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbb93c7250 con 0x55cbb96214d0 2023-07-13T22:19:16.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.372+0000 7f99377fe700 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9623040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9928003820 tx=0x7f9928005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db02b05eaf0b099a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.379+0000 7f991e7fc700 1 -- 172.21.15.26:0/3067432423 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928015070 con 0x55cbb96214d0 2023-07-13T22:19:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.379+0000 7f991e7fc700 1 -- 172.21.15.26:0/3067432423 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f992801cd10 con 0x55cbb96214d0 2023-07-13T22:19:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.379+0000 7f991e7fc700 1 -- 172.21.15.26:0/3067432423 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928004660 con 0x55cbb96214d0 2023-07-13T22:19:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.385+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3067432423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 msgr2=0x55cbb9623040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.385+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9623040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9928003820 tx=0x7f9928005a90 comp rx=0 tx=0).stop 2023-07-13T22:19:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.394+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3067432423 shutdown_connections 2023-07-13T22:19:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.394+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb9646ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.394+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9623040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.394+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3067432423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb962a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.394+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3067432423 >> 172.21.15.26:0/3067432423 conn(0x55cbb9616da0 msgr2=0x55cbb963e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.406+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3067432423 shutdown_connections 2023-07-13T22:19:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.418+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3067432423 wait complete. 2023-07-13T22:19:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.419+0000 7f9951ad0b80 1 -- start start 2023-07-13T22:19:16.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.419+0000 7f9951ad0b80 1 Processor -- start 2023-07-13T22:19:16.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9644d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb96452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9646820 con 0x55cbb9621e20 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9646990 con 0x55cbb9630820 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbb9646b50 con 0x55cbb96214d0 2023-07-13T22:19:16.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.429+0000 7f9951ad0b80 10 client.0 _ll_register_callbacks cb 0x7f99300083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.436+0000 7f9937fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.436+0000 7f99377fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9644d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.436+0000 7f9937fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35906/0 (socket says 172.21.15.26:35906) 2023-07-13T22:19:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.436+0000 7f9937fff700 1 -- 172.21.15.26:0/3680640829 learned_addr learned my addr 172.21.15.26:0/3680640829 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:16.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.442+0000 7f9936ffd700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb96452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 -- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 msgr2=0x55cbb9644d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9644d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 -- 172.21.15.26:0/3680640829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 msgr2=0x55cbb96452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb96452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 -- 172.21.15.26:0/3680640829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938117640 con 0x55cbb9630820 2023-07-13T22:19:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9937fff700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9930006130 tx=0x7f9930023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be051e9d8c2883c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:16.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.447+0000 7f9916ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.458+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930030040 con 0x55cbb9630820 2023-07-13T22:19:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.458+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f993003a020 con 0x55cbb9630820 2023-07-13T22:19:16.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.458+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9930010040 con 0x55cbb9630820 2023-07-13T22:19:16.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.462+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99300048a0 con 0x55cbb9630820 2023-07-13T22:19:16.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.462+0000 7f9951ad0b80 10 client.26282 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:16.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.462+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cbb939f2b0 con 0x55cbb9630820 2023-07-13T22:19:16.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.470+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9930010040 con 0x55cbb9630820 2023-07-13T22:19:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.470+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9930028bc0 con 0x55cbb9630820 2023-07-13T22:19:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 10 client.26282 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:19:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 20 client.26282 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:16.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb968c690 0x55cbb968eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 4 client.26282 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:16.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.472+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbb952e050 con 0x55cbb968c690 2023-07-13T22:19:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.473+0000 7f99377fe700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb968c690 0x55cbb968eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.474+0000 7f99377fe700 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb968c690 0x55cbb968eaa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e69b47261e4ec1aa server_cookie=2924b9c42fadac28 in_seq=0 out_seq=0 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9916ffd700 10 client.26282 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9916ffd700 1 -- 172.21.15.26:0/3680640829 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbb952e050 con 0x55cbb968c690 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9916ffd700 10 client.26282 handle_command_reply: tid=0 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9951ad0b80 1 client.26282 shutdown 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 msgr2=0x55cbb96462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.484+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9930006130 tx=0x7f9930023000 comp rx=0 tx=0).stop 2023-07-13T22:19:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 shutdown_connections 2023-07-13T22:19:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cbb968c690 0x55cbb968eaa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbb9630820 0x55cbb96462e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cbb96214d0 0x55cbb9644d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 --2- 172.21.15.26:0/3680640829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cbb9621e20 0x55cbb96452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 >> 172.21.15.26:0/3680640829 conn(0x55cbb9616da0 msgr2=0x55cbb96251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.495+0000 7f9916ffd700 0 client.26282 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:16.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.513+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 shutdown_connections 2023-07-13T22:19:16.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.526+0000 7f9951ad0b80 1 -- 172.21.15.26:0/3680640829 wait complete. 2023-07-13T22:19:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.552+0000 7f9951ad0b80 20 client.26282 trim_cache size 0 max 16384 2023-07-13T22:19:16.542 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.573+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 msgr2=0x7f99200a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.573+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 0x7f99200a3710 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9930007400 tx=0x7f9930029000 comp rx=0 tx=0).stop 2023-07-13T22:19:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.573+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 msgr2=0x7f99381c2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.573+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b500 tx=0x7f992c00b8c0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 shutdown_connections 2023-07-13T22:19:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f99200a1260 0x7f99200a3710 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938070ee0 0x7f99381c07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f993810bd60 0x7f99381c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 --2- 172.21.15.26:0/4167396522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9938071830 0x7f99381c2cf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:16.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.596+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 >> 172.21.15.26:0/4167396522 conn(0x7f993806ca90 msgr2=0x7f993806dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:16.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.598+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 shutdown_connections 2023-07-13T22:19:16.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:16.598+0000 7f993e1ff700 1 -- 172.21.15.26:0/4167396522 wait complete. 2023-07-13T22:19:16.617 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:19:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.312+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2169400915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc071830 msgr2=0x7fb9cc10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.312+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2169400915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc071830 0x7fb9cc10b7d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009990 tx=0x7fb9c8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2169400915 shutdown_connections 2023-07-13T22:19:17.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2169400915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9cc070ee0 0x7fb9cc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2169400915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2169400915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc071830 0x7fb9cc10b7d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2169400915 >> 172.21.15.26:0/2169400915 conn(0x7fb9cc06ca90 msgr2=0x7fb9cc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2169400915 shutdown_connections 2023-07-13T22:19:17.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.313+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2169400915 wait complete. 2023-07-13T22:19:17.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.315+0000 7fb9d45ff700 1 Processor -- start 2023-07-13T22:19:17.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 -- start start 2023-07-13T22:19:17.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc070ee0 0x7fb9cc1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9cc071830 0x7fb9cc1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1ac1f0 con 0x7fb9cc071830 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc10ec40 con 0x7fb9cc070ee0 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.316+0000 7fb9d45ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc10edd0 con 0x7fb9cc10bd60 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.317+0000 7fb9d2b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.317+0000 7fb9d2b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52410/0 (socket says 172.21.15.26:52410) 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.317+0000 7fb9d2b9c700 1 -- 172.21.15.26:0/2179041047 learned_addr learned my addr 172.21.15.26:0/2179041047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:17.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.317+0000 7fb9d239b700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc070ee0 0x7fb9cc1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.317+0000 7fb9d2b9c700 1 -- 172.21.15.26:0/2179041047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc070ee0 msgr2=0x7fb9cc1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d2b9c700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc070ee0 0x7fb9cc1a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d2b9c700 1 -- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9cc071830 msgr2=0x7fb9cc1a4bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d2b9c700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9cc071830 0x7fb9cc1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d2b9c700 1 -- 172.21.15.26:0/2179041047 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80095f0 con 0x7fb9cc10bd60 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d2b9c700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400ef30 tx=0x7fb9c400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9c37fe700 1 -- 172.21.15.26:0/2179041047 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4018070 con 0x7fb9cc10bd60 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.318+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc10f050 con 0x7fb9cc10bd60 2023-07-13T22:19:17.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.319+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc10f560 con 0x7fb9cc10bd60 2023-07-13T22:19:17.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.319+0000 7fb9c37fe700 1 -- 172.21.15.26:0/2179041047 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9c400f040 con 0x7fb9cc10bd60 2023-07-13T22:19:17.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.319+0000 7fb9c37fe700 1 -- 172.21.15.26:0/2179041047 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4007800 con 0x7fb9cc10bd60 2023-07-13T22:19:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.327+0000 7fb9e7f4cb80 1 Processor -- start 2023-07-13T22:19:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.327+0000 7fb9c37fe700 1 -- 172.21.15.26:0/2179041047 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9c4004ad0 con 0x7fb9cc10bd60 2023-07-13T22:19:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.328+0000 7fb9e7f4cb80 1 -- start start 2023-07-13T22:19:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bd78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bd90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2adccb0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2b2fa40 con 0x55eeb2bd74d0 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.329+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2b28ed0 con 0x55eeb2bd8c70 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.331+0000 7fb9d239b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.331+0000 7fb9d239b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43138/0 (socket says 172.21.15.26:43138) 2023-07-13T22:19:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.331+0000 7fb9d239b700 1 -- 172.21.15.26:0/1856116506 learned_addr learned my addr 172.21.15.26:0/1856116506 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.331+0000 7fb9c37fe700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 0x7fb9b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.331+0000 7fb9c37fe700 1 -- 172.21.15.26:0/2179041047 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9c40cc570 con 0x7fb9cc10bd60 2023-07-13T22:19:17.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.333+0000 7fb9d2b9c700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 0x7fb9b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.334+0000 7fb9d239b700 1 -- 172.21.15.26:0/1856116506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 msgr2=0x55eeb2bd90e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.334+0000 7fb9d239b700 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bd90e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.334+0000 7fb9d239b700 1 -- 172.21.15.26:0/1856116506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 msgr2=0x55eeb2bd78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.334+0000 7fb9d239b700 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bd78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.334+0000 7fb9d239b700 1 -- 172.21.15.26:0/1856116506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb2956de0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.337+0000 7fb9d239b700 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009d80 tx=0x7fb9bc033730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe7a3ea39d9e3c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.342+0000 7fb9d2b9c700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 0x7fb9b40a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40090a0 tx=0x7fb9c40955b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.343+0000 7fb9c0bfe700 1 -- 172.21.15.26:0/1856116506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc004230 con 0x55eeb2bd7e20 2023-07-13T22:19:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.344+0000 7fb9c0bfe700 1 -- 172.21.15.26:0/1856116506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc004390 con 0x55eeb2bd7e20 2023-07-13T22:19:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.344+0000 7fb9c0bfe700 1 -- 172.21.15.26:0/1856116506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc03b8c0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.348+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/1856116506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 msgr2=0x55eeb2bd86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.348+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009d80 tx=0x7fb9bc033730 comp rx=0 tx=0).stop 2023-07-13T22:19:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.350+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/1856116506 shutdown_connections 2023-07-13T22:19:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.350+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bd90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.351+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bd78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.351+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/1856116506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2bd86e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.351+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/1856116506 >> 172.21.15.26:0/1856116506 conn(0x55eeb2bccda0 msgr2=0x55eeb2bd21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.359+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/1856116506 shutdown_connections 2023-07-13T22:19:17.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.359+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/1856116506 wait complete. 2023-07-13T22:19:17.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.360+0000 7fb9e7f4cb80 1 -- start start 2023-07-13T22:19:17.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.361+0000 7fb9e7f4cb80 1 Processor -- start 2023-07-13T22:19:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2ac8ac0 con 0x55eeb2bd8c70 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2ac87c0 con 0x55eeb2bd74d0 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb29ac920 con 0x55eeb2bd7e20 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.362+0000 7fb9e7f4cb80 10 client.0 _ll_register_callbacks cb 0x55eeb2ac6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.363+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.364+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.366+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35954/0 (socket says 172.21.15.26:35954) 2023-07-13T22:19:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.366+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/603085052 learned_addr learned my addr 172.21.15.26:0/603085052 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.367+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/603085052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd7e20 msgr2=0x55eeb2be1770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.367+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.367+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/603085052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd8c70 msgr2=0x55eeb2be87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.367+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.367+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/603085052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc0099e0 con 0x55eeb2bd74d0 2023-07-13T22:19:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.369+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8015000 tx=0x7fb9c8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b71cb598a2fe38 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.369+0000 7fb9b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.370+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8005220 con 0x55eeb2bd74d0 2023-07-13T22:19:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.370+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9c8005380 con 0x55eeb2bd74d0 2023-07-13T22:19:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.370+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eeb2a824f0 con 0x55eeb2bd74d0 2023-07-13T22:19:17.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.374+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c802d030 con 0x55eeb2bd74d0 2023-07-13T22:19:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.385+0000 7fb9e7f4cb80 10 client.26291 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.385+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eeb293fc60 con 0x55eeb2bd74d0 2023-07-13T22:19:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eeb2a824f0 con 0x55eeb2bd74d0 2023-07-13T22:19:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9c8004740 con 0x55eeb2bd74d0 2023-07-13T22:19:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 10 client.26291 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 10 client.26291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:19:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 20 client.26291 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2bed040 0x55eeb2bef450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 4 client.26291 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.405+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeb2b21eb0 con 0x55eeb2bed040 2023-07-13T22:19:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.406+0000 7fb9d239b700 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2bed040 0x55eeb2bef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.407+0000 7fb9d239b700 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2bed040 0x55eeb2bef450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbf39ead9ea20858 server_cookie=79eff8758b83b7c9 in_seq=0 out_seq=0 2023-07-13T22:19:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.407+0000 7fb9b1ffb700 10 client.26291 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:17.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.409+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/603085052 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eeb2b21eb0 con 0x55eeb2bed040 2023-07-13T22:19:17.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.410+0000 7fb9b1ffb700 10 client.26291 handle_command_reply: tid=0 2023-07-13T22:19:17.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.410+0000 7fb9e7f4cb80 1 client.26291 shutdown 2023-07-13T22:19:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 msgr2=0x55eeb2be1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8015000 tx=0x7fb9c8005fb0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 shutdown_connections 2023-07-13T22:19:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2bed040 0x55eeb2bef450 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be1230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/603085052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.411+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 >> 172.21.15.26:0/603085052 conn(0x55eeb2bccda0 msgr2=0x55eeb2bd18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.412+0000 7fb9b1ffb700 0 client.26291 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.412+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 shutdown_connections 2023-07-13T22:19:17.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.413+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/603085052 wait complete. 2023-07-13T22:19:17.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.413+0000 7fb9e7f4cb80 20 client.26291 trim_cache size 0 max 16384 2023-07-13T22:19:17.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.444+0000 7fb9e7f4cb80 1 Processor -- start 2023-07-13T22:19:17.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 -- start start 2023-07-13T22:19:17.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bdfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2bd7c60 con 0x55eeb2bd7e20 2023-07-13T22:19:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2bcc970 con 0x55eeb2bd74d0 2023-07-13T22:19:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.445+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2becd40 con 0x55eeb2bd8c70 2023-07-13T22:19:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35964/0 (socket says 172.21.15.26:35964) 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/609261048 learned_addr learned my addr 172.21.15.26:0/609261048 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bdfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/609261048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 msgr2=0x55eeb2bdfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bdfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/609261048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 msgr2=0x55eeb2be8770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be8770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.446+0000 7fb9d1b9a700 1 -- 172.21.15.26:0/609261048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeb297d250 con 0x55eeb2bd74d0 2023-07-13T22:19:17.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.447+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8004eb0 tx=0x7fb9c8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3d026d5de0e3607 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.470+0000 7fb9c13ff700 1 -- 172.21.15.26:0/609261048 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c802d040 con 0x55eeb2bd74d0 2023-07-13T22:19:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.470+0000 7fb9c13ff700 1 -- 172.21.15.26:0/609261048 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9c80057c0 con 0x55eeb2bd74d0 2023-07-13T22:19:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.470+0000 7fb9c13ff700 1 -- 172.21.15.26:0/609261048 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8004920 con 0x55eeb2bd74d0 2023-07-13T22:19:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.474+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/609261048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 msgr2=0x55eeb2bdf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.474+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8004eb0 tx=0x7fb9c8009310 comp rx=0 tx=0).stop 2023-07-13T22:19:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.475+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/609261048 shutdown_connections 2023-07-13T22:19:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.475+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2bdfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.475+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd74d0 0x55eeb2bdf040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.475+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/609261048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.475+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/609261048 >> 172.21.15.26:0/609261048 conn(0x55eeb2bccda0 msgr2=0x55eeb2bf3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.495+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/609261048 shutdown_connections 2023-07-13T22:19:17.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.496+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/609261048 wait complete. 2023-07-13T22:19:17.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.498+0000 7fb9e7f4cb80 1 -- start start 2023-07-13T22:19:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.499+0000 7fb9e7f4cb80 1 Processor -- start 2023-07-13T22:19:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2be8030 con 0x55eeb2bd74d0 2023-07-13T22:19:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2be81a0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeb2be8360 con 0x55eeb2bd8c70 2023-07-13T22:19:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.503+0000 7fb9e7f4cb80 10 client.0 _ll_register_callbacks cb 0x7fb9c8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.505+0000 7fb9d239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.505+0000 7fb9d1b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.505+0000 7fb9d239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35974/0 (socket says 172.21.15.26:35974) 2023-07-13T22:19:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.505+0000 7fb9d239b700 1 -- 172.21.15.26:0/2069557837 learned_addr learned my addr 172.21.15.26:0/2069557837 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 -- 172.21.15.26:0/2069557837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 msgr2=0x55eeb2be7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 -- 172.21.15.26:0/2069557837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd74d0 msgr2=0x55eeb2be45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 -- 172.21.15.26:0/2069557837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeb293fc60 con 0x55eeb2bd7e20 2023-07-13T22:19:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9d239b700 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc036040 tx=0x7fb9bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64f3328ba1693a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.506+0000 7fb9b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.507+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc050070 con 0x55eeb2bd7e20 2023-07-13T22:19:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.507+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9bc04b1d0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.507+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9bc004390 con 0x55eeb2bd7e20 2023-07-13T22:19:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.525+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9bc04b3b0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.525+0000 7fb9e7f4cb80 10 client.26297 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:17.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.525+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eeb29552b0 con 0x55eeb2bd7e20 2023-07-13T22:19:17.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.528+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc004390 con 0x55eeb2bd7e20 2023-07-13T22:19:17.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.529+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9bc053030 con 0x55eeb2bd7e20 2023-07-13T22:19:17.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 10 client.26297 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:17.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 10 client.26297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:19:17.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 20 client.26297 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:17.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2c41ec0 0x55eeb2c442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:17.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 4 client.26297 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:17.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.530+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eeb2ae4050 con 0x55eeb2c41ec0 2023-07-13T22:19:17.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.531+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2c41ec0 0x55eeb2c442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:17.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.532+0000 7fb9d1b9a700 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2c41ec0 0x55eeb2c442d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c83c17fba1a7ff14 server_cookie=d14531ca667e6b2f in_seq=0 out_seq=0 2023-07-13T22:19:17.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.532+0000 7fb9b1ffb700 10 client.26297 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:17.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.532+0000 7fb9b1ffb700 1 -- 172.21.15.26:0/2069557837 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eeb2ae4050 con 0x55eeb2c41ec0 2023-07-13T22:19:17.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.532+0000 7fb9b1ffb700 10 client.26297 handle_command_reply: tid=0 2023-07-13T22:19:17.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.532+0000 7fb9e7f4cb80 1 client.26297 shutdown 2023-07-13T22:19:17.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.547+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 msgr2=0x55eeb2be4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.547+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc036040 tx=0x7fb9bc009780 comp rx=0 tx=0).stop 2023-07-13T22:19:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 shutdown_connections 2023-07-13T22:19:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eeb2bd74d0 0x55eeb2be45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55eeb2c41ec0 0x55eeb2c442d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eeb2bd8c70 0x55eeb2be7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 --2- 172.21.15.26:0/2069557837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eeb2bd7e20 0x55eeb2be4b20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.550+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 >> 172.21.15.26:0/2069557837 conn(0x55eeb2bccda0 msgr2=0x55eeb2bdf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.551+0000 7fb9b1ffb700 0 client.26297 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.559+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 shutdown_connections 2023-07-13T22:19:17.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.575+0000 7fb9e7f4cb80 1 -- 172.21.15.26:0/2069557837 wait complete. 2023-07-13T22:19:17.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.576+0000 7fb9e7f4cb80 20 client.26297 trim_cache size 0 max 16384 2023-07-13T22:19:17.563 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:17.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.585+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 msgr2=0x7fb9b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.585+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 0x7fb9b40a3600 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40090a0 tx=0x7fb9c40955b0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 msgr2=0x7fb9cc1abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400ef30 tx=0x7fb9c400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 shutdown_connections 2023-07-13T22:19:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9cc10bd60 0x7fb9cc1abc20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9cc070ee0 0x7fb9cc1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9b40a1150 0x7fb9b40a3600 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 --2- 172.21.15.26:0/2179041047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9cc071830 0x7fb9cc1a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.586+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 >> 172.21.15.26:0/2179041047 conn(0x7fb9cc06ca90 msgr2=0x7fb9cc06e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.588+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 shutdown_connections 2023-07-13T22:19:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:17.589+0000 7fb9d45ff700 1 -- 172.21.15.26:0/2179041047 wait complete. 2023-07-13T22:19:17.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:19:17.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:17 smithi121 ceph-mon[129187]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:19:17.927 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:17 smithi026 ceph-mon[95647]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:19:17.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:17 smithi117 ceph-mon[129074]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:19:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.183+0000 7fefcd1ff700 1 -- 172.21.15.26:0/3612223091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 msgr2=0x7fefc810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.183+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/3612223091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc810b7d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009990 tx=0x7fefb8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.184+0000 7fefcd1ff700 1 -- 172.21.15.26:0/3612223091 shutdown_connections 2023-07-13T22:19:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.184+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/3612223091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc810bd60 0x7fefc810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.184+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/3612223091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc8070ee0 0x7fefc80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.184+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/3612223091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc810b7d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.184+0000 7fefcd1ff700 1 -- 172.21.15.26:0/3612223091 >> 172.21.15.26:0/3612223091 conn(0x7fefc806ca90 msgr2=0x7fefc806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.185+0000 7fefcd1ff700 1 -- 172.21.15.26:0/3612223091 shutdown_connections 2023-07-13T22:19:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.185+0000 7fefcd1ff700 1 -- 172.21.15.26:0/3612223091 wait complete. 2023-07-13T22:19:18.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.186+0000 7fefcd1ff700 1 Processor -- start 2023-07-13T22:19:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.187+0000 7fefcd1ff700 1 -- start start 2023-07-13T22:19:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.187+0000 7fefcd1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc8070ee0 0x7fefc81a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.187+0000 7fefcd1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.188+0000 7fefcd1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc810bd60 0x7fefc81abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.188+0000 7fefcd1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc81ac200 con 0x7fefc8070ee0 2023-07-13T22:19:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.188+0000 7fefcd1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc810ec40 con 0x7fefc8071830 2023-07-13T22:19:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.188+0000 7fefcd1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc810edd0 con 0x7fefc810bd60 2023-07-13T22:19:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35994/0 (socket says 172.21.15.26:35994) 2023-07-13T22:19:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 -- 172.21.15.26:0/1254647880 learned_addr learned my addr 172.21.15.26:0/1254647880 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 -- 172.21.15.26:0/1254647880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc810bd60 msgr2=0x7fefc81abc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:19:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc810bd60 0x7fefc81abc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 -- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc8070ee0 msgr2=0x7fefc81a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc8070ee0 0x7fefc81a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.189+0000 7fefbffff700 1 -- 172.21.15.26:0/1254647880 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80095f0 con 0x7fefc8071830 2023-07-13T22:19:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.190+0000 7fefbffff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b680 tx=0x7fefb800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.190+0000 7fefc4d99700 1 -- 172.21.15.26:0/1254647880 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb801d070 con 0x7fefc8071830 2023-07-13T22:19:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.190+0000 7fefc4d99700 1 -- 172.21.15.26:0/1254647880 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb800fe40 con 0x7fefc8071830 2023-07-13T22:19:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.190+0000 7fefc4d99700 1 -- 172.21.15.26:0/1254647880 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8017650 con 0x7fefc8071830 2023-07-13T22:19:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.192+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc810f050 con 0x7fefc8071830 2023-07-13T22:19:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.192+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc810f480 con 0x7fefc8071830 2023-07-13T22:19:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.208+0000 7fefe0a95b80 1 Processor -- start 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.211+0000 7fefc4d99700 1 -- 172.21.15.26:0/1254647880 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fefb8021410 con 0x7fefc8071830 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.214+0000 7fefc4d99700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 0x7fefb40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.215+0000 7fefc4d99700 1 -- 172.21.15.26:0/1254647880 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fefb80cd9a0 con 0x7fefc8071830 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 -- start start 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089658c70 0x55f0896590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08955ccb0 con 0x55f089657e20 2023-07-13T22:19:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0895afa40 con 0x55f0896574d0 2023-07-13T22:19:18.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.217+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0895a8ed0 con 0x55f089658c70 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.218+0000 7fefbffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.218+0000 7fefbffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43170/0 (socket says 172.21.15.26:43170) 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.218+0000 7fefbffff700 1 -- 172.21.15.26:0/3296461291 learned_addr learned my addr 172.21.15.26:0/3296461291 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.219+0000 7fefc759e700 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089658c70 0x55f0896590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.219+0000 7fefc759e700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 0x7fefb40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.219+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.220+0000 7fefbffff700 1 -- 172.21.15.26:0/3296461291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089658c70 msgr2=0x55f0896590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.220+0000 7fefbffff700 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089658c70 0x55f0896590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.220+0000 7fefbffff700 1 -- 172.21.15.26:0/3296461291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 msgr2=0x55f0896578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.220+0000 7fefbffff700 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.220+0000 7fefbffff700 1 -- 172.21.15.26:0/3296461291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0893d6de0 con 0x55f089657e20 2023-07-13T22:19:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.221+0000 7fefbffff700 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefb80978f0 tx=0x7fefb8021c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517502136be3ee4a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.222+0000 7fefc759e700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 0x7fefb40a3750 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009690 tx=0x7fefc0004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.222+0000 7fefbd7fa700 1 -- 172.21.15.26:0/3296461291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb809eeb0 con 0x55f089657e20 2023-07-13T22:19:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.222+0000 7fefbd7fa700 1 -- 172.21.15.26:0/3296461291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb80a3240 con 0x55f089657e20 2023-07-13T22:19:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.222+0000 7fefbd7fa700 1 -- 172.21.15.26:0/3296461291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80a3010 con 0x55f089657e20 2023-07-13T22:19:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 -- 172.21.15.26:0/3296461291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 msgr2=0x55f0896586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fefb80978f0 tx=0x7fefb8021c70 comp rx=0 tx=0).stop 2023-07-13T22:19:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 -- 172.21.15.26:0/3296461291 shutdown_connections 2023-07-13T22:19:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089658c70 0x55f0896590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/3296461291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089657e20 0x55f0896586e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.224+0000 7fefe0a95b80 1 -- 172.21.15.26:0/3296461291 >> 172.21.15.26:0/3296461291 conn(0x55f08964cda0 msgr2=0x55f0896521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.225+0000 7fefe0a95b80 1 -- 172.21.15.26:0/3296461291 shutdown_connections 2023-07-13T22:19:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.225+0000 7fefe0a95b80 1 -- 172.21.15.26:0/3296461291 wait complete. 2023-07-13T22:19:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.226+0000 7fefe0a95b80 1 -- start start 2023-07-13T22:19:18.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.226+0000 7fefe0a95b80 1 Processor -- start 2023-07-13T22:19:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f089660df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089668370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089548ac0 con 0x55f089658c70 2023-07-13T22:19:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0895487c0 con 0x55f0896574d0 2023-07-13T22:19:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08942c920 con 0x55f089657e20 2023-07-13T22:19:18.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.227+0000 7fefe0a95b80 10 client.0 _ll_register_callbacks cb 0x55f089546760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.229+0000 7fefc759e700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089668370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.230+0000 7fefbffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.230+0000 7fefbffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52482/0 (socket says 172.21.15.26:52482) 2023-07-13T22:19:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.230+0000 7fefbffff700 1 -- 172.21.15.26:0/1717611180 learned_addr learned my addr 172.21.15.26:0/1717611180 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:18.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.230+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f089660df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.231+0000 7fefbffff700 1 -- 172.21.15.26:0/1717611180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 msgr2=0x55f089660df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.231+0000 7fefbffff700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f089660df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.231+0000 7fefbffff700 1 -- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 msgr2=0x55f089668370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.231+0000 7fefbffff700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089668370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.231+0000 7fefbffff700 1 -- 172.21.15.26:0/1717611180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc810f480 con 0x55f089657e20 2023-07-13T22:19:18.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.232+0000 7fefbffff700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fefb809b4e0 tx=0x7fefb809efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f86e4f5e428d592 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.238+0000 7fefae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.238+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80aeaf0 con 0x55f089657e20 2023-07-13T22:19:18.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.238+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fefb80aec50 con 0x55f089657e20 2023-07-13T22:19:18.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.241+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0895024f0 con 0x55f089657e20 2023-07-13T22:19:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.252+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefb80aee10 con 0x55f089657e20 2023-07-13T22:19:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.252+0000 7fefe0a95b80 10 client.26281 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.252+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f0893bfc60 con 0x55f089657e20 2023-07-13T22:19:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0895024f0 con 0x55f089657e20 2023-07-13T22:19:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fefb80b3430 con 0x55f089657e20 2023-07-13T22:19:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 10 client.26281 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 10 client.26281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:19:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 20 client.26281 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f08966c680 0x55f08966ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 4 client.26281 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0895a1eb0 con 0x55f08966c680 2023-07-13T22:19:18.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.253+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f08966c680 0x55f08966ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.256+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f08966c680 0x55f08966ea60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf119357b97d709e server_cookie=d0473a53a772fcc4 in_seq=0 out_seq=0 2023-07-13T22:19:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.259+0000 7fefae7fc700 10 client.26281 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.264+0000 7fefae7fc700 1 -- 172.21.15.26:0/1717611180 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0895a1eb0 con 0x55f08966c680 2023-07-13T22:19:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.264+0000 7fefae7fc700 10 client.26281 handle_command_reply: tid=0 2023-07-13T22:19:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.264+0000 7fefe0a95b80 1 client.26281 shutdown 2023-07-13T22:19:18.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.266+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 msgr2=0x55f089661330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.266+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fefb809b4e0 tx=0x7fefb809efd0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 shutdown_connections 2023-07-13T22:19:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f08966c680 0x55f08966ea60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089668370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f089660df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1717611180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089661330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.267+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 >> 172.21.15.26:0/1717611180 conn(0x55f08964cda0 msgr2=0x55f0896518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.268+0000 7fefae7fc700 0 client.26281 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:18.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.268+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 shutdown_connections 2023-07-13T22:19:18.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.268+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1717611180 wait complete. 2023-07-13T22:19:18.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.269+0000 7fefe0a95b80 20 client.26281 trim_cache size 0 max 16384 2023-07-13T22:19:18.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 Processor -- start 2023-07-13T22:19:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 -- start start 2023-07-13T22:19:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f089657e20 0x55f089668330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089660ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089661fe0 con 0x55f089658c70 2023-07-13T22:19:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089652ed0 con 0x55f089657e20 2023-07-13T22:19:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.301+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0896602a0 con 0x55f0896574d0 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52486/0 (socket says 172.21.15.26:52486) 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 -- 172.21.15.26:0/1151178545 learned_addr learned my addr 172.21.15.26:0/1151178545 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 -- 172.21.15.26:0/1151178545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f089657e20 msgr2=0x55f089668330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f089657e20 0x55f089668330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.302+0000 7fefc6d9d700 1 -- 172.21.15.26:0/1151178545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 msgr2=0x55f089660ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:18.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.317+0000 7fefc759e700 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089660ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.317+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089660ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.317+0000 7fefc6d9d700 1 -- 172.21.15.26:0/1151178545 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0893fd250 con 0x55f0896574d0 2023-07-13T22:19:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.321+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fefb00094f0 tx=0x7fefb00096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab358b7e63decae0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.327+0000 7fefbdffb700 1 -- 172.21.15.26:0/1151178545 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0018040 con 0x55f0896574d0 2023-07-13T22:19:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.327+0000 7fefbdffb700 1 -- 172.21.15.26:0/1151178545 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb001e070 con 0x55f0896574d0 2023-07-13T22:19:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.327+0000 7fefbdffb700 1 -- 172.21.15.26:0/1151178545 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0003be0 con 0x55f0896574d0 2023-07-13T22:19:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.328+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1151178545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 msgr2=0x55f0896590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.328+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fefb00094f0 tx=0x7fefb00096b0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.329+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1151178545 shutdown_connections 2023-07-13T22:19:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.329+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089660ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.329+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f0896574d0 0x55f0896590c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.329+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/1151178545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f089657e20 0x55f089668330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.329+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1151178545 >> 172.21.15.26:0/1151178545 conn(0x55f08964cda0 msgr2=0x55f08965bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.339+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1151178545 shutdown_connections 2023-07-13T22:19:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.339+0000 7fefe0a95b80 1 -- 172.21.15.26:0/1151178545 wait complete. 2023-07-13T22:19:18.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.340+0000 7fefe0a95b80 1 -- start start 2023-07-13T22:19:18.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.340+0000 7fefe0a95b80 1 Processor -- start 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089664a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089667880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089667dc0 con 0x55f089658c70 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f089667f30 con 0x55f0896574d0 2023-07-13T22:19:18.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0896680a0 con 0x55f089657e20 2023-07-13T22:19:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.341+0000 7fefe0a95b80 10 client.0 _ll_register_callbacks cb 0x7fefb0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.342+0000 7fefbffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089664a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.342+0000 7fefbffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089664a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52502/0 (socket says 172.21.15.26:52502) 2023-07-13T22:19:18.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.342+0000 7fefbffff700 1 -- 172.21.15.26:0/2334477975 learned_addr learned my addr 172.21.15.26:0/2334477975 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:18.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.342+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.345+0000 7fefc759e700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089667880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.350+0000 7fefc759e700 1 -- 172.21.15.26:0/2334477975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 msgr2=0x55f089664a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.350+0000 7fefc759e700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089664a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.350+0000 7fefc759e700 1 -- 172.21.15.26:0/2334477975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 msgr2=0x55f0896644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.350+0000 7fefc759e700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.350+0000 7fefc759e700 1 -- 172.21.15.26:0/2334477975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb00098c0 con 0x55f089658c70 2023-07-13T22:19:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.355+0000 7fefc759e700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089667880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009c00 tx=0x7fefc0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78a555d37adfced server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.355+0000 7fefae7fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:19:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.356+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0027070 con 0x55f089658c70 2023-07-13T22:19:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.356+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fefc0004bb0 con 0x55f089658c70 2023-07-13T22:19:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.356+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefb80a3240 con 0x55f089658c70 2023-07-13T22:19:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.357+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefc0004d70 con 0x55f089658c70 2023-07-13T22:19:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.357+0000 7fefe0a95b80 10 client.15987 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.357+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f0893d52b0 con 0x55f089658c70 2023-07-13T22:19:18.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb80a3240 con 0x55f089658c70 2023-07-13T22:19:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fefc002a020 con 0x55f089658c70 2023-07-13T22:19:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:19:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 20 client.15987 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f0896c16e0 0x55f0896c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 4 client.15987 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.358+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f089564050 con 0x55f0896c16e0 2023-07-13T22:19:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.359+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f0896c16e0 0x55f0896c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.360+0000 7fefc6d9d700 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f0896c16e0 0x55f0896c3af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb2e0097c59d1331 server_cookie=aa37b5d9362b5784 in_seq=0 out_seq=0 2023-07-13T22:19:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.360+0000 7fefae7fc700 10 client.15987 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.361+0000 7fefae7fc700 1 -- 172.21.15.26:0/2334477975 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f089564050 con 0x55f0896c16e0 2023-07-13T22:19:18.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.361+0000 7fefae7fc700 10 client.15987 handle_command_reply: tid=0 2023-07-13T22:19:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.361+0000 7fefe0a95b80 1 client.15987 shutdown 2023-07-13T22:19:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.361+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 msgr2=0x55f089667880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.361+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089667880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009c00 tx=0x7fefc0004b40 comp rx=0 tx=0).stop 2023-07-13T22:19:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 shutdown_connections 2023-07-13T22:19:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f0896c16e0 0x55f0896c3af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f089658c70 0x55f089667880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0896574d0 0x55f0896644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 --2- 172.21.15.26:0/2334477975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f089657e20 0x55f089664a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 >> 172.21.15.26:0/2334477975 conn(0x55f08964cda0 msgr2=0x55f089660ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.362+0000 7fefae7fc700 0 client.15987 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:18.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.364+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 shutdown_connections 2023-07-13T22:19:18.352 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:18.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.365+0000 7fefe0a95b80 1 -- 172.21.15.26:0/2334477975 wait complete. 2023-07-13T22:19:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.365+0000 7fefe0a95b80 20 client.15987 trim_cache size 0 max 16384 2023-07-13T22:19:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.366+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 msgr2=0x7fefb40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.366+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 0x7fefb40a3750 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009690 tx=0x7fefc0004ad0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.366+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 msgr2=0x7fefc81a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.366+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fefb800b680 tx=0x7fefb800f740 comp rx=0 tx=0).stop 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 shutdown_connections 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fefb40a12a0 0x7fefb40a3750 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefc8070ee0 0x7fefc81a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc810bd60 0x7fefc81abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 --2- 172.21.15.26:0/1254647880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefc8071830 0x7fefc81a4bc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:18.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 >> 172.21.15.26:0/1254647880 conn(0x7fefc806ca90 msgr2=0x7fefc806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 shutdown_connections 2023-07-13T22:19:18.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:18.367+0000 7fefcd1ff700 1 -- 172.21.15.26:0/1254647880 wait complete. 2023-07-13T22:19:18.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:19:19.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.034+0000 7f8da2fff700 1 -- 172.21.15.26:0/2776369501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 msgr2=0x7f8d9c109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.034+0000 7f8da2fff700 1 --2- 172.21.15.26:0/2776369501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c109520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 -- 172.21.15.26:0/2776369501 shutdown_connections 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 --2- 172.21.15.26:0/2776369501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 --2- 172.21.15.26:0/2776369501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c109520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 --2- 172.21.15.26:0/2776369501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 0x7f8d9c10bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 -- 172.21.15.26:0/2776369501 >> 172.21.15.26:0/2776369501 conn(0x7f8d9c06ca90 msgr2=0x7f8d9c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 -- 172.21.15.26:0/2776369501 shutdown_connections 2023-07-13T22:19:19.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.035+0000 7f8da2fff700 1 -- 172.21.15.26:0/2776369501 wait complete. 2023-07-13T22:19:19.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.037+0000 7f8da2fff700 1 Processor -- start 2023-07-13T22:19:19.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.037+0000 7f8da2fff700 1 -- start start 2023-07-13T22:19:19.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c10e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 0x7f8d9c10cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c10d8b0 con 0x7f8d9c10c5d0 2023-07-13T22:19:19.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c10da20 con 0x7f8d9c0722c0 2023-07-13T22:19:19.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.038+0000 7f8da2fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c1a81d0 con 0x7f8d9c109af0 2023-07-13T22:19:19.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.039+0000 7f8da159c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.039+0000 7f8da159c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43232/0 (socket says 172.21.15.26:43232) 2023-07-13T22:19:19.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.039+0000 7f8da159c700 1 -- 172.21.15.26:0/624329441 learned_addr learned my addr 172.21.15.26:0/624329441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c10e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 0x7f8d9c10cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 -- 172.21.15.26:0/624329441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 msgr2=0x7f8d9c10cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 0x7f8d9c10cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 -- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 msgr2=0x7f8d9c10d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.040+0000 7f8da0d9b700 1 -- 172.21.15.26:0/624329441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0095f0 con 0x7f8d9c0722c0 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.041+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c10e590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c000c00 tx=0x7f8d8c0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.041+0000 7f8d99ffb700 1 -- 172.21.15.26:0/624329441 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c01d070 con 0x7f8d9c0722c0 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.041+0000 7f8d99ffb700 1 -- 172.21.15.26:0/624329441 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c0173f0 con 0x7f8d9c0722c0 2023-07-13T22:19:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.041+0000 7f8d99ffb700 1 -- 172.21.15.26:0/624329441 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c017ea0 con 0x7f8d9c0722c0 2023-07-13T22:19:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.042+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c1a8450 con 0x7f8d9c0722c0 2023-07-13T22:19:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.042+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1a8820 con 0x7f8d9c0722c0 2023-07-13T22:19:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.049+0000 7f8db68cfb80 1 Processor -- start 2023-07-13T22:19:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.050+0000 7f8d99ffb700 1 -- 172.21.15.26:0/624329441 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8d8c017560 con 0x7f8d9c0722c0 2023-07-13T22:19:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.051+0000 7f8db68cfb80 1 -- start start 2023-07-13T22:19:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 0x55c9f08f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f8480 0x55c9f08f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f07f3cb0 con 0x55c9f08f8480 2023-07-13T22:19:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f0846a40 con 0x55c9f08f7630 2023-07-13T22:19:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.052+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f083fed0 con 0x55c9f08f6ce0 2023-07-13T22:19:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.053+0000 7f8da159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.054+0000 7f8da159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52522/0 (socket says 172.21.15.26:52522) 2023-07-13T22:19:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.054+0000 7f8da159c700 1 -- 172.21.15.26:0/2532456758 learned_addr learned my addr 172.21.15.26:0/2532456758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.054+0000 7f8d99ffb700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 0x7f8d840a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.055+0000 7f8d99ffb700 1 -- 172.21.15.26:0/624329441 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8d8c0cde80 con 0x7f8d9c0722c0 2023-07-13T22:19:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.055+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 0x55c9f08f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.056+0000 7f8da0d9b700 1 -- 172.21.15.26:0/2532456758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 msgr2=0x55c9f08f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.056+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.056+0000 7f8da0d9b700 1 -- 172.21.15.26:0/2532456758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f8480 msgr2=0x55c9f08f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.056+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f8480 0x55c9f08f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.056+0000 7f8da0d9b700 1 -- 172.21.15.26:0/2532456758 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9f066dde0 con 0x55c9f08f7630 2023-07-13T22:19:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.057+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 0x55c9f08f7ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c017810 tx=0x7f8d8c09aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc2a38333f30279 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.059+0000 7f8d82ffd700 1 -- 172.21.15.26:0/2532456758 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09b2d0 con 0x55c9f08f7630 2023-07-13T22:19:19.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.059+0000 7f8d82ffd700 1 -- 172.21.15.26:0/2532456758 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c09f300 con 0x55c9f08f7630 2023-07-13T22:19:19.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.059+0000 7f8d82ffd700 1 -- 172.21.15.26:0/2532456758 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09b530 con 0x55c9f08f7630 2023-07-13T22:19:19.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8da159c700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 0x7f8d840a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 -- 172.21.15.26:0/2532456758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 msgr2=0x55c9f08f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 0x55c9f08f7ef0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c017810 tx=0x7f8d8c09aee0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 -- 172.21.15.26:0/2532456758 shutdown_connections 2023-07-13T22:19:19.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f8480 0x55c9f08f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/2532456758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f7630 0x55c9f08f7ef0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.060+0000 7f8db68cfb80 1 -- 172.21.15.26:0/2532456758 >> 172.21.15.26:0/2532456758 conn(0x55c9f08e3da0 msgr2=0x55c9f08e91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.061+0000 7f8db68cfb80 1 -- 172.21.15.26:0/2532456758 shutdown_connections 2023-07-13T22:19:19.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.061+0000 7f8db68cfb80 1 -- 172.21.15.26:0/2532456758 wait complete. 2023-07-13T22:19:19.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.062+0000 7f8db68cfb80 1 -- start start 2023-07-13T22:19:19.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.062+0000 7f8db68cfb80 1 Processor -- start 2023-07-13T22:19:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0900690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0900bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0907c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f07dfac0 con 0x55c9f08f7630 2023-07-13T22:19:19.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f07df7c0 con 0x55c9f08f8480 2023-07-13T22:19:19.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f06c3920 con 0x55c9f08f6ce0 2023-07-13T22:19:19.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.063+0000 7f8db68cfb80 10 client.0 _ll_register_callbacks cb 0x55c9f07dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:19.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.067+0000 7f8da159c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0907c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.067+0000 7f8da159c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0907c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36060/0 (socket says 172.21.15.26:36060) 2023-07-13T22:19:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.067+0000 7f8da159c700 1 -- 172.21.15.26:0/1068944351 learned_addr learned my addr 172.21.15.26:0/1068944351 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:19.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.067+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0900bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.070+0000 7f8d9bfff700 1 -- 172.21.15.26:0/1068944351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 msgr2=0x55c9f0900690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.070+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0900690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.070+0000 7f8d9bfff700 1 -- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 msgr2=0x55c9f0907c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.070+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0907c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.070+0000 7f8d9bfff700 1 -- 172.21.15.26:0/1068944351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1a8820 con 0x55c9f08f7630 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.071+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0900bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d900116c0 tx=0x7f8d900119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f9a4d64d86e7f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.071+0000 7f8d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:19:19.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.071+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004020 con 0x55c9f08f7630 2023-07-13T22:19:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.071+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8d90004180 con 0x55c9f08f7630 2023-07-13T22:19:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.071+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9f07994f0 con 0x55c9f08f7630 2023-07-13T22:19:19.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.072+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d90004340 con 0x55c9f08f7630 2023-07-13T22:19:19.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.076+0000 7f8da159c700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 0x7f8d840a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009490 tx=0x7f8d94007730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.076+0000 7f8db68cfb80 10 client.15993 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.076+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9f0656c60 con 0x55c9f08f7630 2023-07-13T22:19:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.077+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9f07994f0 con 0x55c9f08f7630 2023-07-13T22:19:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.077+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d9000e820 con 0x55c9f08f7630 2023-07-13T22:19:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:19.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 20 client.15993 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f090c4f0 0x55c9f090e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 4 client.15993 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9f0838eb0 con 0x55c9f090c4f0 2023-07-13T22:19:19.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.078+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f090c4f0 0x55c9f090e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.080+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f090c4f0 0x55c9f090e900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c208f2e4aae47b12 server_cookie=f8b039081023658a in_seq=0 out_seq=0 2023-07-13T22:19:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.080+0000 7f8d7bfff700 10 client.15993 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:19.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.080+0000 7f8d7bfff700 1 -- 172.21.15.26:0/1068944351 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9f0838eb0 con 0x55c9f090c4f0 2023-07-13T22:19:19.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.080+0000 7f8d7bfff700 10 client.15993 handle_command_reply: tid=0 2023-07-13T22:19:19.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.081+0000 7f8db68cfb80 1 client.15993 shutdown 2023-07-13T22:19:19.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.090+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 msgr2=0x55c9f0900bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.090+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0900bd0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8d900116c0 tx=0x7f8d900119d0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 shutdown_connections 2023-07-13T22:19:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f090c4f0 0x55c9f090e900 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0907c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0900690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/1068944351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0900bd0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 >> 172.21.15.26:0/1068944351 conn(0x55c9f08e3da0 msgr2=0x55c9f08e88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8d7bfff700 0 client.15993 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:19.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 shutdown_connections 2023-07-13T22:19:19.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.091+0000 7f8db68cfb80 1 -- 172.21.15.26:0/1068944351 wait complete. 2023-07-13T22:19:19.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.107+0000 7f8db68cfb80 20 client.15993 trim_cache size 0 max 16384 2023-07-13T22:19:19.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.140+0000 7f8db68cfb80 1 Processor -- start 2023-07-13T22:19:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 -- start start 2023-07-13T22:19:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0907bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0908500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f0909900 con 0x55c9f08f7630 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f08ffb40 con 0x55c9f08f8480 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.152+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f08ead00 con 0x55c9f08f6ce0 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.153+0000 7f8da0d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.153+0000 7f8da0d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52546/0 (socket says 172.21.15.26:52546) 2023-07-13T22:19:19.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.153+0000 7f8da0d9b700 1 -- 172.21.15.26:0/455875896 learned_addr learned my addr 172.21.15.26:0/455875896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:19.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 -- 172.21.15.26:0/455875896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 msgr2=0x55c9f0908500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0908500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 -- 172.21.15.26:0/455875896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 msgr2=0x55c9f0907bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0907bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 -- 172.21.15.26:0/455875896 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9f0694250 con 0x55c9f08f6ce0 2023-07-13T22:19:19.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.158+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c09ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bcdf69c6ebdca19 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.159+0000 7f8d837fe700 1 -- 172.21.15.26:0/455875896 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0a11c0 con 0x55c9f08f6ce0 2023-07-13T22:19:19.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.159+0000 7f8d837fe700 1 -- 172.21.15.26:0/455875896 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c099410 con 0x55c9f08f6ce0 2023-07-13T22:19:19.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.159+0000 7f8d837fe700 1 -- 172.21.15.26:0/455875896 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c09aa20 con 0x55c9f08f6ce0 2023-07-13T22:19:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 -- 172.21.15.26:0/455875896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 msgr2=0x55c9f08ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009990 tx=0x7f8d8c09ad30 comp rx=0 tx=0).stop 2023-07-13T22:19:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 -- 172.21.15.26:0/455875896 shutdown_connections 2023-07-13T22:19:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0908500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f08ff390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/455875896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0907bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 -- 172.21.15.26:0/455875896 >> 172.21.15.26:0/455875896 conn(0x55c9f08e3da0 msgr2=0x55c9f0913260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.163+0000 7f8db68cfb80 1 -- 172.21.15.26:0/455875896 shutdown_connections 2023-07-13T22:19:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.167+0000 7f8db68cfb80 1 -- 172.21.15.26:0/455875896 wait complete. 2023-07-13T22:19:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.168+0000 7f8db68cfb80 1 -- start start 2023-07-13T22:19:19.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.168+0000 7f8db68cfb80 1 Processor -- start 2023-07-13T22:19:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0903fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0906fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f0907520 con 0x55c9f08f7630 2023-07-13T22:19:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f0907690 con 0x55c9f08f8480 2023-07-13T22:19:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9f0907850 con 0x55c9f08f6ce0 2023-07-13T22:19:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.169+0000 7f8db68cfb80 10 client.0 _ll_register_callbacks cb 0x55c9f07dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.170+0000 7f8da0d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.170+0000 7f8da0d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52562/0 (socket says 172.21.15.26:52562) 2023-07-13T22:19:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.170+0000 7f8da0d9b700 1 -- 172.21.15.26:0/250000725 learned_addr learned my addr 172.21.15.26:0/250000725 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.174+0000 7f8da0d9b700 1 -- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 msgr2=0x55c9f0906fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.174+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0906fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.174+0000 7f8da0d9b700 1 -- 172.21.15.26:0/250000725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 msgr2=0x55c9f0903fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.174+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0903fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.174+0000 7f8da0d9b700 1 -- 172.21.15.26:0/250000725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9f066dde0 con 0x55c9f08f6ce0 2023-07-13T22:19:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.175+0000 7f8da0d9b700 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0a3160 tx=0x7f8d8c09b380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ca52e47ee9395b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.176+0000 7f8d7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:19.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.176+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c099670 con 0x55c9f08f6ce0 2023-07-13T22:19:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.176+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8d8c09b530 con 0x55c9f08f6ce0 2023-07-13T22:19:19.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.176+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d8c09f300 con 0x55c9f08f6ce0 2023-07-13T22:19:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.177+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d8c09b730 con 0x55c9f08f6ce0 2023-07-13T22:19:19.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.178+0000 7f8db68cfb80 10 client.26302 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:19.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.178+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9f066c2b0 con 0x55c9f08f6ce0 2023-07-13T22:19:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.184+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c09f300 con 0x55c9f08f6ce0 2023-07-13T22:19:19.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.184+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d8c0970c0 con 0x55c9f08f6ce0 2023-07-13T22:19:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 10 client.26302 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:19.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 10 client.26302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 20 client.26302 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f0960ea0 0x55c9f0963280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 4 client.26302 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:19.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9f07fb050 con 0x55c9f0960ea0 2023-07-13T22:19:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.187+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f0960ea0 0x55c9f0963280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.188+0000 7f8d9bfff700 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f0960ea0 0x55c9f0963280 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7590d3dc9848d413 server_cookie=c85955a320a8881d in_seq=0 out_seq=0 2023-07-13T22:19:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.188+0000 7f8d7bfff700 10 client.26302 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.188+0000 7f8d7bfff700 1 -- 172.21.15.26:0/250000725 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9f07fb050 con 0x55c9f0960ea0 2023-07-13T22:19:19.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.188+0000 7f8d7bfff700 10 client.26302 handle_command_reply: tid=0 2023-07-13T22:19:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.188+0000 7f8db68cfb80 1 client.26302 shutdown 2023-07-13T22:19:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 msgr2=0x55c9f0903a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0a3160 tx=0x7f8d8c09b380 comp rx=0 tx=0).stop 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 shutdown_connections 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c9f0960ea0 0x55c9f0963280 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9f08f8480 0x55c9f0906fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f08f6ce0 0x55c9f0903a80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 --2- 172.21.15.26:0/250000725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c9f08f7630 0x55c9f0903fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 >> 172.21.15.26:0/250000725 conn(0x55c9f08e3da0 msgr2=0x55c9f08ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8d7bfff700 0 client.26302 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 shutdown_connections 2023-07-13T22:19:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.189+0000 7f8db68cfb80 1 -- 172.21.15.26:0/250000725 wait complete. 2023-07-13T22:19:19.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.190+0000 7f8db68cfb80 20 client.26302 trim_cache size 0 max 16384 2023-07-13T22:19:19.178 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:19.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.190+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 msgr2=0x7f8d840a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.190+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 0x7f8d840a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009490 tx=0x7f8d94007730 comp rx=0 tx=0).stop 2023-07-13T22:19:19.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.190+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 msgr2=0x7f8d9c10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.190+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c10e590 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c000c00 tx=0x7f8d8c0057b0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 shutdown_connections 2023-07-13T22:19:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8d840a1350 0x7f8d840a3800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d9c10c5d0 0x7f8d9c10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d9c0722c0 0x7f8d9c10e590 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 --2- 172.21.15.26:0/624329441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8d9c109af0 0x7f8d9c10cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:19.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 >> 172.21.15.26:0/624329441 conn(0x7f8d9c06ca90 msgr2=0x7f8d9c1088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 shutdown_connections 2023-07-13T22:19:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:19.191+0000 7f8da2fff700 1 -- 172.21.15.26:0/624329441 wait complete. 2023-07-13T22:19:19.582 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:19 smithi026 ceph-mon[95647]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 279 MiB/s wr, 92 op/s 2023-07-13T22:19:19.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:19 smithi117 ceph-mon[129074]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 279 MiB/s wr, 92 op/s 2023-07-13T22:19:19.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:19 smithi121 ceph-mon[129187]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 279 MiB/s wr, 92 op/s 2023-07-13T22:19:20.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:19:20.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:19:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/3736994441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb8107390 msgr2=0x7f1cb81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/3736994441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb8107390 0x7f1cb81077e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40098f0 tx=0x7f1cb4009c00 comp rx=0 tx=0).stop 2023-07-13T22:19:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/3736994441 shutdown_connections 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/3736994441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107d20 0x7f1cb810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/3736994441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb81061a0 0x7f1cb81065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/3736994441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb8107390 0x7f1cb81077e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.076+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/3736994441 >> 172.21.15.26:0/3736994441 conn(0x7f1cb80751c0 msgr2=0x7f1cb8077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.077+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/3736994441 shutdown_connections 2023-07-13T22:19:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.077+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/3736994441 wait complete. 2023-07-13T22:19:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.078+0000 7f1cbf3ff700 1 Processor -- start 2023-07-13T22:19:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 -- start start 2023-07-13T22:19:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb81061a0 0x7f1cb819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb8107d20 0x7f1cb81a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb81a39f0 con 0x7f1cb8107390 2023-07-13T22:19:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8078d60 con 0x7f1cb8107d20 2023-07-13T22:19:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.079+0000 7f1cbf3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb8078ef0 con 0x7f1cb81061a0 2023-07-13T22:19:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.080+0000 7f1cbc99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.080+0000 7f1cbc99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43282/0 (socket says 172.21.15.26:43282) 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.080+0000 7f1cbc99a700 1 -- 172.21.15.26:0/2933542390 learned_addr learned my addr 172.21.15.26:0/2933542390 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.080+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb81061a0 0x7f1cb819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb8107d20 0x7f1cb81a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 -- 172.21.15.26:0/2933542390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb81061a0 msgr2=0x7f1cb819be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb81061a0 0x7f1cb819be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 -- 172.21.15.26:0/2933542390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb8107d20 msgr2=0x7f1cb81a3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb8107d20 0x7f1cb81a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 -- 172.21.15.26:0/2933542390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009520 con 0x7f1cb8107390 2023-07-13T22:19:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.081+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400b7c0 tx=0x7f1cb400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.082+0000 7f1cae7fc700 1 -- 172.21.15.26:0/2933542390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4005310 con 0x7f1cb8107390 2023-07-13T22:19:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.082+0000 7f1cae7fc700 1 -- 172.21.15.26:0/2933542390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cb400f7d0 con 0x7f1cb8107390 2023-07-13T22:19:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.082+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb8079170 con 0x7f1cb8107390 2023-07-13T22:19:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.082+0000 7f1cae7fc700 1 -- 172.21.15.26:0/2933542390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40176f0 con 0x7f1cb8107390 2023-07-13T22:19:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.083+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb8079680 con 0x7f1cb8107390 2023-07-13T22:19:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.089+0000 7f1cd2c92b80 1 Processor -- start 2023-07-13T22:19:21.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 -- start start 2023-07-13T22:19:21.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741550 0x559780741960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780743160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559780646d30 con 0x559780741550 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559780699ac0 con 0x559780742cf0 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559780692f50 con 0x559780741ea0 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.090+0000 7f1cae7fc700 1 -- 172.21.15.26:0/2933542390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1cb4003680 con 0x7f1cb8107390 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52588/0 (socket says 172.21.15.26:52588) 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 -- 172.21.15.26:0/3158300831 learned_addr learned my addr 172.21.15.26:0/3158300831 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 -- 172.21.15.26:0/3158300831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 msgr2=0x559780743160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780743160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 -- 172.21.15.26:0/3158300831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741550 msgr2=0x559780741960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741550 0x559780741960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.091+0000 7f1cbd19b700 1 -- 172.21.15.26:0/3158300831 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597804c0e60 con 0x559780741ea0 2023-07-13T22:19:21.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.092+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009be0 tx=0x7f1ca8009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589d64cca8156bc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.094+0000 7f1cae7fc700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 0x7f1ca40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.094+0000 7f1cae7fc700 1 -- 172.21.15.26:0/2933542390 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cb40cce40 con 0x7f1cb8107390 2023-07-13T22:19:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.095+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 0x7f1ca40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.095+0000 7f1ca37fe700 1 -- 172.21.15.26:0/3158300831 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8010bd0 con 0x559780741ea0 2023-07-13T22:19:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.095+0000 7f1ca37fe700 1 -- 172.21.15.26:0/3158300831 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca80044f0 con 0x559780741ea0 2023-07-13T22:19:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.095+0000 7f1ca37fe700 1 -- 172.21.15.26:0/3158300831 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8005020 con 0x559780741ea0 2023-07-13T22:19:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.096+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/3158300831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 msgr2=0x559780742760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8009be0 tx=0x7f1ca8009ef0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 0x7f1ca40a3800 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009300 tx=0x7f1cb0009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/3158300831 shutdown_connections 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741550 0x559780741960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780743160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/3158300831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780742760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/3158300831 >> 172.21.15.26:0/3158300831 conn(0x559780736e20 msgr2=0x55978073c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/3158300831 shutdown_connections 2023-07-13T22:19:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.097+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/3158300831 wait complete. 2023-07-13T22:19:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.098+0000 7f1cd2c92b80 1 -- start start 2023-07-13T22:19:21.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.098+0000 7f1cd2c92b80 1 Processor -- start 2023-07-13T22:19:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741550 0x55978074b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x55978074b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559780632b40 con 0x559780741ea0 2023-07-13T22:19:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559780632840 con 0x559780742cf0 2023-07-13T22:19:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597805169a0 con 0x559780741550 2023-07-13T22:19:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.099+0000 7f1cd2c92b80 10 client.0 _ll_register_callbacks cb 0x5597806307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:21.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.100+0000 7f1cbd99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.100+0000 7f1cbd99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36102/0 (socket says 172.21.15.26:36102) 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.100+0000 7f1cbd99c700 1 -- 172.21.15.26:0/1995725969 learned_addr learned my addr 172.21.15.26:0/1995725969 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.100+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x55978074b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 -- 172.21.15.26:0/1995725969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741550 msgr2=0x55978074b2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741550 0x55978074b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 -- 172.21.15.26:0/1995725969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 msgr2=0x55978074b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x55978074b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 -- 172.21.15.26:0/1995725969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca8009840 con 0x559780742cf0 2023-07-13T22:19:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x55978074b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:19:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00166a0 tx=0x7f1cb0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369a9d55e735382b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.101+0000 7f1ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0004db0 con 0x559780742cf0 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cb00145e0 con 0x559780742cf0 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597805ec570 con 0x559780742cf0 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cb00147a0 con 0x559780742cf0 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1cd2c92b80 10 client.26327 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:21.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.102+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5597804a9ce0 con 0x559780742cf0 2023-07-13T22:19:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.103+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5597805ec570 con 0x559780742cf0 2023-07-13T22:19:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.103+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1cb0005590 con 0x559780742cf0 2023-07-13T22:19:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 10 client.26327 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:21.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 20 client.26327 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807680d0 0x55978076a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 4 client.26327 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.104+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55978068bf30 con 0x5597807680d0 2023-07-13T22:19:21.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.105+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807680d0 0x55978076a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.107+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807680d0 0x55978076a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5013bbc03ec8de3c server_cookie=6bc0f9e7e5ababd6 in_seq=0 out_seq=0 2023-07-13T22:19:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.107+0000 7f1ca0ff9700 10 client.26327 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.108+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/1995725969 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55978068bf30 con 0x5597807680d0 2023-07-13T22:19:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.108+0000 7f1ca0ff9700 10 client.26327 handle_command_reply: tid=0 2023-07-13T22:19:21.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.108+0000 7f1cd2c92b80 1 client.26327 shutdown 2023-07-13T22:19:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.108+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 msgr2=0x559780752860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.108+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00166a0 tx=0x7f1cb0004b40 comp rx=0 tx=0).stop 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 shutdown_connections 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807680d0 0x55978076a4e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780742cf0 0x559780752860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741550 0x55978074b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/1995725969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x55978074b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 >> 172.21.15.26:0/1995725969 conn(0x559780736e20 msgr2=0x55978073b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1ca0ff9700 0 client.26327 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 shutdown_connections 2023-07-13T22:19:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.109+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/1995725969 wait complete. 2023-07-13T22:19:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.110+0000 7f1cd2c92b80 20 client.26327 trim_cache size 0 max 16384 2023-07-13T22:19:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.136+0000 7f1cd2c92b80 1 Processor -- start 2023-07-13T22:19:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 -- start start 2023-07-13T22:19:21.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780752820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 0x5597807461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978074a7d0 con 0x559780742cf0 2023-07-13T22:19:21.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597807422a0 con 0x559780741550 2023-07-13T22:19:21.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978074a600 con 0x559780741ea0 2023-07-13T22:19:21.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.137+0000 7f1cbd19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36118/0 (socket says 172.21.15.26:36118) 2023-07-13T22:19:21.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 -- 172.21.15.26:0/591448915 learned_addr learned my addr 172.21.15.26:0/591448915 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780752820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 0x5597807461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 -- 172.21.15.26:0/591448915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 msgr2=0x559780752820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780752820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 -- 172.21.15.26:0/591448915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 msgr2=0x5597807461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 0x5597807461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd19b700 1 -- 172.21.15.26:0/591448915 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597804e72d0 con 0x559780741550 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780752820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:19:21.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.138+0000 7f1cbd99c700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 0x5597807461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:19:21.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.139+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8010700 tx=0x7f1ca8009b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ff75db352ae05c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.139+0000 7f1ca3fff700 1 -- 172.21.15.26:0/591448915 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80109d0 con 0x559780741550 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.139+0000 7f1ca3fff700 1 -- 172.21.15.26:0/591448915 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca8026970 con 0x559780741550 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.139+0000 7f1ca3fff700 1 -- 172.21.15.26:0/591448915 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80129b0 con 0x559780741550 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/591448915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 msgr2=0x559780745c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8010700 tx=0x7f1ca8009b10 comp rx=0 tx=0).stop 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/591448915 shutdown_connections 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780742cf0 0x5597807461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x559780745c90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/591448915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780741ea0 0x559780752820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/591448915 >> 172.21.15.26:0/591448915 conn(0x559780736e20 msgr2=0x559780749130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/591448915 shutdown_connections 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.140+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/591448915 wait complete. 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.141+0000 7f1cd2c92b80 1 -- start start 2023-07-13T22:19:21.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.141+0000 7f1cd2c92b80 1 Processor -- start 2023-07-13T22:19:21.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x5597807516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780742cf0 0x5597807bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597807522e0 con 0x559780741ea0 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597807bc4e0 con 0x559780741550 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597807bc670 con 0x559780742cf0 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.142+0000 7f1cd2c92b80 10 client.0 _ll_register_callbacks cb 0x5597806307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.143+0000 7f1cbc99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.143+0000 7f1cbc99a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43336/0 (socket says 172.21.15.26:43336) 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.143+0000 7f1cbc99a700 1 -- 172.21.15.26:0/402229099 learned_addr learned my addr 172.21.15.26:0/402229099 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 -- 172.21.15.26:0/402229099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780742cf0 msgr2=0x5597807bbfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780742cf0 0x5597807bbfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 -- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 msgr2=0x5597807516c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x5597807516c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 -- 172.21.15.26:0/402229099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597804c0e60 con 0x559780741ea0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.144+0000 7f1cbc99a700 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40cd0e0 tx=0x7f1cb409a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b517881b63add8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.145+0000 7f1ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.145+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4094ea0 con 0x559780741ea0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.145+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1cb40a2650 con 0x559780741ea0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.145+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ca80044f0 con 0x559780741ea0 2023-07-13T22:19:21.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cb409e020 con 0x559780741ea0 2023-07-13T22:19:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1cd2c92b80 10 client.16005 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5597804bf330 con 0x559780741ea0 2023-07-13T22:19:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca80044f0 con 0x559780741ea0 2023-07-13T22:19:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1cb40a21c0 con 0x559780741ea0 2023-07-13T22:19:21.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1cd2c92b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1cd2c92b80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.146+0000 7f1cd2c92b80 20 client.16005 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.147+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807bdcd0 0x5597807c00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.147+0000 7f1cd2c92b80 4 client.16005 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.147+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55978064e0d0 con 0x5597807bdcd0 2023-07-13T22:19:21.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.147+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807bdcd0 0x5597807c00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.149+0000 7f1cbd19b700 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807bdcd0 0x5597807c00e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a02e3823f43509f server_cookie=268a409d6307af36 in_seq=0 out_seq=0 2023-07-13T22:19:21.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.150+0000 7f1ca0ff9700 10 client.16005 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.151+0000 7f1ca0ff9700 1 -- 172.21.15.26:0/402229099 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55978064e0d0 con 0x5597807bdcd0 2023-07-13T22:19:21.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.151+0000 7f1ca0ff9700 10 client.16005 handle_command_reply: tid=0 2023-07-13T22:19:21.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.151+0000 7f1cd2c92b80 1 client.16005 shutdown 2023-07-13T22:19:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.151+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 msgr2=0x559780751c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.151+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40cd0e0 tx=0x7f1cb409a290 comp rx=0 tx=0).stop 2023-07-13T22:19:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 shutdown_connections 2023-07-13T22:19:21.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5597807bdcd0 0x5597807c00e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559780742cf0 0x5597807bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559780741550 0x5597807516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 --2- 172.21.15.26:0/402229099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559780741ea0 0x559780751c00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 >> 172.21.15.26:0/402229099 conn(0x559780736e20 msgr2=0x559780745c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.154+0000 7f1ca0ff9700 0 client.16005 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.156+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 shutdown_connections 2023-07-13T22:19:21.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.156+0000 7f1cd2c92b80 1 -- 172.21.15.26:0/402229099 wait complete. 2023-07-13T22:19:21.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.160+0000 7f1cd2c92b80 20 client.16005 trim_cache size 0 max 16384 2023-07-13T22:19:21.146 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:19:21.146 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:19:21.146 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "9673ada1-c9f9-4612-8d78-1c9378a3b054", 2023-07-13T22:19:21.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:19:21.147 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:19:21.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.168+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 msgr2=0x7f1ca40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.168+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 0x7f1ca40a3800 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009300 tx=0x7f1cb0009920 comp rx=0 tx=0).stop 2023-07-13T22:19:21.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.168+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 msgr2=0x7f1cb819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.168+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400b7c0 tx=0x7f1cb400b8a0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 shutdown_connections 2023-07-13T22:19:21.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cb8107d20 0x7f1cb81a3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1cb81061a0 0x7f1cb819be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1ca40a1350 0x7f1ca40a3800 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 --2- 172.21.15.26:0/2933542390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cb8107390 0x7f1cb819c3b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.169+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 >> 172.21.15.26:0/2933542390 conn(0x7f1cb80751c0 msgr2=0x7f1cb8077450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.179+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 shutdown_connections 2023-07-13T22:19:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.184+0000 7f1cbf3ff700 1 -- 172.21.15.26:0/2933542390 wait complete. 2023-07-13T22:19:21.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:19:21.652 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:21 smithi026 ceph-mon[95647]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:19:21.653 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:21 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:19:21.653 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:21 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:21.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.790+0000 7f17953ff700 1 -- 172.21.15.26:0/1026119304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 msgr2=0x7f179010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.790+0000 7f17953ff700 1 --2- 172.21.15.26:0/1026119304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f179010e1f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1788009f90 tx=0x7f178800b330 comp rx=0 tx=0).stop 2023-07-13T22:19:21.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.808+0000 7f17953ff700 1 -- 172.21.15.26:0/1026119304 shutdown_connections 2023-07-13T22:19:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.808+0000 7f17953ff700 1 --2- 172.21.15.26:0/1026119304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f179010e1f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.808+0000 7f17953ff700 1 --2- 172.21.15.26:0/1026119304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17900712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.808+0000 7f17953ff700 1 --2- 172.21.15.26:0/1026119304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f179010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.808+0000 7f17953ff700 1 -- 172.21.15.26:0/1026119304 >> 172.21.15.26:0/1026119304 conn(0x7f179006ca90 msgr2=0x7f179006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.819+0000 7f17953ff700 1 -- 172.21.15.26:0/1026119304 shutdown_connections 2023-07-13T22:19:21.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.819+0000 7f17953ff700 1 -- 172.21.15.26:0/1026119304 wait complete. 2023-07-13T22:19:21.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.820+0000 7f17953ff700 1 Processor -- start 2023-07-13T22:19:21.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.821+0000 7f17953ff700 1 -- start start 2023-07-13T22:19:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17901a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f17901abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901ac140 con 0x7f179010bd60 2023-07-13T22:19:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901ac280 con 0x7f1790071830 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f17953ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901ac410 con 0x7f1790070ee0 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.831+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.832+0000 7f178effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17901a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.832+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36156/0 (socket says 172.21.15.26:36156) 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.832+0000 7f178e7fc700 1 -- 172.21.15.26:0/1666085067 learned_addr learned my addr 172.21.15.26:0/1666085067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.832+0000 7f178f7fe700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f17901abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 -- 172.21.15.26:0/1666085067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 msgr2=0x7f17901a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17901a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 -- 172.21.15.26:0/1666085067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 msgr2=0x7f17901a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 -- 172.21.15.26:0/1666085067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1788009c40 con 0x7f179010bd60 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178e7fc700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.833+0000 7f178f7fe700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f17901abb70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f178800b770 tx=0x7f1788007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.834+0000 7f1777fff700 1 -- 172.21.15.26:0/1666085067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178800e040 con 0x7f179010bd60 2023-07-13T22:19:21.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.834+0000 7f1777fff700 1 -- 172.21.15.26:0/1666085067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1788004e60 con 0x7f179010bd60 2023-07-13T22:19:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.834+0000 7f1777fff700 1 -- 172.21.15.26:0/1666085067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178801b750 con 0x7f179010bd60 2023-07-13T22:19:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.837+0000 7f178effd700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17901a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:19:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.837+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901ac690 con 0x7f179010bd60 2023-07-13T22:19:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.837+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901aca60 con 0x7f179010bd60 2023-07-13T22:19:21.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.844+0000 7f17a8d30b80 1 Processor -- start 2023-07-13T22:19:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.848+0000 7f1777fff700 1 -- 172.21.15.26:0/1666085067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f178801d9c0 con 0x7f179010bd60 2023-07-13T22:19:21.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.851+0000 7f1777fff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 0x7f17780a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.851+0000 7f1777fff700 1 -- 172.21.15.26:0/1666085067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17880cd580 con 0x7f179010bd60 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.852+0000 7f17a8d30b80 1 -- start start 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fb86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb8c70 0x55eed5fb90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5ebccb0 con 0x55eed5fb74d0 2023-07-13T22:19:21.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5f0fa40 con 0x55eed5fb8c70 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.853+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5f08ed0 con 0x55eed5fb7e20 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.854+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fb86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.854+0000 7f178e7fc700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 0x7f17780a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.854+0000 7f178effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.855+0000 7f178effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43356/0 (socket says 172.21.15.26:43356) 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.855+0000 7f178effd700 1 -- 172.21.15.26:0/253367693 learned_addr learned my addr 172.21.15.26:0/253367693 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.855+0000 7f178f7fe700 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb8c70 0x55eed5fb90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.855+0000 7f178effd700 1 -- 172.21.15.26:0/253367693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 msgr2=0x55eed5fb86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.855+0000 7f178effd700 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fb86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.856+0000 7f178effd700 1 -- 172.21.15.26:0/253367693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb8c70 msgr2=0x55eed5fb90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.856+0000 7f178effd700 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb8c70 0x55eed5fb90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.856+0000 7f178effd700 1 -- 172.21.15.26:0/253367693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed5d36de0 con 0x55eed5fb74d0 2023-07-13T22:19:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.856+0000 7f178effd700 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f178000bfd0 tx=0x7f1780010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24512b7488b3d5ba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.858+0000 7f17757fa700 1 -- 172.21.15.26:0/253367693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178001c680 con 0x55eed5fb74d0 2023-07-13T22:19:21.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.859+0000 7f17757fa700 1 -- 172.21.15.26:0/253367693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1780015070 con 0x55eed5fb74d0 2023-07-13T22:19:21.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.859+0000 7f17757fa700 1 -- 172.21.15.26:0/253367693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011a90 con 0x55eed5fb74d0 2023-07-13T22:19:21.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.860+0000 7f17a8d30b80 1 -- 172.21.15.26:0/253367693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 msgr2=0x55eed5fb78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.860+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f178000bfd0 tx=0x7f1780010830 comp rx=0 tx=0).stop 2023-07-13T22:19:21.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.862+0000 7f17a8d30b80 1 -- 172.21.15.26:0/253367693 shutdown_connections 2023-07-13T22:19:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.862+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb74d0 0x55eed5fb78e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.862+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb8c70 0x55eed5fb90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.862+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/253367693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fb86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.862+0000 7f17a8d30b80 1 -- 172.21.15.26:0/253367693 >> 172.21.15.26:0/253367693 conn(0x55eed5facda0 msgr2=0x55eed5fb21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.863+0000 7f17a8d30b80 1 -- 172.21.15.26:0/253367693 shutdown_connections 2023-07-13T22:19:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.863+0000 7f17a8d30b80 1 -- 172.21.15.26:0/253367693 wait complete. 2023-07-13T22:19:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.864+0000 7f17a8d30b80 1 -- start start 2023-07-13T22:19:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.865+0000 7f17a8d30b80 1 Processor -- start 2023-07-13T22:19:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.865+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5ea8ac0 con 0x55eed5fb7e20 2023-07-13T22:19:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5ea87c0 con 0x55eed5fb74d0 2023-07-13T22:19:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5d8c920 con 0x55eed5fb8c70 2023-07-13T22:19:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.866+0000 7f17a8d30b80 10 client.0 _ll_register_callbacks cb 0x55eed5ea6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.868+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.868+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43372/0 (socket says 172.21.15.26:43372) 2023-07-13T22:19:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.868+0000 7f178e7fc700 1 -- 172.21.15.26:0/116392486 learned_addr learned my addr 172.21.15.26:0/116392486 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 0x7f17780a3800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f178400afd0 tx=0x7f178400a3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 -- 172.21.15.26:0/116392486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 msgr2=0x55eed5fd9770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd9770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 -- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 msgr2=0x55eed5fd21f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd21f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.869+0000 7f178e7fc700 1 -- 172.21.15.26:0/116392486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178000bdb0 con 0x55eed5fb7e20 2023-07-13T22:19:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.870+0000 7f178e7fc700 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f178401bc00 tx=0x7f1784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b4c98791665987 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.873+0000 7f1772ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:19:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.873+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784020070 con 0x55eed5fb7e20 2023-07-13T22:19:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.873+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f178400e8f0 con 0x55eed5fb7e20 2023-07-13T22:19:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.873+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eed5e624f0 con 0x55eed5fb7e20 2023-07-13T22:19:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.874+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f178400eab0 con 0x55eed5fb7e20 2023-07-13T22:19:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.874+0000 7f17a8d30b80 10 client.16017 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.874+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eed5d1fc60 con 0x55eed5fb7e20 2023-07-13T22:19:21.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eed5e624f0 con 0x55eed5fb7e20 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f178402e070 con 0x55eed5fb7e20 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 20 client.16017 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed5fddfd0 0x55eed5fe03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 4 client.16017 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:21.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.875+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eed5f01eb0 con 0x55eed5fddfd0 2023-07-13T22:19:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.876+0000 7f178effd700 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed5fddfd0 0x55eed5fe03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.878+0000 7f178effd700 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed5fddfd0 0x55eed5fe03e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e704648206b7946 server_cookie=1cabec174382a45b in_seq=0 out_seq=0 2023-07-13T22:19:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.878+0000 7f1772ff5700 10 client.16017 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.878+0000 7f1772ff5700 1 -- 172.21.15.26:0/116392486 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eed5f01eb0 con 0x55eed5fddfd0 2023-07-13T22:19:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f1772ff5700 10 client.16017 handle_command_reply: tid=0 2023-07-13T22:19:21.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 client.16017 shutdown 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 msgr2=0x55eed5fd2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f178401bc00 tx=0x7f1784004750 comp rx=0 tx=0).stop 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 shutdown_connections 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed5fddfd0 0x55eed5fe03e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/116392486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd2730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.879+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 >> 172.21.15.26:0/116392486 conn(0x55eed5facda0 msgr2=0x55eed5fb18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.880+0000 7f1772ff5700 0 client.16017 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.880+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 shutdown_connections 2023-07-13T22:19:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.880+0000 7f17a8d30b80 1 -- 172.21.15.26:0/116392486 wait complete. 2023-07-13T22:19:21.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.881+0000 7f17a8d30b80 20 client.16017 trim_cache size 0 max 16384 2023-07-13T22:19:21.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.900+0000 7f17a8d30b80 1 Processor -- start 2023-07-13T22:19:21.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.900+0000 7f17a8d30b80 1 -- start start 2023-07-13T22:19:21.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fd9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb8c70 0x55eed5fbbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fb34c0 con 0x55eed5fb8c70 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fd16a0 con 0x55eed5fb74d0 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fb2ed0 con 0x55eed5fb7e20 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f178effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f178effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:36176/0 (socket says 172.21.15.26:36176) 2023-07-13T22:19:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.901+0000 7f178effd700 1 -- 172.21.15.26:0/2281146698 learned_addr learned my addr 172.21.15.26:0/2281146698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178e7fc700 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fd9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178f7fe700 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb8c70 0x55eed5fbbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 -- 172.21.15.26:0/2281146698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 msgr2=0x55eed5fd9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fd9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 -- 172.21.15.26:0/2281146698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb8c70 msgr2=0x55eed5fbbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb8c70 0x55eed5fbbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 -- 172.21.15.26:0/2281146698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eed5d5d250 con 0x55eed5fb74d0 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.902+0000 7f178effd700 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a6cfaf127ebe422 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.903+0000 7f1775ffb700 1 -- 172.21.15.26:0/2281146698 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780020040 con 0x55eed5fb74d0 2023-07-13T22:19:21.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.903+0000 7f1775ffb700 1 -- 172.21.15.26:0/2281146698 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f178002c020 con 0x55eed5fb74d0 2023-07-13T22:19:21.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.903+0000 7f1775ffb700 1 -- 172.21.15.26:0/2281146698 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800283f0 con 0x55eed5fb74d0 2023-07-13T22:19:21.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 -- 172.21.15.26:0/2281146698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 msgr2=0x55eed5fbf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1780000c00 tx=0x7f1780005da0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 -- 172.21.15.26:0/2281146698 shutdown_connections 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb8c70 0x55eed5fbbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fbf030 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/2281146698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb7e20 0x55eed5fd9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 -- 172.21.15.26:0/2281146698 >> 172.21.15.26:0/2281146698 conn(0x55eed5facda0 msgr2=0x55eed5fdffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 -- 172.21.15.26:0/2281146698 shutdown_connections 2023-07-13T22:19:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.904+0000 7f17a8d30b80 1 -- 172.21.15.26:0/2281146698 wait complete. 2023-07-13T22:19:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.905+0000 7f17a8d30b80 1 -- start start 2023-07-13T22:19:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.905+0000 7f17a8d30b80 1 Processor -- start 2023-07-13T22:19:21.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.905+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.905+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.905+0000 7f17a8d30b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fd9330 con 0x55eed5fb7e20 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fd94a0 con 0x55eed5fb74d0 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f17a8d30b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eed5fd9660 con 0x55eed5fb8c70 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f17a8d30b80 10 client.0 _ll_register_callbacks cb 0x7f178401b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f178effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43384/0 (socket says 172.21.15.26:43384) 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f178e7fc700 1 -- 172.21.15.26:0/392547611 learned_addr learned my addr 172.21.15.26:0/392547611 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:21.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.906+0000 7f178f7fe700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 -- 172.21.15.26:0/392547611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 msgr2=0x55eed5fd8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 -- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 msgr2=0x55eed5fd5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 -- 172.21.15.26:0/392547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eed5d36de0 con 0x55eed5fb7e20 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f178e7fc700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f178400a660 tx=0x7f178400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ea1dd320f04542 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:21.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.907+0000 7f1772ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:19:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784020070 con 0x55eed5fb7e20 2023-07-13T22:19:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f17840049d0 con 0x55eed5fb7e20 2023-07-13T22:19:21.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1780015070 con 0x55eed5fb7e20 2023-07-13T22:19:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1784004b90 con 0x55eed5fb7e20 2023-07-13T22:19:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f17a8d30b80 10 client.16023 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:21.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eed5d352b0 con 0x55eed5fb7e20 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.908+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1780015070 con 0x55eed5fb7e20 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f178401bce0 con 0x55eed5fb7e20 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 20 client.16023 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed60331b0 0x55eed60355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:21.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 4 client.16023 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eed5ec4050 con 0x55eed60331b0 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.909+0000 7f178effd700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed60331b0 0x55eed60355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f178effd700 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed60331b0 0x55eed60355c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5e3bdb49e2158f server_cookie=6b2d55ea2e1349d2 in_seq=0 out_seq=0 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f1772ff5700 10 client.16023 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f1772ff5700 1 -- 172.21.15.26:0/392547611 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eed5ec4050 con 0x55eed60331b0 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f1772ff5700 10 client.16023 handle_command_reply: tid=0 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f17a8d30b80 1 client.16023 shutdown 2023-07-13T22:19:21.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 msgr2=0x55eed5fd5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.910+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f178400a660 tx=0x7f178400ef70 comp rx=0 tx=0).stop 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 shutdown_connections 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eed60331b0 0x55eed60355c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eed5fb8c70 0x55eed5fd8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eed5fb74d0 0x55eed5fd5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 --2- 172.21.15.26:0/392547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eed5fb7e20 0x55eed5fd5dd0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f1772ff5700 0 client.16023 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:21.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 >> 172.21.15.26:0/392547611 conn(0x55eed5facda0 msgr2=0x55eed5fbf030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 shutdown_connections 2023-07-13T22:19:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 1 -- 172.21.15.26:0/392547611 wait complete. 2023-07-13T22:19:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.911+0000 7f17a8d30b80 20 client.16023 trim_cache size 0 max 16384 2023-07-13T22:19:21.901 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:19:21.901 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: "9673ada1-c9f9-4612-8d78-1c9378a3b054": { 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "9673ada1-c9f9-4612-8d78-1c9378a3b054", 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:19:21.902 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:19:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 msgr2=0x7f17780a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 0x7f17780a3800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f178400afd0 tx=0x7f178400a3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 msgr2=0x7f17901abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f17901abb70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f178800b770 tx=0x7f1788007730 comp rx=0 tx=0).stop 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 shutdown_connections 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17780a1350 0x7f17780a3800 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f179010bd60 0x7f17901abb70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1790070ee0 0x7f17901a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 --2- 172.21.15.26:0/1666085067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1790071830 0x7f17901a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.912+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 >> 172.21.15.26:0/1666085067 conn(0x7f179006ca90 msgr2=0x7f179006dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.913+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 shutdown_connections 2023-07-13T22:19:21.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:21.913+0000 7f17953ff700 1 -- 172.21.15.26:0/1666085067 wait complete. 2023-07-13T22:19:21.921 INFO:tasks.cephfs.filesystem:scrub status for tag:9673ada1-c9f9-4612-8d78-1c9378a3b054 - {'path': '/', 'tag': '9673ada1-c9f9-4612-8d78-1c9378a3b054', 'options': 'recursive,force'} 2023-07-13T22:19:21.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:21 smithi117 ceph-mon[129074]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:19:21.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:21 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:19:21.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:21 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:21.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:21 smithi121 ceph-mon[129187]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:19:21.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:21 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:19:21.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:21 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:22.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:22 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:19:22.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:22 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:22.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:22 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:19:22.991 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:22 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:23.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:22 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:19:23.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:22 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:23.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:22 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:19:23.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:22 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:23.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:22 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:19:23.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:22 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:23.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:22 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:19:23.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:22 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:19:24.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:23 smithi117 ceph-mon[129074]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:19:24.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:23 smithi026 ceph-mon[95647]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:19:24.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:23 smithi121 ceph-mon[129187]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:19:25.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:24 smithi117 ceph-mon[129074]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:19:25.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:24 smithi026 ceph-mon[95647]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:19:25.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:24 smithi121 ceph-mon[129187]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:19:27.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:27 smithi117 ceph-mon[129074]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:19:27.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:27 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:19:27.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:27 smithi026 ceph-mon[95647]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:19:27.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:27 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:19:27.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:27 smithi121 ceph-mon[129187]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 274 MiB/s wr, 89 op/s 2023-07-13T22:19:27.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:27 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:19:28.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:28 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:19:28.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:28 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:19:28.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:28 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:19:29.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:29 smithi117 ceph-mon[129074]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 97 op/s 2023-07-13T22:19:29.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:29 smithi026 ceph-mon[95647]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 97 op/s 2023-07-13T22:19:29.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:29 smithi121 ceph-mon[129187]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 290 MiB/s wr, 97 op/s 2023-07-13T22:19:31.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:31 smithi117 ceph-mon[129074]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:19:31.452 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:31 smithi026 ceph-mon[95647]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:19:31.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:31 smithi121 ceph-mon[129187]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:19:33.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:33 smithi117 ceph-mon[129074]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 MiB/s wr, 91 op/s 2023-07-13T22:19:33.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:33 smithi026 ceph-mon[95647]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 MiB/s wr, 91 op/s 2023-07-13T22:19:33.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:33 smithi121 ceph-mon[129187]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 278 MiB/s wr, 91 op/s 2023-07-13T22:19:35.440 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:35 smithi117 ceph-mon[129074]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 319 MiB/s wr, 102 op/s 2023-07-13T22:19:35.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:35 smithi026 ceph-mon[95647]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 319 MiB/s wr, 102 op/s 2023-07-13T22:19:35.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:35 smithi121 ceph-mon[129187]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 319 MiB/s wr, 102 op/s 2023-07-13T22:19:37.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:37 smithi117 ceph-mon[129074]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:37.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:37 smithi026 ceph-mon[95647]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:37.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:37 smithi121 ceph-mon[129187]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:39.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:39 smithi117 ceph-mon[129074]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 94 op/s 2023-07-13T22:19:39.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:39 smithi026 ceph-mon[95647]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 94 op/s 2023-07-13T22:19:39.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:39 smithi121 ceph-mon[129187]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 287 MiB/s wr, 94 op/s 2023-07-13T22:19:41.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:41 smithi117 ceph-mon[129074]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:19:41.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:41 smithi026 ceph-mon[95647]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:19:41.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:41 smithi121 ceph-mon[129187]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 92 op/s 2023-07-13T22:19:42.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:42 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:19:42.707 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:42 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:19:42.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:42 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:19:43.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:43 smithi117 ceph-mon[129074]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:19:43.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:43 smithi026 ceph-mon[95647]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:19:43.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:43 smithi121 ceph-mon[129187]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:19:44.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:19:44.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:19:44.690 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:44 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:19:44.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:19:44.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:19:44.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:44 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:19:44.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:19:44.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:19:44.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:44 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:19:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:45 smithi117 ceph-mon[129074]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:19:45.715 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:45 smithi026 ceph-mon[95647]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:19:45.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:45 smithi121 ceph-mon[129187]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:19:47.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:47 smithi117 ceph-mon[129074]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 82 op/s 2023-07-13T22:19:47.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:47 smithi026 ceph-mon[95647]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 82 op/s 2023-07-13T22:19:47.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:47 smithi121 ceph-mon[129187]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 262 MiB/s wr, 82 op/s 2023-07-13T22:19:49.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:49 smithi117 ceph-mon[129074]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:49.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:49 smithi026 ceph-mon[95647]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:49.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:49 smithi121 ceph-mon[129187]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:19:51.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:51 smithi121 ceph-mon[129187]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-13T22:19:51.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:19:51.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:51 smithi117 ceph-mon[129074]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-13T22:19:51.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:51 smithi026 ceph-mon[95647]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 85 op/s 2023-07-13T22:19:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.416+0000 7f6714fff700 1 -- 172.21.15.26:0/3219621194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710070ee0 msgr2=0x7f67100712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.416+0000 7f6714fff700 1 --2- 172.21.15.26:0/3219621194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710070ee0 0x7f67100712f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009990 tx=0x7f66f8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 -- 172.21.15.26:0/3219621194 shutdown_connections 2023-07-13T22:19:52.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 --2- 172.21.15.26:0/3219621194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f671010bd60 0x7f671010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 --2- 172.21.15.26:0/3219621194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6710070ee0 0x7f67100712f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 --2- 172.21.15.26:0/3219621194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 0x7f671010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 -- 172.21.15.26:0/3219621194 >> 172.21.15.26:0/3219621194 conn(0x7f671006ca90 msgr2=0x7f671006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 -- 172.21.15.26:0/3219621194 shutdown_connections 2023-07-13T22:19:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.422+0000 7f6714fff700 1 -- 172.21.15.26:0/3219621194 wait complete. 2023-07-13T22:19:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.424+0000 7f6714fff700 1 Processor -- start 2023-07-13T22:19:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.424+0000 7f6714fff700 1 -- start start 2023-07-13T22:19:52.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6710070ee0 0x7f67101a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 0x7f67101a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101ac260 con 0x7f6710071830 2023-07-13T22:19:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101ac3a0 con 0x7f671010bd60 2023-07-13T22:19:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f6714fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67101ac530 con 0x7f6710070ee0 2023-07-13T22:19:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f670ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.425+0000 7f670ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39398/0 (socket says 172.21.15.26:39398) 2023-07-13T22:19:52.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.426+0000 7f670ed9d700 1 -- 172.21.15.26:0/2535340951 learned_addr learned my addr 172.21.15.26:0/2535340951 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.452+0000 7f670e59c700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6710070ee0 0x7f67101a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.453+0000 7f6707fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 0x7f67101a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.455+0000 7f670ed9d700 1 -- 172.21.15.26:0/2535340951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6710070ee0 msgr2=0x7f67101a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.455+0000 7f670ed9d700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6710070ee0 0x7f67101a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.455+0000 7f670ed9d700 1 -- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 msgr2=0x7f67101a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.455+0000 7f670ed9d700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 0x7f67101a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.455+0000 7f670ed9d700 1 -- 172.21.15.26:0/2535340951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80095f0 con 0x7f671010bd60 2023-07-13T22:19:52.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.472+0000 7f670ed9d700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6708007ee0 tx=0x7f670800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.489+0000 7f67077fe700 1 -- 172.21.15.26:0/2535340951 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708017070 con 0x7f671010bd60 2023-07-13T22:19:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.489+0000 7f67077fe700 1 -- 172.21.15.26:0/2535340951 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670800f040 con 0x7f671010bd60 2023-07-13T22:19:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.489+0000 7f67077fe700 1 -- 172.21.15.26:0/2535340951 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6708005740 con 0x7f671010bd60 2023-07-13T22:19:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.489+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67101ac810 con 0x7f671010bd60 2023-07-13T22:19:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.490+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67101acd50 con 0x7f671010bd60 2023-07-13T22:19:52.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.507+0000 7f6728884b80 1 Processor -- start 2023-07-13T22:19:52.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 -- start start 2023-07-13T22:19:52.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d71e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d71f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d622cb0 con 0x55773d71de20 2023-07-13T22:19:52.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d675a40 con 0x55773d71ec70 2023-07-13T22:19:52.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.535+0000 7f6728884b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d66eed0 con 0x55773d71d4d0 2023-07-13T22:19:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.540+0000 7f67077fe700 1 -- 172.21.15.26:0/2535340951 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f67080058a0 con 0x7f671010bd60 2023-07-13T22:19:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.542+0000 7f67077fe700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 0x7f66fc0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f67077fe700 1 -- 172.21.15.26:0/2535340951 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67080cb620 con 0x7f671010bd60 2023-07-13T22:19:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f670ed9d700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 0x7f66fc0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f6707fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f6707fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40946/0 (socket says 172.21.15.26:40946) 2023-07-13T22:19:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f6707fff700 1 -- 172.21.15.26:0/2084397788 learned_addr learned my addr 172.21.15.26:0/2084397788 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.543+0000 7f6707fff700 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d71f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.544+0000 7f670ed9d700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 0x7f66fc0a3780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f67080979a0 tx=0x7f670800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 -- 172.21.15.26:0/2084397788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 msgr2=0x55773d71f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d71f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 -- 172.21.15.26:0/2084397788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 msgr2=0x55773d71e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d71e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 -- 172.21.15.26:0/2084397788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773d49cde0 con 0x55773d71d4d0 2023-07-13T22:19:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.566+0000 7f6707fff700 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6700016580 tx=0x7f670000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5288cfa0ddccc2cc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.595+0000 7f6704ff9700 1 -- 172.21.15.26:0/2084397788 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700016ba0 con 0x55773d71d4d0 2023-07-13T22:19:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.595+0000 7f6704ff9700 1 -- 172.21.15.26:0/2084397788 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670000c070 con 0x55773d71d4d0 2023-07-13T22:19:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.595+0000 7f6704ff9700 1 -- 172.21.15.26:0/2084397788 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700012390 con 0x55773d71d4d0 2023-07-13T22:19:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 -- 172.21.15.26:0/2084397788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 msgr2=0x55773d71d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6700016580 tx=0x7f670000a340 comp rx=0 tx=0).stop 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 -- 172.21.15.26:0/2084397788 shutdown_connections 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d71f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d71d8e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 --2- 172.21.15.26:0/2084397788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d71e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 -- 172.21.15.26:0/2084397788 >> 172.21.15.26:0/2084397788 conn(0x55773d712da0 msgr2=0x55773d7181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.597+0000 7f6728884b80 1 -- 172.21.15.26:0/2084397788 shutdown_connections 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.598+0000 7f6728884b80 1 -- 172.21.15.26:0/2084397788 wait complete. 2023-07-13T22:19:52.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.598+0000 7f6728884b80 1 -- start start 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.599+0000 7f6728884b80 1 Processor -- start 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71d4d0 0x55773d727200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71ec70 0x55773d72e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d60eac0 con 0x55773d71ec70 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d60e7c0 con 0x55773d71d4d0 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d4f2920 con 0x55773d71de20 2023-07-13T22:19:52.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.600+0000 7f6728884b80 10 client.0 _ll_register_callbacks cb 0x55773d60c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.601+0000 7f670e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.601+0000 7f670e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40960/0 (socket says 172.21.15.26:40960) 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.601+0000 7f670e59c700 1 -- 172.21.15.26:0/3973434742 learned_addr learned my addr 172.21.15.26:0/3973434742 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.601+0000 7f6707fff700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71d4d0 0x55773d727200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 -- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71d4d0 msgr2=0x55773d727200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71d4d0 0x55773d727200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 -- 172.21.15.26:0/3973434742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71ec70 msgr2=0x55773d72e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71ec70 0x55773d72e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 -- 172.21.15.26:0/3973434742 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670000e040 con 0x55773d71de20 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.610+0000 7f670e59c700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f800bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=736309ce08a6fe26 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.619+0000 7f66f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.619+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f801d030 con 0x55773d71de20 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.619+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f66f80050c0 con 0x55773d71de20 2023-07-13T22:19:52.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.619+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55773d5c84f0 con 0x55773d71de20 2023-07-13T22:19:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.630+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f8003e60 con 0x55773d71de20 2023-07-13T22:19:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.630+0000 7f6728884b80 10 client.26332 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.630+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55773d485c60 con 0x55773d71de20 2023-07-13T22:19:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.664+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55773d5c84f0 con 0x55773d71de20 2023-07-13T22:19:52.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.664+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f66f802b070 con 0x55773d71de20 2023-07-13T22:19:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 10 client.26332 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 20 client.26332 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d732fb0 0x55773d7353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 4 client.26332 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:52.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.665+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55773d667eb0 con 0x55773d732fb0 2023-07-13T22:19:52.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.676+0000 7f6707fff700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d732fb0 0x55773d7353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.681+0000 7f6707fff700 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d732fb0 0x55773d7353c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9e159696be89709 server_cookie=7c34077286db863e in_seq=0 out_seq=0 2023-07-13T22:19:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.681+0000 7f66f5ffb700 10 client.26332 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.683+0000 7f66f5ffb700 1 -- 172.21.15.26:0/3973434742 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55773d667eb0 con 0x55773d732fb0 2023-07-13T22:19:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.683+0000 7f66f5ffb700 10 client.26332 handle_command_reply: tid=0 2023-07-13T22:19:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.683+0000 7f6728884b80 1 client.26332 shutdown 2023-07-13T22:19:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.683+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 msgr2=0x55773d727740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.683+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f800bc00 comp rx=0 tx=0).stop 2023-07-13T22:19:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 shutdown_connections 2023-07-13T22:19:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71ec70 0x55773d72e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d732fb0 0x55773d7353c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71d4d0 0x55773d727200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 --2- 172.21.15.26:0/3973434742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71de20 0x55773d727740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.701+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 >> 172.21.15.26:0/3973434742 conn(0x55773d712da0 msgr2=0x55773d7178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.706+0000 7f66f5ffb700 0 client.26332 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:52.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.707+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 shutdown_connections 2023-07-13T22:19:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.713+0000 7f6728884b80 1 -- 172.21.15.26:0/3973434742 wait complete. 2023-07-13T22:19:52.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.717+0000 7f6728884b80 20 client.26332 trim_cache size 0 max 16384 2023-07-13T22:19:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 Processor -- start 2023-07-13T22:19:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 -- start start 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d725e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d727200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d726d70 con 0x55773d71de20 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d719d00 con 0x55773d71ec70 2023-07-13T22:19:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.749+0000 7f6728884b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d712970 con 0x55773d71d4d0 2023-07-13T22:19:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.750+0000 7f670e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.750+0000 7f670e59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42306/0 (socket says 172.21.15.26:42306) 2023-07-13T22:19:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.750+0000 7f670e59c700 1 -- 172.21.15.26:0/2165951060 learned_addr learned my addr 172.21.15.26:0/2165951060 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.750+0000 7f6707fff700 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d727200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.750+0000 7f6707fff700 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d725e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.751+0000 7f6707fff700 1 -- 172.21.15.26:0/2165951060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 msgr2=0x55773d725e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.751+0000 7f6707fff700 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d725e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.751+0000 7f6707fff700 1 -- 172.21.15.26:0/2165951060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 msgr2=0x55773d72e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.751+0000 7f6707fff700 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.751+0000 7f6707fff700 1 -- 172.21.15.26:0/2165951060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773d71f310 con 0x55773d71ec70 2023-07-13T22:19:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.752+0000 7f6707fff700 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d727200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55773d735ca0 tx=0x7f670001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a2894aaf3f830c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.752+0000 7f67057fa700 1 -- 172.21.15.26:0/2165951060 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670002ad80 con 0x55773d71ec70 2023-07-13T22:19:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.752+0000 7f67057fa700 1 -- 172.21.15.26:0/2165951060 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6700030030 con 0x55773d71ec70 2023-07-13T22:19:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.752+0000 7f67057fa700 1 -- 172.21.15.26:0/2165951060 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67000281a0 con 0x55773d71ec70 2023-07-13T22:19:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.753+0000 7f6728884b80 1 -- 172.21.15.26:0/2165951060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 msgr2=0x55773d727200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.753+0000 7f6728884b80 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d727200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55773d735ca0 tx=0x7f670001a040 comp rx=0 tx=0).stop 2023-07-13T22:19:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 -- 172.21.15.26:0/2165951060 shutdown_connections 2023-07-13T22:19:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d727200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d725e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 --2- 172.21.15.26:0/2165951060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 -- 172.21.15.26:0/2165951060 >> 172.21.15.26:0/2165951060 conn(0x55773d712da0 msgr2=0x55773d734fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 -- 172.21.15.26:0/2165951060 shutdown_connections 2023-07-13T22:19:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.754+0000 7f6728884b80 1 -- 172.21.15.26:0/2165951060 wait complete. 2023-07-13T22:19:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.755+0000 7f6728884b80 1 -- start start 2023-07-13T22:19:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.755+0000 7f6728884b80 1 Processor -- start 2023-07-13T22:19:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d72da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d72df90 con 0x55773d71de20 2023-07-13T22:19:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d72e100 con 0x55773d71ec70 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773d72e2c0 con 0x55773d71d4d0 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6728884b80 10 client.0 _ll_register_callbacks cb 0x7f66f800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6707fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.756+0000 7f6707fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40972/0 (socket says 172.21.15.26:40972) 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 -- 172.21.15.26:0/2707411001 learned_addr learned my addr 172.21.15.26:0/2707411001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d72da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 -- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 msgr2=0x55773d72da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d72da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 -- 172.21.15.26:0/2707411001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 msgr2=0x55773d72ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.757+0000 7f6707fff700 1 -- 172.21.15.26:0/2707411001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670001c040 con 0x55773d71d4d0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.758+0000 7f6707fff700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6700006ea0 tx=0x7f670000a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a610fd48b307ff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.758+0000 7f66f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.758+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700016c30 con 0x55773d71d4d0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.758+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6700000e40 con 0x55773d71d4d0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.759+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f670000c070 con 0x55773d71d4d0 2023-07-13T22:19:52.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.759+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6700016d90 con 0x55773d71d4d0 2023-07-13T22:19:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.759+0000 7f6728884b80 10 client.26338 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.759+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55773d4c3250 con 0x55773d71d4d0 2023-07-13T22:19:52.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.759+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f670000c070 con 0x55773d71d4d0 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f670002a480 con 0x55773d71d4d0 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 10 client.26338 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 10 client.26338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 20 client.26338 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d787a10 0x55773d789e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 4 client.26338 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:19:52.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55773d62a050 con 0x55773d787a10 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.760+0000 7f670e59c700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d787a10 0x55773d789e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.761+0000 7f670e59c700 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d787a10 0x55773d789e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de20fe2517bec77 server_cookie=4f992ddde4c1346b in_seq=0 out_seq=0 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.761+0000 7f66f5ffb700 10 client.26338 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.761+0000 7f66f5ffb700 1 -- 172.21.15.26:0/2707411001 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55773d62a050 con 0x55773d787a10 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.761+0000 7f66f5ffb700 10 client.26338 handle_command_reply: tid=0 2023-07-13T22:19:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.761+0000 7f6728884b80 1 client.26338 shutdown 2023-07-13T22:19:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 msgr2=0x55773d72c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6700006ea0 tx=0x7f670000a750 comp rx=0 tx=0).stop 2023-07-13T22:19:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 shutdown_connections 2023-07-13T22:19:52.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55773d787a10 0x55773d789e20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55773d71ec70 0x55773d72da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55773d71d4d0 0x55773d72c500 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 --2- 172.21.15.26:0/2707411001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55773d71de20 0x55773d72ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 >> 172.21.15.26:0/2707411001 conn(0x55773d712da0 msgr2=0x55773d725e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.762+0000 7f66f5ffb700 0 client.26338 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.763+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 shutdown_connections 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.763+0000 7f6728884b80 1 -- 172.21.15.26:0/2707411001 wait complete. 2023-07-13T22:19:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.763+0000 7f6728884b80 20 client.26338 trim_cache size 0 max 16384 2023-07-13T22:19:52.752 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:19:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:19:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:19:52.752 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:19:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 msgr2=0x7f66fc0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 0x7f66fc0a3780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f67080979a0 tx=0x7f670800b040 comp rx=0 tx=0).stop 2023-07-13T22:19:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 msgr2=0x7f67101abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:52.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6708007ee0 tx=0x7f670800d3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 shutdown_connections 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f671010bd60 0x7f67101abc90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6710070ee0 0x7f67101a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66fc0a12d0 0x7f66fc0a3780 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 --2- 172.21.15.26:0/2535340951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6710071830 0x7f67101a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.764+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 >> 172.21.15.26:0/2535340951 conn(0x7f671006ca90 msgr2=0x7f671006dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.765+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 shutdown_connections 2023-07-13T22:19:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:52.765+0000 7f6714fff700 1 -- 172.21.15.26:0/2535340951 wait complete. 2023-07-13T22:19:52.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:19:52.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:19:53.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.348+0000 7f6f999ff700 1 -- 172.21.15.26:0/424226340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94071820 msgr2=0x7f6f9410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:53.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.348+0000 7f6f999ff700 1 --2- 172.21.15.26:0/424226340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94071820 0x7f6f9410b7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009990 tx=0x7f6f88009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 -- 172.21.15.26:0/424226340 shutdown_connections 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 --2- 172.21.15.26:0/424226340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f9410bd60 0x7f6f9410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 --2- 172.21.15.26:0/424226340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f94070ed0 0x7f6f940712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 --2- 172.21.15.26:0/424226340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94071820 0x7f6f9410b7d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 -- 172.21.15.26:0/424226340 >> 172.21.15.26:0/424226340 conn(0x7f6f9406ca40 msgr2=0x7f6f9406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 -- 172.21.15.26:0/424226340 shutdown_connections 2023-07-13T22:19:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.349+0000 7f6f999ff700 1 -- 172.21.15.26:0/424226340 wait complete. 2023-07-13T22:19:53.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.351+0000 7f6f999ff700 1 Processor -- start 2023-07-13T22:19:53.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 -- start start 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94070ed0 0x7f6f94114c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f9410bd60 0x7f6f94116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94116750 con 0x7f6f94070ed0 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f941168c0 con 0x7f6f9410bd60 2023-07-13T22:19:53.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.352+0000 7f6f999ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f94116a80 con 0x7f6f94071820 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.353+0000 7f6f927fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.353+0000 7f6f927fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50946/0 (socket says 172.21.15.26:50946) 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.353+0000 7f6f927fc700 1 -- 172.21.15.26:0/3309901744 learned_addr learned my addr 172.21.15.26:0/3309901744 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.353+0000 7f6f927fc700 1 -- 172.21.15.26:0/3309901744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f9410bd60 msgr2=0x7f6f94116180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.353+0000 7f6f927fc700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f9410bd60 0x7f6f94116180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.354+0000 7f6f927fc700 1 -- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94070ed0 msgr2=0x7f6f94114c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:53.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.354+0000 7f6f927fc700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94070ed0 0x7f6f94114c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:53.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.354+0000 7f6f927fc700 1 -- 172.21.15.26:0/3309901744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880095f0 con 0x7f6f94071820 2023-07-13T22:19:53.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.354+0000 7f6f927fc700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800f670 tx=0x7f6f8800f750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:53.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.354+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8801d070 con 0x7f6f94071820 2023-07-13T22:19:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.355+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f8800fbf0 con 0x7f6f94071820 2023-07-13T22:19:53.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.355+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88017650 con 0x7f6f94071820 2023-07-13T22:19:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.356+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f94116ca0 con 0x7f6f94071820 2023-07-13T22:19:53.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.356+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f941170d0 con 0x7f6f94071820 2023-07-13T22:19:53.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.358+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9410f3c0 con 0x7f6f94071820 2023-07-13T22:19:53.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.363+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6f88005080 con 0x7f6f94071820 2023-07-13T22:19:53.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.366+0000 7f6f989fd700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 0x7f6f7c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.366+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6f880cde20 con 0x7f6f94071820 2023-07-13T22:19:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.366+0000 7f6f92ffd700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 0x7f6f7c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:53.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.366+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6f880ce260 con 0x7f6f94071820 2023-07-13T22:19:53.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:53.367+0000 7f6f92ffd700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 0x7f6f7c0a37f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00b420 tx=0x7f6f8c008540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:53.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:53 smithi117 ceph-mon[129074]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 85 op/s 2023-07-13T22:19:53.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:53 smithi026 ceph-mon[95647]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 85 op/s 2023-07-13T22:19:53.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:53 smithi121 ceph-mon[129187]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 269 MiB/s wr, 85 op/s 2023-07-13T22:19:54.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.117+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f9410f5b0 con 0x7f6f94071820 2023-07-13T22:19:54.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.119+0000 7f6f989fd700 1 -- 172.21.15.26:0/3309901744 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f6f88097630 con 0x7f6f94071820 2023-07-13T22:19:54.106 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:19:54.106 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:19:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 msgr2=0x7f6f7c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 0x7f6f7c0a37f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c00b420 tx=0x7f6f8c008540 comp rx=0 tx=0).stop 2023-07-13T22:19:54.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 msgr2=0x7f6f94115140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800f670 tx=0x7f6f8800f750 comp rx=0 tx=0).stop 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 shutdown_connections 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6f7c0a1340 0x7f6f7c0a37f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f94070ed0 0x7f6f94114c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f9410bd60 0x7f6f94116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 --2- 172.21.15.26:0/3309901744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f94071820 0x7f6f94115140 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 >> 172.21.15.26:0/3309901744 conn(0x7f6f9406ca40 msgr2=0x7f6f9406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 shutdown_connections 2023-07-13T22:19:54.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.124+0000 7f6f999ff700 1 -- 172.21.15.26:0/3309901744 wait complete. 2023-07-13T22:19:54.112 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:19:54.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:19:54.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.799+0000 7fa6083ff700 1 -- 172.21.15.26:0/27866748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa60010bd60 msgr2=0x7fa60010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.799+0000 7fa6083ff700 1 --2- 172.21.15.26:0/27866748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa60010bd60 0x7fa60010e1f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b0a0 tx=0x7fa5f800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.801+0000 7fa6083ff700 1 -- 172.21.15.26:0/27866748 shutdown_connections 2023-07-13T22:19:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.801+0000 7fa6083ff700 1 --2- 172.21.15.26:0/27866748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa60010bd60 0x7fa60010e1f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.801+0000 7fa6083ff700 1 --2- 172.21.15.26:0/27866748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa600070ee0 0x7fa6000712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.801+0000 7fa6083ff700 1 --2- 172.21.15.26:0/27866748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa600071830 0x7fa60010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.801+0000 7fa6083ff700 1 -- 172.21.15.26:0/27866748 >> 172.21.15.26:0/27866748 conn(0x7fa60006ca90 msgr2=0x7fa60006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.802+0000 7fa6083ff700 1 -- 172.21.15.26:0/27866748 shutdown_connections 2023-07-13T22:19:54.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.803+0000 7fa6083ff700 1 -- 172.21.15.26:0/27866748 wait complete. 2023-07-13T22:19:54.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.804+0000 7fa6083ff700 1 Processor -- start 2023-07-13T22:19:54.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 -- start start 2023-07-13T22:19:54.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa600070ee0 0x7fa600113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60010bd60 0x7fa6001140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600114670 con 0x7fa600070ee0 2023-07-13T22:19:54.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001147e0 con 0x7fa6001b2970 2023-07-13T22:19:54.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.805+0000 7fa6083ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001b5290 con 0x7fa60010bd60 2023-07-13T22:19:54.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.807+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.807+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44782/0 (socket says 172.21.15.26:44782) 2023-07-13T22:19:54.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.807+0000 7fa60699c700 1 -- 172.21.15.26:0/597426974 learned_addr learned my addr 172.21.15.26:0/597426974 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:54.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.807+0000 7fa60599a700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60010bd60 0x7fa6001140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 -- 172.21.15.26:0/597426974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60010bd60 msgr2=0x7fa6001140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60010bd60 0x7fa6001140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 -- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa600070ee0 msgr2=0x7fa600113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa600070ee0 0x7fa600113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 -- 172.21.15.26:0/597426974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009d00 con 0x7fa6001b2970 2023-07-13T22:19:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.808+0000 7fa60699c700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b070 tx=0x7fa5f80123f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.809+0000 7fa5f77fe700 1 -- 172.21.15.26:0/597426974 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800e040 con 0x7fa6001b2970 2023-07-13T22:19:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.809+0000 7fa5f77fe700 1 -- 172.21.15.26:0/597426974 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f801d480 con 0x7fa6001b2970 2023-07-13T22:19:54.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.809+0000 7fa5f77fe700 1 -- 172.21.15.26:0/597426974 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f801b690 con 0x7fa6001b2970 2023-07-13T22:19:54.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.809+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001b5510 con 0x7fa6001b2970 2023-07-13T22:19:54.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.809+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001b59c0 con 0x7fa6001b2970 2023-07-13T22:19:54.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.819+0000 7fa5f77fe700 1 -- 172.21.15.26:0/597426974 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa5f801d5f0 con 0x7fa6001b2970 2023-07-13T22:19:54.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.821+0000 7fa61bcefb80 1 Processor -- start 2023-07-13T22:19:54.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.822+0000 7fa5f77fe700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 0x7fa5ec0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.822+0000 7fa61bcefb80 1 -- start start 2023-07-13T22:19:54.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.822+0000 7fa5f77fe700 1 -- 172.21.15.26:0/597426974 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa5f80ce7b0 con 0x7fa6001b2970 2023-07-13T22:19:54.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.822+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465685c70 0x5584656860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465589cb0 con 0x558465684e20 2023-07-13T22:19:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584655dca40 con 0x5584656844d0 2023-07-13T22:19:54.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584655d5ed0 con 0x558465685c70 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa60619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa60619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44790/0 (socket says 172.21.15.26:44790) 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.823+0000 7fa60619b700 1 -- 172.21.15.26:0/52945456 learned_addr learned my addr 172.21.15.26:0/52945456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60599a700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 0x7fa5ec0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60619b700 1 -- 172.21.15.26:0/52945456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465685c70 msgr2=0x5584656860e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:54.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60619b700 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465685c70 0x5584656860e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60619b700 1 -- 172.21.15.26:0/52945456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 msgr2=0x5584656856e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60619b700 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.824+0000 7fa60619b700 1 -- 172.21.15.26:0/52945456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558465403de0 con 0x5584656844d0 2023-07-13T22:19:54.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.825+0000 7fa60619b700 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009870 tx=0x7fa5f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec1fea9896499dec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.826+0000 7fa5f4bfe700 1 -- 172.21.15.26:0/52945456 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0011830 con 0x5584656844d0 2023-07-13T22:19:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.826+0000 7fa5f4bfe700 1 -- 172.21.15.26:0/52945456 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f0011e50 con 0x5584656844d0 2023-07-13T22:19:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.826+0000 7fa5f4bfe700 1 -- 172.21.15.26:0/52945456 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0010380 con 0x5584656844d0 2023-07-13T22:19:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.827+0000 7fa61bcefb80 1 -- 172.21.15.26:0/52945456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 msgr2=0x5584656848e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.827+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009870 tx=0x7fa5f0009b80 comp rx=0 tx=0).stop 2023-07-13T22:19:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.827+0000 7fa60599a700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 0x7fa5ec0a3800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005fd0 tx=0x7fa5fc005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 -- 172.21.15.26:0/52945456 shutdown_connections 2023-07-13T22:19:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465685c70 0x5584656860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x5584656848e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/52945456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656856e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 -- 172.21.15.26:0/52945456 >> 172.21.15.26:0/52945456 conn(0x558465679da0 msgr2=0x55846567f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 -- 172.21.15.26:0/52945456 shutdown_connections 2023-07-13T22:19:54.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.828+0000 7fa61bcefb80 1 -- 172.21.15.26:0/52945456 wait complete. 2023-07-13T22:19:54.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.829+0000 7fa61bcefb80 1 -- start start 2023-07-13T22:19:54.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.829+0000 7fa61bcefb80 1 Processor -- start 2023-07-13T22:19:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x55846568e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x55846568e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 0x5584656957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465575ac0 con 0x558465685c70 2023-07-13T22:19:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584655757c0 con 0x5584656844d0 2023-07-13T22:19:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.830+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465459920 con 0x558465684e20 2023-07-13T22:19:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.831+0000 7fa61bcefb80 10 client.0 _ll_register_callbacks cb 0x558465573760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x55846568e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60699c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 0x5584656957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60699c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 0x5584656957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52590/0 (socket says 172.21.15.26:52590) 2023-07-13T22:19:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60699c700 1 -- 172.21.15.26:0/2145757800 learned_addr learned my addr 172.21.15.26:0/2145757800 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60619b700 1 -- 172.21.15.26:0/2145757800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 msgr2=0x55846568e7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60619b700 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x55846568e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60619b700 1 -- 172.21.15.26:0/2145757800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 msgr2=0x5584656957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.832+0000 7fa60619b700 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 0x5584656957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.833+0000 7fa60619b700 1 -- 172.21.15.26:0/2145757800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f00094a0 con 0x5584656844d0 2023-07-13T22:19:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.833+0000 7fa60619b700 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x55846568e260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0000c00 tx=0x7fa5f000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622177d5fe00d2c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.834+0000 7fa5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.834+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0011a40 con 0x5584656844d0 2023-07-13T22:19:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.834+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa5f000b690 con 0x5584656844d0 2023-07-13T22:19:54.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.835+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55846552f4f0 con 0x5584656844d0 2023-07-13T22:19:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.835+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f0011ba0 con 0x5584656844d0 2023-07-13T22:19:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.835+0000 7fa61bcefb80 10 client.26369 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.835+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5584653ecc60 con 0x5584656844d0 2023-07-13T22:19:54.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.836+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55846552f4f0 con 0x5584656844d0 2023-07-13T22:19:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.836+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa5f0019430 con 0x5584656844d0 2023-07-13T22:19:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.836+0000 7fa61bcefb80 10 client.26369 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.836+0000 7fa61bcefb80 10 client.26369 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:19:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.836+0000 7fa61bcefb80 20 client.26369 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:54.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.837+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558465699f00 0x55846569c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.837+0000 7fa61bcefb80 4 client.26369 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:54.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.837+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584655ceeb0 con 0x558465699f00 2023-07-13T22:19:54.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.852+0000 7fa60599a700 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558465699f00 0x55846569c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.853+0000 7fa60599a700 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558465699f00 0x55846569c310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a160886c8bb9a796 server_cookie=4616b531e14c523 in_seq=0 out_seq=0 2023-07-13T22:19:54.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.853+0000 7fa5e9ffb700 10 client.26369 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.854+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/2145757800 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5584655ceeb0 con 0x558465699f00 2023-07-13T22:19:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.854+0000 7fa5e9ffb700 10 client.26369 handle_command_reply: tid=0 2023-07-13T22:19:54.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.854+0000 7fa61bcefb80 1 client.26369 shutdown 2023-07-13T22:19:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.854+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 msgr2=0x55846568e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.854+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x55846568e260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0000c00 tx=0x7fa5f000bed0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 shutdown_connections 2023-07-13T22:19:54.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x558465699f00 0x55846569c310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465685c70 0x5584656957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5584656844d0 0x55846568e260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/2145757800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x55846568e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 >> 172.21.15.26:0/2145757800 conn(0x558465679da0 msgr2=0x55846567e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa5e9ffb700 0 client.26369 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:54.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 shutdown_connections 2023-07-13T22:19:54.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.855+0000 7fa61bcefb80 1 -- 172.21.15.26:0/2145757800 wait complete. 2023-07-13T22:19:54.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.859+0000 7fa61bcefb80 20 client.26369 trim_cache size 0 max 16384 2023-07-13T22:19:54.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.888+0000 7fa61bcefb80 1 Processor -- start 2023-07-13T22:19:54.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.888+0000 7fa61bcefb80 1 -- start start 2023-07-13T22:19:54.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584656844d0 0x558465686060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465680ca0 con 0x558465684e20 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465699c50 con 0x558465685c70 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465696610 con 0x5584656844d0 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44814/0 (socket says 172.21.15.26:44814) 2023-07-13T22:19:54.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa60699c700 1 -- 172.21.15.26:0/1094888377 learned_addr learned my addr 172.21.15.26:0/1094888377 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:54.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.889+0000 7fa60619b700 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584656844d0 0x558465686060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.890+0000 7fa60699c700 1 -- 172.21.15.26:0/1094888377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584656844d0 msgr2=0x558465686060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.890+0000 7fa60699c700 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584656844d0 0x558465686060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.890+0000 7fa60699c700 1 -- 172.21.15.26:0/1094888377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 msgr2=0x5584656957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.890+0000 7fa60699c700 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.890+0000 7fa60699c700 1 -- 172.21.15.26:0/1094888377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55846542a250 con 0x558465685c70 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.891+0000 7fa60699c700 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8098670 tx=0x7fa5f8009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7a59a8ed3f9aa7f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.891+0000 7fa5f53ff700 1 -- 172.21.15.26:0/1094888377 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f809bc00 con 0x558465685c70 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.891+0000 7fa5f53ff700 1 -- 172.21.15.26:0/1094888377 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f809fe90 con 0x558465685c70 2023-07-13T22:19:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.891+0000 7fa5f53ff700 1 -- 172.21.15.26:0/1094888377 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80b4220 con 0x558465685c70 2023-07-13T22:19:54.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1094888377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 msgr2=0x5584656960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8098670 tx=0x7fa5f8009040 comp rx=0 tx=0).stop 2023-07-13T22:19:54.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1094888377 shutdown_connections 2023-07-13T22:19:54.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x5584656960d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5584656844d0 0x558465686060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1094888377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558465684e20 0x5584656957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.892+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1094888377 >> 172.21.15.26:0/1094888377 conn(0x558465679da0 msgr2=0x5584656a0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.893+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1094888377 shutdown_connections 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.893+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1094888377 wait complete. 2023-07-13T22:19:54.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.893+0000 7fa61bcefb80 1 -- start start 2023-07-13T22:19:54.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.894+0000 7fa61bcefb80 1 Processor -- start 2023-07-13T22:19:54.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.894+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584656844d0 0x558465691930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.894+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x558465691e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa61bcefb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584656953d0 con 0x5584656844d0 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465695540 con 0x558465685c70 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa61bcefb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465695700 con 0x558465684e20 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa61bcefb80 10 client.0 _ll_register_callbacks cb 0x7fa5f00096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa60599a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x558465691e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa60699c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44822/0 (socket says 172.21.15.26:44822) 2023-07-13T22:19:54.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.895+0000 7fa60699c700 1 -- 172.21.15.26:0/1152344402 learned_addr learned my addr 172.21.15.26:0/1152344402 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:54.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 -- 172.21.15.26:0/1152344402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 msgr2=0x558465691e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x558465691e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 -- 172.21.15.26:0/1152344402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584656844d0 msgr2=0x558465691930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584656844d0 0x558465691930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 -- 172.21.15.26:0/1152344402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001b59c0 con 0x558465685c70 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa60699c700 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009a20 tx=0x7fa5f809a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b7b5da6b2cc94b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f809bc00 con 0x558465685c70 2023-07-13T22:19:54.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.896+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa5f80a3fc0 con 0x558465685c70 2023-07-13T22:19:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.897+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5f0011e50 con 0x558465685c70 2023-07-13T22:19:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.897+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f809b0f0 con 0x558465685c70 2023-07-13T22:19:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.897+0000 7fa61bcefb80 10 client.26375 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.897+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5584654022b0 con 0x558465685c70 2023-07-13T22:19:54.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.898+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f0011e50 con 0x558465685c70 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa5f8012ae0 con 0x558465685c70 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 10 client.26375 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 10 client.26375 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 20 client.26375 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5584656eece0 0x5584656f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 4 client.26375 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:54.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.899+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558465591050 con 0x5584656eece0 2023-07-13T22:19:54.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.900+0000 7fa60619b700 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5584656eece0 0x5584656f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:54.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.901+0000 7fa60619b700 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5584656eece0 0x5584656f10f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3cb4195cb50aad73 server_cookie=b6afed72555f032e in_seq=0 out_seq=0 2023-07-13T22:19:54.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.901+0000 7fa5e9ffb700 10 client.26375 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:54.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.901+0000 7fa5e9ffb700 1 -- 172.21.15.26:0/1152344402 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558465591050 con 0x5584656eece0 2023-07-13T22:19:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.901+0000 7fa5e9ffb700 10 client.26375 handle_command_reply: tid=0 2023-07-13T22:19:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.901+0000 7fa61bcefb80 1 client.26375 shutdown 2023-07-13T22:19:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 msgr2=0x558465694e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009a20 tx=0x7fa5f809a180 comp rx=0 tx=0).stop 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 shutdown_connections 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x5584656eece0 0x5584656f10f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5584656844d0 0x558465691930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558465685c70 0x558465694e90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 --2- 172.21.15.26:0/1152344402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558465684e20 0x558465691e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 >> 172.21.15.26:0/1152344402 conn(0x558465679da0 msgr2=0x55846568c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.902+0000 7fa5e9ffb700 0 client.26375 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:19:54.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.903+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 shutdown_connections 2023-07-13T22:19:54.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.903+0000 7fa61bcefb80 1 -- 172.21.15.26:0/1152344402 wait complete. 2023-07-13T22:19:54.890 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.903+0000 7fa61bcefb80 20 client.26375 trim_cache size 0 max 16384 2023-07-13T22:19:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.908+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 msgr2=0x7fa5ec0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.908+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 0x7fa5ec0a3800 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005fd0 tx=0x7fa5fc005e80 comp rx=0 tx=0).stop 2023-07-13T22:19:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.908+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 msgr2=0x7fa6001b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.908+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b070 tx=0x7fa5f80123f0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 shutdown_connections 2023-07-13T22:19:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5ec0a1350 0x7fa5ec0a3800 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa600070ee0 0x7fa600113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6001b2970 0x7fa6001b4d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 --2- 172.21.15.26:0/597426974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60010bd60 0x7fa6001140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.910+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 >> 172.21.15.26:0/597426974 conn(0x7fa60006ca90 msgr2=0x7fa60006e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.912+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 shutdown_connections 2023-07-13T22:19:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:54.912+0000 7fa6083ff700 1 -- 172.21.15.26:0/597426974 wait complete. 2023-07-13T22:19:54.939 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:54 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3309901744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:54.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:54 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3309901744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:54.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:19:54.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:54 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3309901744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:19:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.617+0000 7fcbadbff700 1 -- 172.21.15.26:0/1143017586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8077210 msgr2=0x7fcba8079690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.617+0000 7fcbadbff700 1 --2- 172.21.15.26:0/1143017586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8077210 0x7fcba8079690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcba000b0a0 tx=0x7fcba000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.630+0000 7fcbadbff700 1 -- 172.21.15.26:0/1143017586 shutdown_connections 2023-07-13T22:19:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.630+0000 7fcbadbff700 1 --2- 172.21.15.26:0/1143017586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8071200 0x7fcba8071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.630+0000 7fcbadbff700 1 --2- 172.21.15.26:0/1143017586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8077210 0x7fcba8079690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.630+0000 7fcbadbff700 1 --2- 172.21.15.26:0/1143017586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba8076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.630+0000 7fcbadbff700 1 -- 172.21.15.26:0/1143017586 >> 172.21.15.26:0/1143017586 conn(0x7fcba806ca90 msgr2=0x7fcba806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.631+0000 7fcbadbff700 1 -- 172.21.15.26:0/1143017586 shutdown_connections 2023-07-13T22:19:55.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.631+0000 7fcbadbff700 1 -- 172.21.15.26:0/1143017586 wait complete. 2023-07-13T22:19:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.632+0000 7fcbadbff700 1 Processor -- start 2023-07-13T22:19:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.632+0000 7fcbadbff700 1 -- start start 2023-07-13T22:19:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 0x7fcba8082480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba80829c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8077210 0x7fcba812f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba812fdd0 con 0x7fcba8077210 2023-07-13T22:19:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba812ff40 con 0x7fcba8071200 2023-07-13T22:19:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.634+0000 7fcbadbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba81300b0 con 0x7fcba8074850 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 0x7fcba8082480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba80829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba80829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51018/0 (socket says 172.21.15.26:51018) 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba6ffd700 1 -- 172.21.15.26:0/3936140943 learned_addr learned my addr 172.21.15.26:0/3936140943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba7fff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8077210 0x7fcba812f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 -- 172.21.15.26:0/3936140943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 msgr2=0x7fcba80829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba80829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 -- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8077210 msgr2=0x7fcba812f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8077210 0x7fcba812f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.635+0000 7fcba77fe700 1 -- 172.21.15.26:0/3936140943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0009d00 con 0x7fcba8071200 2023-07-13T22:19:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.636+0000 7fcba77fe700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 0x7fcba8082480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800b450 tx=0x7fcb9800b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.637+0000 7fcba4ff9700 1 -- 172.21.15.26:0/3936140943 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98011830 con 0x7fcba8071200 2023-07-13T22:19:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.637+0000 7fcba4ff9700 1 -- 172.21.15.26:0/3936140943 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb98011e50 con 0x7fcba8071200 2023-07-13T22:19:55.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.637+0000 7fcba4ff9700 1 -- 172.21.15.26:0/3936140943 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800f2e0 con 0x7fcba8071200 2023-07-13T22:19:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.637+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba8130390 con 0x7fcba8071200 2023-07-13T22:19:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.637+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81308d0 con 0x7fcba8071200 2023-07-13T22:19:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.644+0000 7fcba4ff9700 1 -- 172.21.15.26:0/3936140943 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcb98011990 con 0x7fcba8071200 2023-07-13T22:19:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.649+0000 7fcbc1445b80 1 Processor -- start 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 -- start start 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab67c70 0x55b04ab680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04aa6bcb0 con 0x55b04ab66e20 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04aabea40 con 0x55b04ab664d0 2023-07-13T22:19:55.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.650+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04aab7ed0 con 0x55b04ab67c70 2023-07-13T22:19:55.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44832/0 (socket says 172.21.15.26:44832) 2023-07-13T22:19:55.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 -- 172.21.15.26:0/2935237244 learned_addr learned my addr 172.21.15.26:0/2935237244 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:55.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 -- 172.21.15.26:0/2935237244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab67c70 msgr2=0x55b04ab680e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab67c70 0x55b04ab680e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 -- 172.21.15.26:0/2935237244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 msgr2=0x55b04ab676e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab676e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.651+0000 7fcba7fff700 1 -- 172.21.15.26:0/2935237244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b04a8e5de0 con 0x55b04ab664d0 2023-07-13T22:19:55.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.652+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcba0015000 tx=0x7fcba0007580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd20dc6efc69be1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.655+0000 7fcba4ff9700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 0x7fcb900a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.655+0000 7fcba4ff9700 1 -- 172.21.15.26:0/3936140943 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb980cc1e0 con 0x7fcba8071200 2023-07-13T22:19:55.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.660+0000 7fcba77fe700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 0x7fcb900a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.670+0000 7fcb8dffb700 1 -- 172.21.15.26:0/2935237244 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000e050 con 0x55b04ab664d0 2023-07-13T22:19:55.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.670+0000 7fcb8dffb700 1 -- 172.21.15.26:0/2935237244 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba0007890 con 0x55b04ab664d0 2023-07-13T22:19:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.670+0000 7fcb8dffb700 1 -- 172.21.15.26:0/2935237244 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0005720 con 0x55b04ab664d0 2023-07-13T22:19:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.671+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2935237244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 msgr2=0x55b04ab668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.671+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcba0015000 tx=0x7fcba0007580 comp rx=0 tx=0).stop 2023-07-13T22:19:55.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.680+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2935237244 shutdown_connections 2023-07-13T22:19:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.680+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab67c70 0x55b04ab680e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.680+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab664d0 0x55b04ab668e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.680+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2935237244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.680+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2935237244 >> 172.21.15.26:0/2935237244 conn(0x55b04ab5bda0 msgr2=0x55b04ab611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.682+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2935237244 shutdown_connections 2023-07-13T22:19:55.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.682+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2935237244 wait complete. 2023-07-13T22:19:55.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.682+0000 7fcbc1445b80 1 -- start start 2023-07-13T22:19:55.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.683+0000 7fcbc1445b80 1 Processor -- start 2023-07-13T22:19:55.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.684+0000 7fcba77fe700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 0x7fcb900a3780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcb98095aa0 tx=0x7fcb98004860 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.684+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.684+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.684+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.686+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04aa57ac0 con 0x55b04ab67c70 2023-07-13T22:19:55.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.686+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04aa577c0 con 0x55b04ab66e20 2023-07-13T22:19:55.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.686+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04a93b920 con 0x55b04ab664d0 2023-07-13T22:19:55.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.686+0000 7fcbc1445b80 10 client.0 _ll_register_callbacks cb 0x55b04aa55760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:55.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.687+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.687+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51038/0 (socket says 172.21.15.26:51038) 2023-07-13T22:19:55.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.687+0000 7fcba7fff700 1 -- 172.21.15.26:0/2807871174 learned_addr learned my addr 172.21.15.26:0/2807871174 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:55.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 -- 172.21.15.26:0/2807871174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 msgr2=0x55b04ab70670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab70670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 -- 172.21.15.26:0/2807871174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 msgr2=0x55b04ab776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.688+0000 7fcba7fff700 1 -- 172.21.15.26:0/2807871174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba807a0c0 con 0x55b04ab664d0 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.689+0000 7fcba7fff700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba0007ab0 tx=0x7fcba00074e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91eb1c737d87b51b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.689+0000 7fcb86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:55.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.689+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000e040 con 0x55b04ab664d0 2023-07-13T22:19:55.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.689+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcba0004c50 con 0x55b04ab664d0 2023-07-13T22:19:55.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.690+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b04aa114f0 con 0x55b04ab664d0 2023-07-13T22:19:55.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.716+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba0004e10 con 0x55b04ab664d0 2023-07-13T22:19:55.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.717+0000 7fcbc1445b80 10 client.26359 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:55.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.717+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b04a8cec60 con 0x55b04ab664d0 2023-07-13T22:19:55.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.717+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b04aa114f0 con 0x55b04ab664d0 2023-07-13T22:19:55.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.717+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcba000b0d0 con 0x55b04ab664d0 2023-07-13T22:19:55.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 10 client.26359 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:55.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 10 client.26359 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:19:55.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 20 client.26359 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:55.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04ab7b900 0x55b04ab7dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 4 client.26359 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:55.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.719+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b04aab0eb0 con 0x55b04ab7b900 2023-07-13T22:19:55.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.720+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04ab7b900 0x55b04ab7dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.722+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04ab7b900 0x55b04ab7dd10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3d5532c0d39a3100 server_cookie=d7db695aac597336 in_seq=0 out_seq=0 2023-07-13T22:19:55.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.722+0000 7fcb86ffd700 10 client.26359 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:55.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.723+0000 7fcb86ffd700 1 -- 172.21.15.26:0/2807871174 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b04aab0eb0 con 0x55b04ab7b900 2023-07-13T22:19:55.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.723+0000 7fcb86ffd700 10 client.26359 handle_command_reply: tid=0 2023-07-13T22:19:55.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.723+0000 7fcbc1445b80 1 client.26359 shutdown 2023-07-13T22:19:55.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 msgr2=0x55b04ab70130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba0007ab0 tx=0x7fcba00074e0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 shutdown_connections 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04ab7b900 0x55b04ab7dd10 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab70130 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/2807871174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab70670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 >> 172.21.15.26:0/2807871174 conn(0x55b04ab5bda0 msgr2=0x55b04ab608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.724+0000 7fcb86ffd700 0 client.26359 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:55.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.725+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 shutdown_connections 2023-07-13T22:19:55.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.725+0000 7fcbc1445b80 1 -- 172.21.15.26:0/2807871174 wait complete. 2023-07-13T22:19:55.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.726+0000 7fcbc1445b80 20 client.26359 trim_cache size 0 max 16384 2023-07-13T22:19:55.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.756+0000 7fcbc1445b80 1 Processor -- start 2023-07-13T22:19:55.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.756+0000 7fcbc1445b80 1 -- start start 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab6f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 0x55b04ab77fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab5b970 con 0x55b04ab66e20 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab6ddc0 con 0x55b04ab67c70 2023-07-13T22:19:55.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.757+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab784e0 con 0x55b04ab664d0 2023-07-13T22:19:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.758+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab6f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.758+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab6f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51046/0 (socket says 172.21.15.26:51046) 2023-07-13T22:19:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.758+0000 7fcba7fff700 1 -- 172.21.15.26:0/3867097631 learned_addr learned my addr 172.21.15.26:0/3867097631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.758+0000 7fcba7fff700 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 0x55b04ab77fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.758+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.759+0000 7fcba7fff700 1 -- 172.21.15.26:0/3867097631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 msgr2=0x55b04ab6f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:55.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.759+0000 7fcba7fff700 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab6f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.759+0000 7fcba7fff700 1 -- 172.21.15.26:0/3867097631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 msgr2=0x55b04ab77670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:55.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.759+0000 7fcba7fff700 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab77670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.759+0000 7fcba7fff700 1 -- 172.21.15.26:0/3867097631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b04a90c250 con 0x55b04ab67c70 2023-07-13T22:19:55.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.760+0000 7fcba7fff700 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 0x55b04ab77fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcba00124d0 tx=0x7fcba0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd67d78d837fede server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.771+0000 7fcb8e7fc700 1 -- 172.21.15.26:0/3867097631 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000e040 con 0x55b04ab67c70 2023-07-13T22:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.771+0000 7fcb8e7fc700 1 -- 172.21.15.26:0/3867097631 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba00127c0 con 0x55b04ab67c70 2023-07-13T22:19:55.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.771+0000 7fcb8e7fc700 1 -- 172.21.15.26:0/3867097631 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0012c60 con 0x55b04ab67c70 2023-07-13T22:19:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 -- 172.21.15.26:0/3867097631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 msgr2=0x55b04ab77fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 0x55b04ab77fa0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcba00124d0 tx=0x7fcba0000f40 comp rx=0 tx=0).stop 2023-07-13T22:19:55.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 -- 172.21.15.26:0/3867097631 shutdown_connections 2023-07-13T22:19:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab67c70 0x55b04ab77fa0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab6f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/3867097631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab66e20 0x55b04ab77670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.773+0000 7fcbc1445b80 1 -- 172.21.15.26:0/3867097631 >> 172.21.15.26:0/3867097631 conn(0x55b04ab5bda0 msgr2=0x55b04ab82670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.774+0000 7fcbc1445b80 1 -- 172.21.15.26:0/3867097631 shutdown_connections 2023-07-13T22:19:55.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.775+0000 7fcbc1445b80 1 -- 172.21.15.26:0/3867097631 wait complete. 2023-07-13T22:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.775+0000 7fcbc1445b80 1 -- start start 2023-07-13T22:19:55.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.776+0000 7fcbc1445b80 1 Processor -- start 2023-07-13T22:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.776+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.776+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab73de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcbc1445b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab76700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab76c40 con 0x55b04ab67c70 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab76db0 con 0x55b04ab66e20 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcbc1445b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b04ab76f70 con 0x55b04ab664d0 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcbc1445b80 10 client.0 _ll_register_callbacks cb 0x7fcba0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcba7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51062/0 (socket says 172.21.15.26:51062) 2023-07-13T22:19:55.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.777+0000 7fcba7fff700 1 -- 172.21.15.26:0/353451656 learned_addr learned my addr 172.21.15.26:0/353451656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:55.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab73de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab76700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 -- 172.21.15.26:0/353451656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 msgr2=0x55b04ab73de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab73de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 -- 172.21.15.26:0/353451656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 msgr2=0x55b04ab76700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab76700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.782+0000 7fcba7fff700 1 -- 172.21.15.26:0/353451656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b04a8cec60 con 0x55b04ab664d0 2023-07-13T22:19:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.783+0000 7fcba7fff700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006070 tx=0x7fcba00099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b583ea9efae13e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.783+0000 7fcb86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.783+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000e040 con 0x55b04ab664d0 2023-07-13T22:19:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.784+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcba0012e10 con 0x55b04ab664d0 2023-07-13T22:19:55.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.784+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba0007890 con 0x55b04ab664d0 2023-07-13T22:19:55.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba0032480 con 0x55b04ab664d0 2023-07-13T22:19:55.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 10 client.26365 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:55.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b04a8e42b0 con 0x55b04ab664d0 2023-07-13T22:19:55.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba0007890 con 0x55b04ab664d0 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcba002f4a0 con 0x55b04ab664d0 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 10 client.26365 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 20 client.26365 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.785+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04abcff60 0x55b04abd2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.786+0000 7fcbc1445b80 4 client.26365 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:55.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.786+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b04aa73050 con 0x55b04abcff60 2023-07-13T22:19:55.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.786+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04abcff60 0x55b04abd2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:55.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.787+0000 7fcba6ffd700 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04abcff60 0x55b04abd2340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=edc2c776666311c6 server_cookie=9a832091c3948a3d in_seq=0 out_seq=0 2023-07-13T22:19:55.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.787+0000 7fcb86ffd700 10 client.26365 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:55.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.787+0000 7fcb86ffd700 1 -- 172.21.15.26:0/353451656 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b04aa73050 con 0x55b04abcff60 2023-07-13T22:19:55.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.787+0000 7fcb86ffd700 10 client.26365 handle_command_reply: tid=0 2023-07-13T22:19:55.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.787+0000 7fcbc1445b80 1 client.26365 shutdown 2023-07-13T22:19:55.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 msgr2=0x55b04ab738a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006070 tx=0x7fcba00099e0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 shutdown_connections 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55b04abcff60 0x55b04abd2340 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b04ab67c70 0x55b04ab76700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b04ab664d0 0x55b04ab738a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 --2- 172.21.15.26:0/353451656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b04ab66e20 0x55b04ab73de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcb86ffd700 0 client.26365 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:19:55.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 >> 172.21.15.26:0/353451656 conn(0x55b04ab5bda0 msgr2=0x55b04ab82670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 shutdown_connections 2023-07-13T22:19:55.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.788+0000 7fcbc1445b80 1 -- 172.21.15.26:0/353451656 wait complete. 2023-07-13T22:19:55.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.789+0000 7fcbc1445b80 20 client.26365 trim_cache size 0 max 16384 2023-07-13T22:19:55.776 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:55.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 msgr2=0x7fcb900a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 0x7fcb900a3780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcb98095aa0 tx=0x7fcb98004860 comp rx=0 tx=0).stop 2023-07-13T22:19:55.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 msgr2=0x7fcba8082480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:55.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 0x7fcba8082480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800b450 tx=0x7fcb9800b810 comp rx=0 tx=0).stop 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 shutdown_connections 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcb900a12d0 0x7fcb900a3780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcba8077210 0x7fcba812f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcba8071200 0x7fcba8082480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 --2- 172.21.15.26:0/3936140943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcba8074850 0x7fcba80829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 >> 172.21.15.26:0/3936140943 conn(0x7fcba806ca90 msgr2=0x7fcba8076010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:55.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.790+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 shutdown_connections 2023-07-13T22:19:55.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:55.791+0000 7fcbadbff700 1 -- 172.21.15.26:0/3936140943 wait complete. 2023-07-13T22:19:55.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:19:55.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:55 smithi117 ceph-mon[129074]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 302 MiB/s wr, 92 op/s 2023-07-13T22:19:55.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:55 smithi026 ceph-mon[95647]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 302 MiB/s wr, 92 op/s 2023-07-13T22:19:55.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:55 smithi121 ceph-mon[129187]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 302 MiB/s wr, 92 op/s 2023-07-13T22:19:56.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.487+0000 7f1068dff700 1 -- 172.21.15.26:0/619304070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 msgr2=0x7f10640712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.487+0000 7f1068dff700 1 --2- 172.21.15.26:0/619304070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10640712f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1054009990 tx=0x7f1054009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.488+0000 7f1068dff700 1 -- 172.21.15.26:0/619304070 shutdown_connections 2023-07-13T22:19:56.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.488+0000 7f1068dff700 1 --2- 172.21.15.26:0/619304070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f106410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.488+0000 7f1068dff700 1 --2- 172.21.15.26:0/619304070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10640712f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.488+0000 7f1068dff700 1 --2- 172.21.15.26:0/619304070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1064071830 0x7f106410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.488+0000 7f1068dff700 1 -- 172.21.15.26:0/619304070 >> 172.21.15.26:0/619304070 conn(0x7f106406ca90 msgr2=0x7f106406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.489+0000 7f1068dff700 1 -- 172.21.15.26:0/619304070 shutdown_connections 2023-07-13T22:19:56.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.489+0000 7f1068dff700 1 -- 172.21.15.26:0/619304070 wait complete. 2023-07-13T22:19:56.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.491+0000 7f1068dff700 1 Processor -- start 2023-07-13T22:19:56.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.491+0000 7f1068dff700 1 -- start start 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10641a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1064071830 0x7f10641a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10641ac2d0 con 0x7f1064071830 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106410ec40 con 0x7f106410bd60 2023-07-13T22:19:56.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.492+0000 7f1068dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106410edd0 con 0x7f1064070ee0 2023-07-13T22:19:56.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44878/0 (socket says 172.21.15.26:44878) 2023-07-13T22:19:56.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 -- 172.21.15.26:0/2324070037 learned_addr learned my addr 172.21.15.26:0/2324070037 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f106259c700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10641a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 -- 172.21.15.26:0/2324070037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 msgr2=0x7f10641a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10641a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 -- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1064071830 msgr2=0x7f10641a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1064071830 0x7f10641a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.495+0000 7f1062d9d700 1 -- 172.21.15.26:0/2324070037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540095f0 con 0x7f106410bd60 2023-07-13T22:19:56.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.496+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f105c007ee0 tx=0x7f105c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.496+0000 7f10537fe700 1 -- 172.21.15.26:0/2324070037 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c017070 con 0x7f106410bd60 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.497+0000 7f10537fe700 1 -- 172.21.15.26:0/2324070037 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105c00f040 con 0x7f106410bd60 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.497+0000 7f10537fe700 1 -- 172.21.15.26:0/2324070037 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c005740 con 0x7f106410bd60 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.497+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106410f0b0 con 0x7f106410bd60 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.497+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f106410f570 con 0x7f106410bd60 2023-07-13T22:19:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.505+0000 7f107c75cb80 1 Processor -- start 2023-07-13T22:19:56.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.505+0000 7f107c75cb80 1 -- start start 2023-07-13T22:19:56.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfa48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfa56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dea9cb0 con 0x56368dfa44d0 2023-07-13T22:19:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368defca40 con 0x56368dfa4e20 2023-07-13T22:19:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.506+0000 7f107c75cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368def5ed0 con 0x56368dfa5c70 2023-07-13T22:19:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.508+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfa48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.508+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.509+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51098/0 (socket says 172.21.15.26:51098) 2023-07-13T22:19:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.509+0000 7f1061d9b700 1 -- 172.21.15.26:0/1219710120 learned_addr learned my addr 172.21.15.26:0/1219710120 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.509+0000 7f106259c700 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfa56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 -- 172.21.15.26:0/1219710120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 msgr2=0x56368dfa56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfa56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 -- 172.21.15.26:0/1219710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 msgr2=0x56368dfa48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfa48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 -- 172.21.15.26:0/1219710120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368dd23de0 con 0x56368dfa5c70 2023-07-13T22:19:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.510+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f105800abd0 tx=0x7f1058004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866fb55b861a9464 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.514+0000 7f10537fe700 1 -- 172.21.15.26:0/2324070037 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f105c0058a0 con 0x7f106410bd60 2023-07-13T22:19:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.517+0000 7f10537fe700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 0x7f104c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.518+0000 7f10537fe700 1 -- 172.21.15.26:0/2324070037 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f105c0cb7c0 con 0x7f106410bd60 2023-07-13T22:19:56.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.518+0000 7f1050ff9700 1 -- 172.21.15.26:0/1219710120 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800fb20 con 0x56368dfa5c70 2023-07-13T22:19:56.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.519+0000 7f1050ff9700 1 -- 172.21.15.26:0/1219710120 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105800d070 con 0x56368dfa5c70 2023-07-13T22:19:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.519+0000 7f1050ff9700 1 -- 172.21.15.26:0/1219710120 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580133a0 con 0x56368dfa5c70 2023-07-13T22:19:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 -- 172.21.15.26:0/1219710120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 msgr2=0x56368dfa60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f105800abd0 tx=0x7f1058004870 comp rx=0 tx=0).stop 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 -- 172.21.15.26:0/1219710120 shutdown_connections 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfa48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfa60e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1219710120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfa56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.520+0000 7f107c75cb80 1 -- 172.21.15.26:0/1219710120 >> 172.21.15.26:0/1219710120 conn(0x56368df99da0 msgr2=0x56368df9f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.521+0000 7f107c75cb80 1 -- 172.21.15.26:0/1219710120 shutdown_connections 2023-07-13T22:19:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.521+0000 7f107c75cb80 1 -- 172.21.15.26:0/1219710120 wait complete. 2023-07-13T22:19:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.522+0000 7f107c75cb80 1 -- start start 2023-07-13T22:19:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.522+0000 7f107c75cb80 1 Processor -- start 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f106259c700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 0x7f104c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368de95ac0 con 0x56368dfa44d0 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368de957c0 con 0x56368dfa4e20 2023-07-13T22:19:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dd79920 con 0x56368dfa5c70 2023-07-13T22:19:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.523+0000 7f107c75cb80 10 client.0 _ll_register_callbacks cb 0x56368de93760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44890/0 (socket says 172.21.15.26:44890) 2023-07-13T22:19:56.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 -- 172.21.15.26:0/3754645456 learned_addr learned my addr 172.21.15.26:0/3754645456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:56.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1062d9d700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 -- 172.21.15.26:0/3754645456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 msgr2=0x56368dfb5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 -- 172.21.15.26:0/3754645456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 msgr2=0x56368dfae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.524+0000 7f1061d9b700 1 -- 172.21.15.26:0/3754645456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105800a8b0 con 0x56368dfa4e20 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.525+0000 7f1061d9b700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1058001f70 tx=0x7f1058006f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d374e770f23d230e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.533+0000 7f106259c700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 0x7f104c0a3800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x56368dfaf480 tx=0x7f1054009f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.534+0000 7f1049ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:56.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.536+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058014810 con 0x56368dfa4e20 2023-07-13T22:19:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.536+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1058021030 con 0x56368dfa4e20 2023-07-13T22:19:56.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.537+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56368de4f4f0 con 0x56368dfa4e20 2023-07-13T22:19:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.541+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1058014970 con 0x56368dfa4e20 2023-07-13T22:19:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.541+0000 7f107c75cb80 10 client.26402 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:56.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.541+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56368dd0cc60 con 0x56368dfa4e20 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.545+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56368de4f4f0 con 0x56368dfa4e20 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f105801d5a0 con 0x56368dfa4e20 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 10 client.26402 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 10 client.26402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 20 client.26402 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:56.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368dfb9e30 0x56368dfbc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 4 client.26402 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:56.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.546+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56368deeeeb0 con 0x56368dfb9e30 2023-07-13T22:19:56.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.551+0000 7f106259c700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368dfb9e30 0x56368dfbc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.558+0000 7f106259c700 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368dfb9e30 0x56368dfbc240 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdf5901611b9fa0c server_cookie=ea9353be327431ac in_seq=0 out_seq=0 2023-07-13T22:19:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.565+0000 7f1049ffb700 10 client.26402 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.565+0000 7f1049ffb700 1 -- 172.21.15.26:0/3754645456 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56368deeeeb0 con 0x56368dfb9e30 2023-07-13T22:19:56.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.565+0000 7f1049ffb700 10 client.26402 handle_command_reply: tid=0 2023-07-13T22:19:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.565+0000 7f107c75cb80 1 client.26402 shutdown 2023-07-13T22:19:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 msgr2=0x56368dfae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1058001f70 tx=0x7f1058006f40 comp rx=0 tx=0).stop 2023-07-13T22:19:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 shutdown_connections 2023-07-13T22:19:56.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368dfb9e30 0x56368dfbc240 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 --2- 172.21.15.26:0/3754645456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa4e20 0x56368dfae7d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.566+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 >> 172.21.15.26:0/3754645456 conn(0x56368df99da0 msgr2=0x56368df9f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.567+0000 7f1049ffb700 0 client.26402 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:56.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.567+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 shutdown_connections 2023-07-13T22:19:56.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.567+0000 7f107c75cb80 1 -- 172.21.15.26:0/3754645456 wait complete. 2023-07-13T22:19:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.575+0000 7f107c75cb80 20 client.26402 trim_cache size 0 max 16384 2023-07-13T22:19:56.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.627+0000 7f107c75cb80 1 Processor -- start 2023-07-13T22:19:56.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 -- start start 2023-07-13T22:19:56.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfb6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 0x56368dfb57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 0x56368dfacee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dfad790 con 0x56368dfa44d0 2023-07-13T22:19:56.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dfade30 con 0x56368dfa5c70 2023-07-13T22:19:56.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.631+0000 7f107c75cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368df9ff20 con 0x56368dfa4e20 2023-07-13T22:19:56.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.632+0000 7f106259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfb6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.641+0000 7f1061d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 0x56368dfb57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.641+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 0x56368dfacee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.641+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 0x56368dfacee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44898/0 (socket says 172.21.15.26:44898) 2023-07-13T22:19:56.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.641+0000 7f1062d9d700 1 -- 172.21.15.26:0/1044914360 learned_addr learned my addr 172.21.15.26:0/1044914360 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.643+0000 7f1061d9b700 1 -- 172.21.15.26:0/1044914360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 msgr2=0x56368dfacee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.643+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 0x56368dfacee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.643+0000 7f1061d9b700 1 -- 172.21.15.26:0/1044914360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 msgr2=0x56368dfb6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.643+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfb6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.643+0000 7f1061d9b700 1 -- 172.21.15.26:0/1044914360 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56368dd4a250 con 0x56368dfa4e20 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.644+0000 7f1061d9b700 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 0x56368dfb57d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f105801d930 tx=0x7f105800a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38035ba1e8dfb02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.645+0000 7f10517fa700 1 -- 172.21.15.26:0/1044914360 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105801cf00 con 0x56368dfa4e20 2023-07-13T22:19:56.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.645+0000 7f10517fa700 1 -- 172.21.15.26:0/1044914360 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105801dba0 con 0x56368dfa4e20 2023-07-13T22:19:56.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.645+0000 7f10517fa700 1 -- 172.21.15.26:0/1044914360 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105801c360 con 0x56368dfa4e20 2023-07-13T22:19:56.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.648+0000 7f107c75cb80 1 -- 172.21.15.26:0/1044914360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 msgr2=0x56368dfb57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.648+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 0x56368dfb57d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f105801d930 tx=0x7f105800a510 comp rx=0 tx=0).stop 2023-07-13T22:19:56.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.657+0000 7f107c75cb80 1 -- 172.21.15.26:0/1044914360 shutdown_connections 2023-07-13T22:19:56.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.657+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa44d0 0x56368dfb6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.657+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa5c70 0x56368dfacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.657+0000 7f107c75cb80 1 --2- 172.21.15.26:0/1044914360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa4e20 0x56368dfb57d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.657+0000 7f107c75cb80 1 -- 172.21.15.26:0/1044914360 >> 172.21.15.26:0/1044914360 conn(0x56368df99da0 msgr2=0x56368dfc0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.658+0000 7f107c75cb80 1 -- 172.21.15.26:0/1044914360 shutdown_connections 2023-07-13T22:19:56.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.658+0000 7f107c75cb80 1 -- 172.21.15.26:0/1044914360 wait complete. 2023-07-13T22:19:56.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.659+0000 7f107c75cb80 1 -- start start 2023-07-13T22:19:56.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.659+0000 7f107c75cb80 1 Processor -- start 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa44d0 0x56368dfb30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa4e20 0x56368dfb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dfb4b80 con 0x56368dfa4e20 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dfb4cf0 con 0x56368dfa44d0 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56368dfb4eb0 con 0x56368dfa5c70 2023-07-13T22:19:56.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.660+0000 7f107c75cb80 10 client.0 _ll_register_callbacks cb 0x7f105800aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:56.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.662+0000 7f106259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa44d0 0x56368dfb30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.662+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.663+0000 7f1062d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51116/0 (socket says 172.21.15.26:51116) 2023-07-13T22:19:56.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.663+0000 7f1062d9d700 1 -- 172.21.15.26:0/2406008490 learned_addr learned my addr 172.21.15.26:0/2406008490 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:56.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.664+0000 7f1062d9d700 1 -- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa44d0 msgr2=0x56368dfb30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.664+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa44d0 0x56368dfb30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.664+0000 7f1062d9d700 1 -- 172.21.15.26:0/2406008490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa4e20 msgr2=0x56368dfb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.664+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa4e20 0x56368dfb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.664+0000 7f1062d9d700 1 -- 172.21.15.26:0/2406008490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56368dd0cc60 con 0x56368dfa5c70 2023-07-13T22:19:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.666+0000 7f1062d9d700 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f105c0956a0 tx=0x7f105c098d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae07caf24b33d0c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:56.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.706+0000 7f1049ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.706+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c098e90 con 0x56368dfa5c70 2023-07-13T22:19:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.706+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f105c098ff0 con 0x56368dfa5c70 2023-07-13T22:19:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.706+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f105800d070 con 0x56368dfa5c70 2023-07-13T22:19:56.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.710+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105c098120 con 0x56368dfa5c70 2023-07-13T22:19:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.712+0000 7f107c75cb80 10 client.26383 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:56.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.712+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56368dd222b0 con 0x56368dfa5c70 2023-07-13T22:19:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.714+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f105800d070 con 0x56368dfa5c70 2023-07-13T22:19:56.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.714+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f105c004fb0 con 0x56368dfa5c70 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 10 client.26383 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 10 client.26383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 20 client.26383 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368e00e790 0x56368e010ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 4 client.26383 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:56.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.715+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56368deb1050 con 0x56368e00e790 2023-07-13T22:19:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.727+0000 7f106259c700 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368e00e790 0x56368e010ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.727+0000 7f106259c700 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368e00e790 0x56368e010ba0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34c6096a9810f1b7 server_cookie=7528f4ed532e1593 in_seq=0 out_seq=0 2023-07-13T22:19:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f1049ffb700 10 client.26383 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f1049ffb700 1 -- 172.21.15.26:0/2406008490 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56368deb1050 con 0x56368e00e790 2023-07-13T22:19:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f1049ffb700 10 client.26383 handle_command_reply: tid=0 2023-07-13T22:19:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f107c75cb80 1 client.26383 shutdown 2023-07-13T22:19:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 msgr2=0x56368dfb4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.738+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f105c0956a0 tx=0x7f105c098d70 comp rx=0 tx=0).stop 2023-07-13T22:19:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.745+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 shutdown_connections 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.745+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x56368e00e790 0x56368e010ba0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.745+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56368dfa5c70 0x56368dfb4640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.745+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56368dfa44d0 0x56368dfb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.745+0000 7f107c75cb80 1 --2- 172.21.15.26:0/2406008490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56368dfa4e20 0x56368dfb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.746+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 >> 172.21.15.26:0/2406008490 conn(0x56368df99da0 msgr2=0x56368dfacee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.747+0000 7f1049ffb700 0 client.26383 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:19:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.763+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 shutdown_connections 2023-07-13T22:19:56.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.763+0000 7f107c75cb80 1 -- 172.21.15.26:0/2406008490 wait complete. 2023-07-13T22:19:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.764+0000 7f107c75cb80 20 client.26383 trim_cache size 0 max 16384 2023-07-13T22:19:56.750 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.765+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 msgr2=0x7f104c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.765+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 0x7f104c0a3800 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x56368dfaf480 tx=0x7f1054009f50 comp rx=0 tx=0).stop 2023-07-13T22:19:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.765+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 msgr2=0x7f10641abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:56.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.765+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f105c007ee0 tx=0x7f105c00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 shutdown_connections 2023-07-13T22:19:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f106410bd60 0x7f10641abd00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1064070ee0 0x7f10641a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f104c0a1350 0x7f104c0a3800 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 --2- 172.21.15.26:0/2324070037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1064071830 0x7f10641a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 >> 172.21.15.26:0/2324070037 conn(0x7f106406ca90 msgr2=0x7f106406e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.774+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 shutdown_connections 2023-07-13T22:19:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:56.775+0000 7f1068dff700 1 -- 172.21.15.26:0/2324070037 wait complete. 2023-07-13T22:19:56.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:19:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 -- 172.21.15.26:0/881959101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 msgr2=0x7f800c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 --2- 172.21.15.26:0/881959101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 0x7f800c10b7d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098f0 tx=0x7f7ffc009c00 comp rx=0 tx=0).stop 2023-07-13T22:19:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 -- 172.21.15.26:0/881959101 shutdown_connections 2023-07-13T22:19:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 --2- 172.21.15.26:0/881959101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c10bd60 0x7f800c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 --2- 172.21.15.26:0/881959101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f800c070ee0 0x7f800c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 --2- 172.21.15.26:0/881959101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 0x7f800c10b7d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.521+0000 7f80123ff700 1 -- 172.21.15.26:0/881959101 >> 172.21.15.26:0/881959101 conn(0x7f800c06ca90 msgr2=0x7f800c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.525+0000 7f80123ff700 1 -- 172.21.15.26:0/881959101 shutdown_connections 2023-07-13T22:19:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.526+0000 7f80123ff700 1 -- 172.21.15.26:0/881959101 wait complete. 2023-07-13T22:19:57.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.527+0000 7f80123ff700 1 Processor -- start 2023-07-13T22:19:57.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 -- start start 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 0x7f800c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f800c10bd60 0x7f800c1abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1ac280 con 0x7f800c071830 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c10ec40 con 0x7f800c070ee0 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.528+0000 7f80123ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c10edd0 con 0x7f800c10bd60 2023-07-13T22:19:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44942/0 (socket says 172.21.15.26:44942) 2023-07-13T22:19:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 -- 172.21.15.26:0/778008219 learned_addr learned my addr 172.21.15.26:0/778008219 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 -- 172.21.15.26:0/778008219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f800c10bd60 msgr2=0x7f800c1abcb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:19:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f800c10bd60 0x7f800c1abcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 -- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 msgr2=0x7f800c1a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 0x7f800c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.529+0000 7f800bfff700 1 -- 172.21.15.26:0/778008219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009520 con 0x7f800c070ee0 2023-07-13T22:19:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.530+0000 7f800bfff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f800400c0c0 tx=0x7f800400c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.531+0000 7f80097fa700 1 -- 172.21.15.26:0/778008219 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004018070 con 0x7f800c070ee0 2023-07-13T22:19:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.531+0000 7f80097fa700 1 -- 172.21.15.26:0/778008219 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f800400f040 con 0x7f800c070ee0 2023-07-13T22:19:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.531+0000 7f80097fa700 1 -- 172.21.15.26:0/778008219 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400e020 con 0x7f800c070ee0 2023-07-13T22:19:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.531+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c10f050 con 0x7f800c070ee0 2023-07-13T22:19:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.531+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c10f510 con 0x7f800c070ee0 2023-07-13T22:19:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.538+0000 7f80097fa700 1 -- 172.21.15.26:0/778008219 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80040075c0 con 0x7f800c070ee0 2023-07-13T22:19:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.539+0000 7f8025d52b80 1 Processor -- start 2023-07-13T22:19:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.539+0000 7f8025d52b80 1 -- start start 2023-07-13T22:19:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632eda9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9dc70 0x5632eda9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed9a1cb0 con 0x5632eda9ce20 2023-07-13T22:19:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed9f4a40 con 0x5632eda9c4d0 2023-07-13T22:19:57.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f8025d52b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed9eded0 con 0x5632eda9dc70 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.540+0000 7f800bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52720/0 (socket says 172.21.15.26:52720) 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 -- 172.21.15.26:0/1839540947 learned_addr learned my addr 172.21.15.26:0/1839540947 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800b7fe700 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9dc70 0x5632eda9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f801099c700 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632eda9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 -- 172.21.15.26:0/1839540947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9dc70 msgr2=0x5632eda9e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9dc70 0x5632eda9e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 -- 172.21.15.26:0/1839540947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 msgr2=0x5632eda9c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632eda9c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.541+0000 7f800bfff700 1 -- 172.21.15.26:0/1839540947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ed81bde0 con 0x5632eda9ce20 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.542+0000 7f80097fa700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 0x7f7ff40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.542+0000 7f800bfff700 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f80040cff40 tx=0x7f80040d43c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b035bde36fb71033 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.542+0000 7f80097fa700 1 -- 172.21.15.26:0/778008219 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80040cc0f0 con 0x7f800c070ee0 2023-07-13T22:19:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.543+0000 7f7ff27fc700 1 -- 172.21.15.26:0/1839540947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040957f0 con 0x5632eda9ce20 2023-07-13T22:19:57.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.543+0000 7f801099c700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 0x7f7ff40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.543+0000 7f7ff27fc700 1 -- 172.21.15.26:0/1839540947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8004099280 con 0x5632eda9ce20 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.543+0000 7f7ff27fc700 1 -- 172.21.15.26:0/1839540947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040cfb60 con 0x5632eda9ce20 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 -- 172.21.15.26:0/1839540947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 msgr2=0x5632eda9d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f80040cff40 tx=0x7f80040d43c0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f801099c700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 0x7f7ff40a3800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b30 tx=0x7f8000009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 -- 172.21.15.26:0/1839540947 shutdown_connections 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9dc70 0x5632eda9e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632eda9c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 --2- 172.21.15.26:0/1839540947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632eda9d6e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.544+0000 7f8025d52b80 1 -- 172.21.15.26:0/1839540947 >> 172.21.15.26:0/1839540947 conn(0x5632eda91da0 msgr2=0x5632eda971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.545+0000 7f8025d52b80 1 -- 172.21.15.26:0/1839540947 shutdown_connections 2023-07-13T22:19:57.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.545+0000 7f8025d52b80 1 -- 172.21.15.26:0/1839540947 wait complete. 2023-07-13T22:19:57.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.545+0000 7f8025d52b80 1 -- start start 2023-07-13T22:19:57.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.546+0000 7f8025d52b80 1 Processor -- start 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.546+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.546+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9ce20 0x5632edaa6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.546+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f8025d52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed98dac0 con 0x5632eda9dc70 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f8025d52b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed98d7c0 con 0x5632eda9ce20 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f8025d52b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ed871920 con 0x5632eda9c4d0 2023-07-13T22:19:57.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f8025d52b80 10 client.0 _ll_register_callbacks cb 0x5632ed98b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51146/0 (socket says 172.21.15.26:51146) 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 -- 172.21.15.26:0/891763228 learned_addr learned my addr 172.21.15.26:0/891763228 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 -- 172.21.15.26:0/891763228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9ce20 msgr2=0x5632edaa6710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9ce20 0x5632edaa6710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 -- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 msgr2=0x5632edaad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.547+0000 7f800bfff700 1 -- 172.21.15.26:0/891763228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c10f510 con 0x5632eda9c4d0 2023-07-13T22:19:57.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f800bfff700 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8004099440 tx=0x7f8004009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b5b115e825db92 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f7feb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040957f0 con 0x5632eda9c4d0 2023-07-13T22:19:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f800401a2c0 con 0x5632eda9c4d0 2023-07-13T22:19:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632ed9474f0 con 0x5632eda9c4d0 2023-07-13T22:19:57.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80040cfb60 con 0x5632eda9c4d0 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.548+0000 7f8025d52b80 10 client.26392 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5632ed804c60 con 0x5632eda9c4d0 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632ed9474f0 con 0x5632eda9c4d0 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f800409d120 con 0x5632eda9c4d0 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 10 client.26392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:19:57.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 20 client.26392 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edab1b40 0x5632edab3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 4 client.26392 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.549+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632ed9e6eb0 con 0x5632edab1b40 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.550+0000 7f800b7fe700 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edab1b40 0x5632edab3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.550+0000 7f800b7fe700 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edab1b40 0x5632edab3f50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35f0c8ae4e4c0892 server_cookie=1902ae3c99efae4b in_seq=0 out_seq=0 2023-07-13T22:19:57.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.550+0000 7f7feb7fe700 10 client.26392 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.551+0000 7f7feb7fe700 1 -- 172.21.15.26:0/891763228 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5632ed9e6eb0 con 0x5632edab1b40 2023-07-13T22:19:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.551+0000 7f7feb7fe700 10 client.26392 handle_command_reply: tid=0 2023-07-13T22:19:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 client.26392 shutdown 2023-07-13T22:19:57.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 msgr2=0x5632edaa61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8004099440 tx=0x7f8004009960 comp rx=0 tx=0).stop 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 shutdown_connections 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edab1b40 0x5632edab3f50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632edaa61d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 --2- 172.21.15.26:0/891763228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9ce20 0x5632edaa6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 >> 172.21.15.26:0/891763228 conn(0x5632eda91da0 msgr2=0x5632eda968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.552+0000 7f7feb7fe700 0 client.26392 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.553+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 shutdown_connections 2023-07-13T22:19:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.553+0000 7f8025d52b80 1 -- 172.21.15.26:0/891763228 wait complete. 2023-07-13T22:19:57.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.553+0000 7f8025d52b80 20 client.26392 trim_cache size 0 max 16384 2023-07-13T22:19:57.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.592+0000 7f8025d52b80 1 Processor -- start 2023-07-13T22:19:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.592+0000 7f8025d52b80 1 -- start start 2023-07-13T22:19:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632eda9e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9dc70 0x5632edaa4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eda98070 con 0x5632eda9ce20 2023-07-13T22:19:57.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632edaa53f0 con 0x5632eda9dc70 2023-07-13T22:19:57.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.593+0000 7f8025d52b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632eda984c0 con 0x5632eda9c4d0 2023-07-13T22:19:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52728/0 (socket says 172.21.15.26:52728) 2023-07-13T22:19:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 -- 172.21.15.26:0/3561751213 learned_addr learned my addr 172.21.15.26:0/3561751213 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f801099c700 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9dc70 0x5632edaa4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 -- 172.21.15.26:0/3561751213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 msgr2=0x5632eda9e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632eda9e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 -- 172.21.15.26:0/3561751213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9dc70 msgr2=0x5632edaa4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9dc70 0x5632edaa4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.594+0000 7f800b7fe700 1 -- 172.21.15.26:0/3561751213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ed842250 con 0x5632eda9ce20 2023-07-13T22:19:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.595+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f800c072540 tx=0x7f7ffc005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0eecb7737286e42 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.599+0000 7f7ff2ffd700 1 -- 172.21.15.26:0/3561751213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01c040 con 0x5632eda9ce20 2023-07-13T22:19:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.599+0000 7f7ff2ffd700 1 -- 172.21.15.26:0/3561751213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc00f580 con 0x5632eda9ce20 2023-07-13T22:19:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.599+0000 7f7ff2ffd700 1 -- 172.21.15.26:0/3561751213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc00b980 con 0x5632eda9ce20 2023-07-13T22:19:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.600+0000 7f8025d52b80 1 -- 172.21.15.26:0/3561751213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 msgr2=0x5632edaad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.600+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f800c072540 tx=0x7f7ffc005ce0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.601+0000 7f8025d52b80 1 -- 172.21.15.26:0/3561751213 shutdown_connections 2023-07-13T22:19:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.601+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9dc70 0x5632edaa4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.601+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9c4d0 0x5632eda9e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.601+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3561751213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9ce20 0x5632edaad710 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.601+0000 7f8025d52b80 1 -- 172.21.15.26:0/3561751213 >> 172.21.15.26:0/3561751213 conn(0x5632eda91da0 msgr2=0x5632edab88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.604+0000 7f8025d52b80 1 -- 172.21.15.26:0/3561751213 shutdown_connections 2023-07-13T22:19:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.604+0000 7f8025d52b80 1 -- 172.21.15.26:0/3561751213 wait complete. 2023-07-13T22:19:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.605+0000 7f8025d52b80 1 -- start start 2023-07-13T22:19:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.605+0000 7f8025d52b80 1 Processor -- start 2023-07-13T22:19:57.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 0x5632edaa9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632edaacda0 con 0x5632eda9dc70 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632edaacf10 con 0x5632eda9c4d0 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632edaad0d0 con 0x5632eda9ce20 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f8025d52b80 10 client.0 _ll_register_callbacks cb 0x7f80040cfd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f800bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f800bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44966/0 (socket says 172.21.15.26:44966) 2023-07-13T22:19:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.606+0000 7f800bfff700 1 -- 172.21.15.26:0/3458735989 learned_addr learned my addr 172.21.15.26:0/3458735989 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 0x5632edaa9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 -- 172.21.15.26:0/3458735989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 msgr2=0x5632edaa9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 0x5632edaa9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 -- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 msgr2=0x5632edaac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 -- 172.21.15.26:0/3458735989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c117640 con 0x5632eda9c4d0 2023-07-13T22:19:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.607+0000 7f800bfff700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8004095f20 tx=0x7f8004009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1030ce09a7c5356 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.608+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 0x5632edaa9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:19:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.610+0000 7f7feb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.610+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040a1900 con 0x5632eda9c4d0 2023-07-13T22:19:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.610+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80040cf540 con 0x5632eda9c4d0 2023-07-13T22:19:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.611+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632ed9a9050 con 0x5632eda9c4d0 2023-07-13T22:19:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.611+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80040957f0 con 0x5632eda9c4d0 2023-07-13T22:19:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.611+0000 7f8025d52b80 10 client.26417 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.611+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5632ed81a2b0 con 0x5632eda9c4d0 2023-07-13T22:19:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.611+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632ed9a9050 con 0x5632eda9c4d0 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f80040a1470 con 0x5632eda9c4d0 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 10 client.26417 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 10 client.26417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 20 client.26417 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edb06290 0x5632edb08670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 4 client.26417 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.612+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5632ed83cf70 con 0x5632edb06290 2023-07-13T22:19:57.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.614+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edb06290 0x5632edb08670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.622+0000 7f7feb7fe700 10 client.26417 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.622+0000 7f800b7fe700 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edb06290 0x5632edb08670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab178046fb722636 server_cookie=80d793f98f0bc253 in_seq=0 out_seq=0 2023-07-13T22:19:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.622+0000 7f7feb7fe700 1 -- 172.21.15.26:0/3458735989 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5632ed83cf70 con 0x5632edb06290 2023-07-13T22:19:57.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.622+0000 7f7feb7fe700 10 client.26417 handle_command_reply: tid=0 2023-07-13T22:19:57.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.623+0000 7f8025d52b80 1 client.26417 shutdown 2023-07-13T22:19:57.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.623+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 msgr2=0x5632edaa98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.623+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8004095f20 tx=0x7f8004009a50 comp rx=0 tx=0).stop 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 shutdown_connections 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5632edb06290 0x5632edb08670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5632eda9dc70 0x5632edaac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5632eda9c4d0 0x5632edaa98a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 --2- 172.21.15.26:0/3458735989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5632eda9ce20 0x5632edaa9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 >> 172.21.15.26:0/3458735989 conn(0x5632eda91da0 msgr2=0x5632edaadfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f7feb7fe700 0 client.26417 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:19:57.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 shutdown_connections 2023-07-13T22:19:57.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.626+0000 7f8025d52b80 1 -- 172.21.15.26:0/3458735989 wait complete. 2023-07-13T22:19:57.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.628+0000 7f8025d52b80 20 client.26417 trim_cache size 0 max 16384 2023-07-13T22:19:57.620 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:57.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.638+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 msgr2=0x7f7ff40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.638+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 0x7f7ff40a3800 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8000009b30 tx=0x7f8000009260 comp rx=0 tx=0).stop 2023-07-13T22:19:57.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.638+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 msgr2=0x7f800c1a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:57.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.638+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f800400c0c0 tx=0x7f800400c480 comp rx=0 tx=0).stop 2023-07-13T22:19:57.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 shutdown_connections 2023-07-13T22:19:57.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f800c10bd60 0x7f800c1abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f800c070ee0 0x7f800c1a4700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ff40a1350 0x7f7ff40a3800 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 --2- 172.21.15.26:0/778008219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f800c071830 0x7f800c1a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:57.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.639+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 >> 172.21.15.26:0/778008219 conn(0x7f800c06ca90 msgr2=0x7f800c06dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:57.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.640+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 shutdown_connections 2023-07-13T22:19:57.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:57.640+0000 7f80123ff700 1 -- 172.21.15.26:0/778008219 wait complete. 2023-07-13T22:19:57.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:19:57.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:57 smithi026 ceph-mon[95647]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 82 op/s 2023-07-13T22:19:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.183+0000 7f517d9ff700 1 -- 172.21.15.26:0/1278540893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178070ee0 msgr2=0x7f51780712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.183+0000 7f517d9ff700 1 --2- 172.21.15.26:0/1278540893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178070ee0 0x7f51780712f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168009ca0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 -- 172.21.15.26:0/1278540893 shutdown_connections 2023-07-13T22:19:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 --2- 172.21.15.26:0/1278540893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517810bd60 0x7f517810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 --2- 172.21.15.26:0/1278540893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178070ee0 0x7f51780712f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 --2- 172.21.15.26:0/1278540893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178071830 0x7f517810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 -- 172.21.15.26:0/1278540893 >> 172.21.15.26:0/1278540893 conn(0x7f517806ca90 msgr2=0x7f517806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 -- 172.21.15.26:0/1278540893 shutdown_connections 2023-07-13T22:19:58.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.184+0000 7f517d9ff700 1 -- 172.21.15.26:0/1278540893 wait complete. 2023-07-13T22:19:58.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.186+0000 7f517d9ff700 1 Processor -- start 2023-07-13T22:19:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.186+0000 7f517d9ff700 1 -- start start 2023-07-13T22:19:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178071830 0x7f51781a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517810bd60 0x7f51781abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51781ac2b0 con 0x7f517810bd60 2023-07-13T22:19:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51781ac3f0 con 0x7f5178071830 2023-07-13T22:19:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.187+0000 7f517d9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51781ac580 con 0x7f5178070ee0 2023-07-13T22:19:58.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.188+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.188+0000 7f5176ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51182/0 (socket says 172.21.15.26:51182) 2023-07-13T22:19:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.188+0000 7f5176ffd700 1 -- 172.21.15.26:0/178524534 learned_addr learned my addr 172.21.15.26:0/178524534 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.188+0000 7f51767fc700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178071830 0x7f51781a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.189+0000 7f5176ffd700 1 -- 172.21.15.26:0/178524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178071830 msgr2=0x7f51781a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.189+0000 7f5176ffd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178071830 0x7f51781a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.189+0000 7f5176ffd700 1 -- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517810bd60 msgr2=0x7f51781abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.189+0000 7f5176ffd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517810bd60 0x7f51781abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.189+0000 7f5176ffd700 1 -- 172.21.15.26:0/178524534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680095f0 con 0x7f5178070ee0 2023-07-13T22:19:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.191+0000 7f5176ffd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f5168003cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.191+0000 7f517c9fd700 1 -- 172.21.15.26:0/178524534 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516801d070 con 0x7f5178070ee0 2023-07-13T22:19:58.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.191+0000 7f517c9fd700 1 -- 172.21.15.26:0/178524534 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516800f460 con 0x7f5178070ee0 2023-07-13T22:19:58.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.191+0000 7f517c9fd700 1 -- 172.21.15.26:0/178524534 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168017650 con 0x7f5178070ee0 2023-07-13T22:19:58.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.192+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51781ac800 con 0x7f5178070ee0 2023-07-13T22:19:58.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.192+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51781acc50 con 0x7f5178070ee0 2023-07-13T22:19:58.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.200+0000 7f519135bb80 1 Processor -- start 2023-07-13T22:19:58.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.200+0000 7f519135bb80 1 -- start start 2023-07-13T22:19:58.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x55579101f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x5557910210e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790f24cb0 con 0x55579101fe20 2023-07-13T22:19:58.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790f77a40 con 0x555791020c70 2023-07-13T22:19:58.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f519135bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790f70ed0 con 0x55579101f4d0 2023-07-13T22:19:58.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.201+0000 7f51777fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x55579101f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.202+0000 7f51767fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x5557910210e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.202+0000 7f51767fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x5557910210e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:44998/0 (socket says 172.21.15.26:44998) 2023-07-13T22:19:58.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.202+0000 7f51767fc700 1 -- 172.21.15.26:0/2683803956 learned_addr learned my addr 172.21.15.26:0/2683803956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:58.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:57 smithi117 ceph-mon[129074]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 82 op/s 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f51777fe700 1 -- 172.21.15.26:0/2683803956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 msgr2=0x5557910210e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f51777fe700 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x5557910210e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f51777fe700 1 -- 172.21.15.26:0/2683803956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 msgr2=0x5557910206e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f51777fe700 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910206e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f51777fe700 1 -- 172.21.15.26:0/2683803956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555790d9ede0 con 0x55579101f4d0 2023-07-13T22:19:58.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.203+0000 7f517c9fd700 1 -- 172.21.15.26:0/178524534 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f516800f860 con 0x7f5178070ee0 2023-07-13T22:19:58.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.206+0000 7f517c9fd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 0x7f51600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.207+0000 7f517c9fd700 1 -- 172.21.15.26:0/178524534 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f51680cda30 con 0x7f5178070ee0 2023-07-13T22:19:58.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.212+0000 7f51777fe700 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x55579101f8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f516c00ba50 tx=0x7f516c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c4a696581a94b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.213+0000 7f5176ffd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 0x7f51600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.220+0000 7f5165ffb700 1 -- 172.21.15.26:0/2683803956 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c004220 con 0x55579101f4d0 2023-07-13T22:19:58.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.220+0000 7f5165ffb700 1 -- 172.21.15.26:0/2683803956 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516c00b500 con 0x55579101f4d0 2023-07-13T22:19:58.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.220+0000 7f5165ffb700 1 -- 172.21.15.26:0/2683803956 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c00dd50 con 0x55579101f4d0 2023-07-13T22:19:58.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.222+0000 7f519135bb80 1 -- 172.21.15.26:0/2683803956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 msgr2=0x55579101f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.222+0000 7f519135bb80 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x55579101f8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f516c00ba50 tx=0x7f516c00bd60 comp rx=0 tx=0).stop 2023-07-13T22:19:58.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.229+0000 7f5176ffd700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 0x7f51600a3800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51680979a0 tx=0x7f516800fc00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.246+0000 7f519135bb80 1 -- 172.21.15.26:0/2683803956 shutdown_connections 2023-07-13T22:19:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.246+0000 7f519135bb80 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x5557910210e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.246+0000 7f519135bb80 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x55579101f8e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.246+0000 7f519135bb80 1 --2- 172.21.15.26:0/2683803956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.246+0000 7f519135bb80 1 -- 172.21.15.26:0/2683803956 >> 172.21.15.26:0/2683803956 conn(0x555791014da0 msgr2=0x55579101a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.249+0000 7f519135bb80 1 -- 172.21.15.26:0/2683803956 shutdown_connections 2023-07-13T22:19:58.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.249+0000 7f519135bb80 1 -- 172.21.15.26:0/2683803956 wait complete. 2023-07-13T22:19:58.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.250+0000 7f519135bb80 1 -- start start 2023-07-13T22:19:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.250+0000 7f519135bb80 1 Processor -- start 2023-07-13T22:19:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x5557910291a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x555791030720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790f10ac0 con 0x55579101fe20 2023-07-13T22:19:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790f107c0 con 0x555791020c70 2023-07-13T22:19:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555790df4920 con 0x55579101f4d0 2023-07-13T22:19:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.251+0000 7f519135bb80 10 client.0 _ll_register_callbacks cb 0x555790f0e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.252+0000 7f51767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.252+0000 7f51777fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x5557910291a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.252+0000 7f51767fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910296e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52770/0 (socket says 172.21.15.26:52770) 2023-07-13T22:19:58.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.252+0000 7f51767fc700 1 -- 172.21.15.26:0/1845254306 learned_addr learned my addr 172.21.15.26:0/1845254306 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:58.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.252+0000 7f51777fe700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x555791030720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:57 smithi121 ceph-mon[129187]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 268 MiB/s wr, 82 op/s 2023-07-13T22:19:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.253+0000 7f51777fe700 1 -- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 msgr2=0x555791030720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.253+0000 7f51777fe700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x555791030720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.253+0000 7f51777fe700 1 -- 172.21.15.26:0/1845254306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 msgr2=0x5557910296e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.253+0000 7f51777fe700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910296e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.253+0000 7f51777fe700 1 -- 172.21.15.26:0/1845254306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c00b680 con 0x55579101f4d0 2023-07-13T22:19:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.255+0000 7f51777fe700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x5557910291a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f516c00bfd0 tx=0x7f516c00d7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c2ea5aa845be49 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.256+0000 7f515affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:19:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.256+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c01e070 con 0x55579101f4d0 2023-07-13T22:19:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.256+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f516c004320 con 0x55579101f4d0 2023-07-13T22:19:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.257+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555790f69eb0 con 0x55579101f4d0 2023-07-13T22:19:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.257+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516c0044e0 con 0x55579101f4d0 2023-07-13T22:19:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.257+0000 7f519135bb80 10 client.26401 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.257+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555790d87c60 con 0x55579101f4d0 2023-07-13T22:19:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555790f69eb0 con 0x55579101f4d0 2023-07-13T22:19:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f516c0054c0 con 0x55579101f4d0 2023-07-13T22:19:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 10 client.26401 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 20 client.26401 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791034e50 0x555791037260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 4 client.26401 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:19:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.258+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555790f2c050 con 0x555791034e50 2023-07-13T22:19:58.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.259+0000 7f51767fc700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791034e50 0x555791037260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.260+0000 7f51767fc700 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791034e50 0x555791037260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c0d594cb0bc5bf server_cookie=2aef7dc37696145a in_seq=0 out_seq=0 2023-07-13T22:19:58.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.260+0000 7f515affd700 10 client.26401 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:58.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.262+0000 7f515affd700 1 -- 172.21.15.26:0/1845254306 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555790f2c050 con 0x555791034e50 2023-07-13T22:19:58.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.262+0000 7f515affd700 10 client.26401 handle_command_reply: tid=0 2023-07-13T22:19:58.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.263+0000 7f519135bb80 1 client.26401 shutdown 2023-07-13T22:19:58.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.263+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 msgr2=0x5557910291a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.263+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x5557910291a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f516c00bfd0 tx=0x7f516c00d7d0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.263+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 shutdown_connections 2023-07-13T22:19:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.263+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791034e50 0x555791037260 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.264+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555791020c70 0x555791030720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.264+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55579101f4d0 0x5557910291a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.264+0000 7f519135bb80 1 --2- 172.21.15.26:0/1845254306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x5557910296e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.264+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 >> 172.21.15.26:0/1845254306 conn(0x555791014da0 msgr2=0x5557910198b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.264+0000 7f515affd700 0 client.26401 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:58.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.270+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 shutdown_connections 2023-07-13T22:19:58.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.272+0000 7f519135bb80 1 -- 172.21.15.26:0/1845254306 wait complete. 2023-07-13T22:19:58.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.283+0000 7f519135bb80 20 client.26401 trim_cache size 0 max 16384 2023-07-13T22:19:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.314+0000 7f519135bb80 1 Processor -- start 2023-07-13T22:19:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.314+0000 7f519135bb80 1 -- start start 2023-07-13T22:19:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.314+0000 7f519135bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101f4d0 0x555791036e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.314+0000 7f519135bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101fe20 0x5557910306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.314+0000 7f519135bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.315+0000 7f519135bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555791028d70 con 0x55579101f4d0 2023-07-13T22:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.315+0000 7f519135bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555791014970 con 0x55579101fe20 2023-07-13T22:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.315+0000 7f519135bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579101b4c0 con 0x555791020c70 2023-07-13T22:19:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.315+0000 7f51777fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.316+0000 7f51777fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51206/0 (socket says 172.21.15.26:51206) 2023-07-13T22:19:58.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.316+0000 7f51777fe700 1 -- 172.21.15.26:0/3423160338 learned_addr learned my addr 172.21.15.26:0/3423160338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:58.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.316+0000 7f51767fc700 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101fe20 0x5557910306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.325+0000 7f51777fe700 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101f4d0 0x555791036e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.326+0000 7f51777fe700 1 -- 172.21.15.26:0/3423160338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101fe20 msgr2=0x5557910306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.326+0000 7f51777fe700 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101fe20 0x5557910306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.326+0000 7f51777fe700 1 -- 172.21.15.26:0/3423160338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101f4d0 msgr2=0x555791036e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.326+0000 7f51777fe700 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101f4d0 0x555791036e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.326+0000 7f51777fe700 1 -- 172.21.15.26:0/3423160338 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555790dc5250 con 0x555791020c70 2023-07-13T22:19:58.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.328+0000 7f51777fe700 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f516c00b970 tx=0x7f516c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e7d9646144033f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.329+0000 7f51667fc700 1 -- 172.21.15.26:0/3423160338 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c01e070 con 0x555791020c70 2023-07-13T22:19:58.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.329+0000 7f51667fc700 1 -- 172.21.15.26:0/3423160338 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516c00d980 con 0x555791020c70 2023-07-13T22:19:58.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.329+0000 7f51667fc700 1 -- 172.21.15.26:0/3423160338 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c014030 con 0x555791020c70 2023-07-13T22:19:58.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.330+0000 7f519135bb80 1 -- 172.21.15.26:0/3423160338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 msgr2=0x555791027100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.330+0000 7f519135bb80 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f516c00b970 tx=0x7f516c00bd60 comp rx=0 tx=0).stop 2023-07-13T22:19:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.335+0000 7f519135bb80 1 -- 172.21.15.26:0/3423160338 shutdown_connections 2023-07-13T22:19:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.335+0000 7f519135bb80 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101f4d0 0x555791036e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.335+0000 7f519135bb80 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x555791027100 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.335+0000 7f519135bb80 1 --2- 172.21.15.26:0/3423160338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101fe20 0x5557910306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.335+0000 7f519135bb80 1 -- 172.21.15.26:0/3423160338 >> 172.21.15.26:0/3423160338 conn(0x555791014da0 msgr2=0x555791027e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.337+0000 7f519135bb80 1 -- 172.21.15.26:0/3423160338 shutdown_connections 2023-07-13T22:19:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.337+0000 7f519135bb80 1 -- 172.21.15.26:0/3423160338 wait complete. 2023-07-13T22:19:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.338+0000 7f519135bb80 1 -- start start 2023-07-13T22:19:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.338+0000 7f519135bb80 1 Processor -- start 2023-07-13T22:19:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x55579102c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x55579102f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579102f6a0 con 0x55579101fe20 2023-07-13T22:19:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579102f810 con 0x55579101f4d0 2023-07-13T22:19:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55579102f9d0 con 0x555791020c70 2023-07-13T22:19:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.339+0000 7f519135bb80 10 client.0 _ll_register_callbacks cb 0x7f516c00ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:19:58.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:45018/0 (socket says 172.21.15.26:45018) 2023-07-13T22:19:58.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 -- 172.21.15.26:0/3358010792 learned_addr learned my addr 172.21.15.26:0/3358010792 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:19:58.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x55579102f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 -- 172.21.15.26:0/3358010792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 msgr2=0x55579102f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x55579102f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 -- 172.21.15.26:0/3358010792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 msgr2=0x55579102c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:19:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x55579102c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.348+0000 7f51777fe700 1 -- 172.21.15.26:0/3358010792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555790d87c60 con 0x55579101f4d0 2023-07-13T22:19:58.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.350+0000 7f51777fe700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f516c003ff0 tx=0x7f516c005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5445d1a696782aab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:19:58.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.350+0000 7f515affd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:19:58.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.350+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c01e030 con 0x55579101f4d0 2023-07-13T22:19:58.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.350+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555790eca4f0 con 0x55579101f4d0 2023-07-13T22:19:58.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.350+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f516c000e40 con 0x55579101f4d0 2023-07-13T22:19:58.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.351+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516c014070 con 0x55579101f4d0 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.351+0000 7f519135bb80 10 client.26435 fetch_fsmap learned FSMap version 29 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.351+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555790d9d2b0 con 0x55579101f4d0 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.351+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555790eca4f0 con 0x55579101f4d0 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.351+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f516c045070 con 0x55579101f4d0 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 10 client.26435 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:19:58.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 10 client.26435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:19:58.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 20 client.26435 populate_metadata read hostname 'smithi026' 2023-07-13T22:19:58.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791099fc0 0x55579109c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:19:58.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 4 client.26435 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:19:58.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555790dbff70 con 0x555791099fc0 2023-07-13T22:19:58.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.352+0000 7f51767fc700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791099fc0 0x55579109c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:19:58.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.353+0000 7f51767fc700 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791099fc0 0x55579109c3a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a5d2f5bee2d79f1 server_cookie=856a5c3ffdf5bee5 in_seq=0 out_seq=0 2023-07-13T22:19:58.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.353+0000 7f515affd700 10 client.26435 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:58.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.353+0000 7f515affd700 1 -- 172.21.15.26:0/3358010792 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555790dbff70 con 0x555791099fc0 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.353+0000 7f515affd700 10 client.26435 handle_command_reply: tid=0 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.353+0000 7f519135bb80 1 client.26435 shutdown 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 msgr2=0x55579102bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f516c003ff0 tx=0x7f516c005a40 comp rx=0 tx=0).stop 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 shutdown_connections 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555791099fc0 0x55579109c3a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555791020c70 0x55579102f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55579101f4d0 0x55579102bc00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 --2- 172.21.15.26:0/3358010792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55579101fe20 0x55579102c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 >> 172.21.15.26:0/3358010792 conn(0x555791014da0 msgr2=0x555791036e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f515affd700 0 client.26435 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:19:58.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 shutdown_connections 2023-07-13T22:19:58.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.354+0000 7f519135bb80 1 -- 172.21.15.26:0/3358010792 wait complete. 2023-07-13T22:19:58.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.355+0000 7f519135bb80 20 client.26435 trim_cache size 0 max 16384 2023-07-13T22:19:58.344 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:19:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.356+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 msgr2=0x7f51600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.356+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 0x7f51600a3800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f51680979a0 tx=0x7f516800fc00 comp rx=0 tx=0).stop 2023-07-13T22:19:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.356+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 msgr2=0x7f51781a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:19:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.356+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5168006010 tx=0x7f5168003cb0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 shutdown_connections 2023-07-13T22:19:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f51600a1350 0x7f51600a3800 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517810bd60 0x7f51781abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5178070ee0 0x7f51781a4730 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 --2- 172.21.15.26:0/178524534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5178071830 0x7f51781a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:19:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 >> 172.21.15.26:0/178524534 conn(0x7f517806ca90 msgr2=0x7f517806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:19:58.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 shutdown_connections 2023-07-13T22:19:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:19:58.357+0000 7f517d9ff700 1 -- 172.21.15.26:0/178524534 wait complete. 2023-07-13T22:19:59.384 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:19:59.384 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:19:59.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:19:59 smithi026 ceph-mon[95647]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:20:00.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.091+0000 7f4a747ff700 1 -- 172.21.15.26:0/3068409675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c105860 msgr2=0x7f4a6c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.091+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3068409675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c105860 0x7f4a6c107c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b0a0 tx=0x7f4a6400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.091+0000 7f4a747ff700 1 -- 172.21.15.26:0/3068409675 shutdown_connections 2023-07-13T22:20:00.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.091+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3068409675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6c100620 0x7f4a6c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.092+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3068409675 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c105860 0x7f4a6c107c40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.092+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3068409675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c102f40 0x7f4a6c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.092+0000 7f4a747ff700 1 -- 172.21.15.26:0/3068409675 >> 172.21.15.26:0/3068409675 conn(0x7f4a6c0fa000 msgr2=0x7f4a6c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.092+0000 7f4a747ff700 1 -- 172.21.15.26:0/3068409675 shutdown_connections 2023-07-13T22:20:00.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.092+0000 7f4a747ff700 1 -- 172.21.15.26:0/3068409675 wait complete. 2023-07-13T22:20:00.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.093+0000 7f4a747ff700 1 Processor -- start 2023-07-13T22:20:00.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.093+0000 7f4a747ff700 1 -- start start 2023-07-13T22:20:00.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c102f40 0x7f4a6c0fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6c105860 0x7f4a6c0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c1a5c90 con 0x7f4a6c105860 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0ff050 con 0x7f4a6c102f40 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a747ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c0ff210 con 0x7f4a6c100620 2023-07-13T22:20:00.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a7259b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a7259b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51240/0 (socket says 172.21.15.26:51240) 2023-07-13T22:20:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.094+0000 7f4a7259b700 1 -- 172.21.15.26:0/3946895053 learned_addr learned my addr 172.21.15.26:0/3946895053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c102f40 0x7f4a6c0fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a7259b700 1 -- 172.21.15.26:0/3946895053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c102f40 msgr2=0x7f4a6c0fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a7259b700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c102f40 0x7f4a6c0fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a7259b700 1 -- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6c105860 msgr2=0x7f4a6c0feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a7259b700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6c105860 0x7f4a6c0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.095+0000 7f4a7259b700 1 -- 172.21.15.26:0/3946895053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c009520 con 0x7f4a6c100620 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.096+0000 7f4a7259b700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e9a0 tx=0x7f4a6800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.096+0000 7f4a637fe700 1 -- 172.21.15.26:0/3946895053 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68009790 con 0x7f4a6c100620 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.097+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a64009d00 con 0x7f4a6c100620 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.097+0000 7f4a637fe700 1 -- 172.21.15.26:0/3946895053 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6800cbd0 con 0x7f4a6c100620 2023-07-13T22:20:00.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.097+0000 7f4a637fe700 1 -- 172.21.15.26:0/3946895053 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68018680 con 0x7f4a6c100620 2023-07-13T22:20:00.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.097+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c070d20 con 0x7f4a6c100620 2023-07-13T22:20:00.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.101+0000 7f4a637fe700 1 -- 172.21.15.26:0/3946895053 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4a6800cd40 con 0x7f4a6c100620 2023-07-13T22:20:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.104+0000 7f4a880b1b80 1 Processor -- start 2023-07-13T22:20:00.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 -- start start 2023-07-13T22:20:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c70760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0b74d30 con 0x5627b0c6f550 2023-07-13T22:20:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0bc7ac0 con 0x5627b0c6fea0 2023-07-13T22:20:00.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.105+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0bc0f50 con 0x5627b0c70cf0 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.106+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.106+0000 7f4a72d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.106+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51254/0 (socket says 172.21.15.26:51254) 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.106+0000 7f4a72d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52802/0 (socket says 172.21.15.26:52802) 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.106+0000 7f4a71d9a700 1 -- 172.21.15.26:0/522083949 learned_addr learned my addr 172.21.15.26:0/522083949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 -- 172.21.15.26:0/522083949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 msgr2=0x5627b0c70760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a7259b700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c70760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c70760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 -- 172.21.15.26:0/522083949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 msgr2=0x5627b0c6f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 -- 172.21.15.26:0/522083949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b09eee60 con 0x5627b0c70cf0 2023-07-13T22:20:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f10 tx=0x7f4a5c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54dac21af8914a71 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a7259b700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:00.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.107+0000 7f4a60ff9700 1 -- 172.21.15.26:0/522083949 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c019630 con 0x5627b0c70cf0 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.108+0000 7f4a60ff9700 1 -- 172.21.15.26:0/522083949 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5c019c50 con 0x5627b0c70cf0 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.108+0000 7f4a60ff9700 1 -- 172.21.15.26:0/522083949 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00ed80 con 0x5627b0c70cf0 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.108+0000 7f4a637fe700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 0x7f4a580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.108+0000 7f4a637fe700 1 -- 172.21.15.26:0/3946895053 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a68014070 con 0x7f4a6c100620 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 -- 172.21.15.26:0/522083949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 msgr2=0x5627b0c71160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f10 tx=0x7f4a5c011720 comp rx=0 tx=0).stop 2023-07-13T22:20:00.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 0x7f4a580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 -- 172.21.15.26:0/522083949 shutdown_connections 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c6f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c71160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/522083949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c70760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 -- 172.21.15.26:0/522083949 >> 172.21.15.26:0/522083949 conn(0x5627b0c64e20 msgr2=0x5627b0c6a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 -- 172.21.15.26:0/522083949 shutdown_connections 2023-07-13T22:20:00.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.109+0000 7f4a880b1b80 1 -- 172.21.15.26:0/522083949 wait complete. 2023-07-13T22:20:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.110+0000 7f4a880b1b80 1 -- start start 2023-07-13T22:20:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.111+0000 7f4a880b1b80 1 Processor -- start 2023-07-13T22:20:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.111+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 0x7f4a580a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b780 tx=0x7f4a64009780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 0x5627b0c80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0b60b40 con 0x5627b0c6f550 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0b60840 con 0x5627b0c70cf0 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0a449a0 con 0x5627b0c6fea0 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.112+0000 7f4a880b1b80 10 client.0 _ll_register_callbacks cb 0x5627b0b5e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51262/0 (socket says 172.21.15.26:51262) 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2068817806 learned_addr learned my addr 172.21.15.26:0/2068817806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a7259b700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 0x5627b0c80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 msgr2=0x5627b0c80790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 0x5627b0c80790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2068817806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 msgr2=0x5627b0c79210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c79210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.113+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2068817806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c108810 con 0x5627b0c6fea0 2023-07-13T22:20:00.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009440 tx=0x7f4a5c009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f76be67dbc2a3fd server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 0x5627b0c80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c004230 con 0x5627b0c6fea0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627b0b1a570 con 0x5627b0c6fea0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.114+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a5c004390 con 0x5627b0c6fea0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.115+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a5c004550 con 0x5627b0c6fea0 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.115+0000 7f4a880b1b80 10 client.26419 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:00.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.115+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f4a5c009b70 con 0x5627b0c6fea0 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627b0b1a570 con 0x5627b0c6fea0 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4a5c004750 con 0x5627b0c6fea0 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 10 client.26419 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 10 client.26419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 20 client.26419 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0c84da0 0x5627b0c871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 4 client.26419 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:00.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627b0bb9f30 con 0x5627b0c84da0 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.116+0000 7f4a7259b700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0c84da0 0x5627b0c871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.117+0000 7f4a55ffb700 10 client.26419 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.117+0000 7f4a7259b700 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0c84da0 0x5627b0c871b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ae7f80a5b0e9b1 server_cookie=b635469e6e8c4d68 in_seq=0 out_seq=0 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.117+0000 7f4a55ffb700 1 -- 172.21.15.26:0/2068817806 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627b0bb9f30 con 0x5627b0c84da0 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.117+0000 7f4a55ffb700 10 client.26419 handle_command_reply: tid=0 2023-07-13T22:20:00.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 client.26419 shutdown 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 msgr2=0x5627b0c79750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009440 tx=0x7f4a5c009470 comp rx=0 tx=0).stop 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 shutdown_connections 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c79210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0c84da0 0x5627b0c871b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c70cf0 0x5627b0c80790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2068817806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c6fea0 0x5627b0c79750 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 >> 172.21.15.26:0/2068817806 conn(0x5627b0c64e20 msgr2=0x5627b0c69930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a55ffb700 0 client.26419 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 shutdown_connections 2023-07-13T22:20:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.118+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2068817806 wait complete. 2023-07-13T22:20:00.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.119+0000 7f4a880b1b80 20 client.26419 trim_cache size 0 max 16384 2023-07-13T22:20:00.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.141+0000 7f4a880b1b80 1 Processor -- start 2023-07-13T22:20:00.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.141+0000 7f4a880b1b80 1 -- start start 2023-07-13T22:20:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c77f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0c6a040 con 0x5627b0c6f550 2023-07-13T22:20:00.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0c6b0f0 con 0x5627b0c6fea0 2023-07-13T22:20:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.142+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0c7a200 con 0x5627b0c70cf0 2023-07-13T22:20:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:45070/0 (socket says 172.21.15.26:45070) 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2461425519 learned_addr learned my addr 172.21.15.26:0/2461425519 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c77f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a7259b700 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2461425519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 msgr2=0x5627b0c77f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c77f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2461425519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 msgr2=0x5627b0c770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.143+0000 7f4a71d9a700 1 -- 172.21.15.26:0/2461425519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b0c6a4f0 con 0x5627b0c6fea0 2023-07-13T22:20:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.144+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f10 tx=0x7f4a5c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68592cffcbcf73a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.144+0000 7f4a617fa700 1 -- 172.21.15.26:0/2461425519 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c00e5c0 con 0x5627b0c6fea0 2023-07-13T22:20:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.144+0000 7f4a617fa700 1 -- 172.21.15.26:0/2461425519 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5c00e720 con 0x5627b0c6fea0 2023-07-13T22:20:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.144+0000 7f4a617fa700 1 -- 172.21.15.26:0/2461425519 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c019430 con 0x5627b0c6fea0 2023-07-13T22:20:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2461425519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 msgr2=0x5627b0c80750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f10 tx=0x7f4a5c011720 comp rx=0 tx=0).stop 2023-07-13T22:20:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2461425519 shutdown_connections 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c77f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/2461425519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c80750 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2461425519 >> 172.21.15.26:0/2461425519 conn(0x5627b0c64e20 msgr2=0x5627b0c69930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2461425519 shutdown_connections 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- 172.21.15.26:0/2461425519 wait complete. 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.145+0000 7f4a880b1b80 1 -- start start 2023-07-13T22:20:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.146+0000 7f4a880b1b80 1 Processor -- start 2023-07-13T22:20:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c7cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0c7d970 con 0x5627b0c6f550 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0c7dae0 con 0x5627b0c6fea0 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b0cd8c10 con 0x5627b0c70cf0 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.147+0000 7f4a880b1b80 10 client.0 _ll_register_callbacks cb 0x7f4a5c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.148+0000 7f4a72d9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.148+0000 7f4a72d9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51276/0 (socket says 172.21.15.26:51276) 2023-07-13T22:20:00.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.148+0000 7f4a72d9c700 1 -- 172.21.15.26:0/3789444007 learned_addr learned my addr 172.21.15.26:0/3789444007 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.148+0000 7f4a71d9a700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c7cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.149+0000 7f4a72d9c700 1 -- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 msgr2=0x5627b0c7cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.149+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c7cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.149+0000 7f4a72d9c700 1 -- 172.21.15.26:0/3789444007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 msgr2=0x5627b0c80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.149+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.149+0000 7f4a72d9c700 1 -- 172.21.15.26:0/3789444007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c009b70 con 0x5627b0c70cf0 2023-07-13T22:20:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.150+0000 7f4a72d9c700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b0a0 tx=0x7f4a64006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b13b70dea7c6bc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.151+0000 7f4a55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:00.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.151+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400e040 con 0x5627b0c70cf0 2023-07-13T22:20:00.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.151+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4a64023040 con 0x5627b0c70cf0 2023-07-13T22:20:00.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.151+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a5c019c50 con 0x5627b0c70cf0 2023-07-13T22:20:00.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.152+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a640054a0 con 0x5627b0c70cf0 2023-07-13T22:20:00.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.152+0000 7f4a880b1b80 10 client.26425 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.152+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5627b09d7ce0 con 0x5627b0c70cf0 2023-07-13T22:20:00.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.152+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a5c019c50 con 0x5627b0c70cf0 2023-07-13T22:20:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4a64005600 con 0x5627b0c70cf0 2023-07-13T22:20:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 10 client.26425 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 10 client.26425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 20 client.26425 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0cda570 0x5627b0cdc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 4 client.26425 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:20:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.154+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5627b0b7c0d0 con 0x5627b0cda570 2023-07-13T22:20:00.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.155+0000 7f4a7259b700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0cda570 0x5627b0cdc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.157+0000 7f4a7259b700 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0cda570 0x5627b0cdc980 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1454eac5d4af85be server_cookie=b674f8eb99333eeb in_seq=0 out_seq=0 2023-07-13T22:20:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.157+0000 7f4a55ffb700 10 client.26425 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.159+0000 7f4a55ffb700 1 -- 172.21.15.26:0/3789444007 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5627b0b7c0d0 con 0x5627b0cda570 2023-07-13T22:20:00.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.159+0000 7f4a55ffb700 10 client.26425 handle_command_reply: tid=0 2023-07-13T22:20:00.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.159+0000 7f4a880b1b80 1 client.26425 shutdown 2023-07-13T22:20:00.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.161+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 msgr2=0x5627b0c7d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.161+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b0a0 tx=0x7f4a64006000 comp rx=0 tx=0).stop 2023-07-13T22:20:00.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 shutdown_connections 2023-07-13T22:20:00.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627b0c6f550 0x5627b0c80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627b0cda570 0x5627b0cdc980 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627b0c70cf0 0x5627b0c7d430 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 --2- 172.21.15.26:0/3789444007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627b0c6fea0 0x5627b0c7cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 >> 172.21.15.26:0/3789444007 conn(0x5627b0c64e20 msgr2=0x5627b0c77f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.169+0000 7f4a55ffb700 0 client.26425 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.171+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 shutdown_connections 2023-07-13T22:20:00.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.171+0000 7f4a880b1b80 1 -- 172.21.15.26:0/3789444007 wait complete. 2023-07-13T22:20:00.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.172+0000 7f4a880b1b80 20 client.26425 trim_cache size 0 max 16384 2023-07-13T22:20:00.159 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:20:00.159 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:20:00.159 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "b02ba038-22a6-4e0e-a1ab-b6b4554e5da4", 2023-07-13T22:20:00.159 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:20:00.160 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:20:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.175+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 msgr2=0x7f4a580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.175+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 0x7f4a580a37d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b780 tx=0x7f4a64009780 comp rx=0 tx=0).stop 2023-07-13T22:20:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.175+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 msgr2=0x7f4a6c0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.175+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800e9a0 tx=0x7f4a6800ecb0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 shutdown_connections 2023-07-13T22:20:00.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4a580a1320 0x7f4a580a37d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a6c105860 0x7f4a6c0feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a6c100620 0x7f4a6c0fff90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 --2- 172.21.15.26:0/3946895053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a6c102f40 0x7f4a6c0fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.176+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 >> 172.21.15.26:0/3946895053 conn(0x7f4a6c0fa000 msgr2=0x7f4a6c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.178+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 shutdown_connections 2023-07-13T22:20:00.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.178+0000 7f4a747ff700 1 -- 172.21.15.26:0/3946895053 wait complete. 2023-07-13T22:20:00.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:19:59 smithi117 ceph-mon[129074]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:20:00.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:20:00.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:19:59 smithi121 ceph-mon[129187]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:20:00.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.842+0000 7f7f477ff700 1 -- 172.21.15.26:0/3447463430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40070ee0 msgr2=0x7f7f400712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.842+0000 7f7f477ff700 1 --2- 172.21.15.26:0/3447463430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40070ee0 0x7f7f400712f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009990 tx=0x7f7f3c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 -- 172.21.15.26:0/3447463430 shutdown_connections 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 --2- 172.21.15.26:0/3447463430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f4010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 --2- 172.21.15.26:0/3447463430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40070ee0 0x7f7f400712f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 --2- 172.21.15.26:0/3447463430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f40071830 0x7f7f4010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 -- 172.21.15.26:0/3447463430 >> 172.21.15.26:0/3447463430 conn(0x7f7f4006ca90 msgr2=0x7f7f4006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 -- 172.21.15.26:0/3447463430 shutdown_connections 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.843+0000 7f7f477ff700 1 -- 172.21.15.26:0/3447463430 wait complete. 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.845+0000 7f7f477ff700 1 Processor -- start 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.845+0000 7f7f477ff700 1 -- start start 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f40070ee0 0x7f7f40114c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 0x7f7f40115160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f401161a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40116770 con 0x7f7f40070ee0 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f401168e0 con 0x7f7f4010bd60 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f477ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40116aa0 con 0x7f7f40071830 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.846+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f401161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.847+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f401161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:45108/0 (socket says 172.21.15.26:45108) 2023-07-13T22:20:00.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.847+0000 7f7f45d9c700 1 -- 172.21.15.26:0/530669949 learned_addr learned my addr 172.21.15.26:0/530669949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.848+0000 7f7f44d9a700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 0x7f7f40115160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.848+0000 7f7f44d9a700 1 -- 172.21.15.26:0/530669949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 msgr2=0x7f7f401161a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.848+0000 7f7f44d9a700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f401161a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.848+0000 7f7f44d9a700 1 -- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f40070ee0 msgr2=0x7f7f40114c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.848+0000 7f7f44d9a700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f40070ee0 0x7f7f40114c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.849+0000 7f7f44d9a700 1 -- 172.21.15.26:0/530669949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0095f0 con 0x7f7f40071830 2023-07-13T22:20:00.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.849+0000 7f7f44d9a700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 0x7f7f40115160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f40072540 tx=0x7f7f3000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.850+0000 7f7f367fc700 1 -- 172.21.15.26:0/530669949 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011830 con 0x7f7f40071830 2023-07-13T22:20:00.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.850+0000 7f7f367fc700 1 -- 172.21.15.26:0/530669949 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f30011e50 con 0x7f7f40071830 2023-07-13T22:20:00.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.850+0000 7f7f367fc700 1 -- 172.21.15.26:0/530669949 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30010de0 con 0x7f7f40071830 2023-07-13T22:20:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.850+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f40116d20 con 0x7f7f40071830 2023-07-13T22:20:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.850+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f40117260 con 0x7f7f40071830 2023-07-13T22:20:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.861+0000 7f7f5b131b80 1 Processor -- start 2023-07-13T22:20:00.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.861+0000 7f7f5b131b80 1 -- start start 2023-07-13T22:20:00.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c184d0 0x55d6b8c188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c1a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8b1dcb0 con 0x55d6b8c184d0 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8b70a40 con 0x55d6b8c18e20 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.862+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8b69ed0 con 0x55d6b8c19c70 2023-07-13T22:20:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.870+0000 7f7f4559b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.870+0000 7f7f4559b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:45114/0 (socket says 172.21.15.26:45114) 2023-07-13T22:20:00.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.870+0000 7f7f4559b700 1 -- 172.21.15.26:0/1729154381 learned_addr learned my addr 172.21.15.26:0/1729154381 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.873+0000 7f7f4559b700 1 -- 172.21.15.26:0/1729154381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 msgr2=0x55d6b8c1a0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.873+0000 7f7f4559b700 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c1a0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.873+0000 7f7f4559b700 1 -- 172.21.15.26:0/1729154381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c184d0 msgr2=0x55d6b8c188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.873+0000 7f7f4559b700 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c184d0 0x55d6b8c188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.873+0000 7f7f4559b700 1 -- 172.21.15.26:0/1729154381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b8997de0 con 0x55d6b8c18e20 2023-07-13T22:20:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.875+0000 7f7f367fc700 1 -- 172.21.15.26:0/530669949 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7f30011990 con 0x7f7f40071830 2023-07-13T22:20:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.877+0000 7f7f367fc700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 0x7f7f2c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.878+0000 7f7f367fc700 1 -- 172.21.15.26:0/530669949 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f300cc210 con 0x7f7f40071830 2023-07-13T22:20:00.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.878+0000 7f7f4559b700 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009f50 tx=0x7f7f3c009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a6559e3fe6ee4e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.894+0000 7f7f2b7fe700 1 -- 172.21.15.26:0/1729154381 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01d070 con 0x55d6b8c18e20 2023-07-13T22:20:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.894+0000 7f7f2b7fe700 1 -- 172.21.15.26:0/1729154381 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3c0038a0 con 0x55d6b8c18e20 2023-07-13T22:20:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.894+0000 7f7f2b7fe700 1 -- 172.21.15.26:0/1729154381 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c005580 con 0x55d6b8c18e20 2023-07-13T22:20:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.895+0000 7f7f5b131b80 1 -- 172.21.15.26:0/1729154381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 msgr2=0x55d6b8c196e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.895+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009f50 tx=0x7f7f3c009f80 comp rx=0 tx=0).stop 2023-07-13T22:20:00.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.896+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 0x7f7f2c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.898+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 0x7f7f2c0a3800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f3804e690 tx=0x7f7f38050040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.900+0000 7f7f5b131b80 1 -- 172.21.15.26:0/1729154381 shutdown_connections 2023-07-13T22:20:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.900+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c184d0 0x55d6b8c188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.900+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c1a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.900+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/1729154381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c18e20 0x55d6b8c196e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.900+0000 7f7f5b131b80 1 -- 172.21.15.26:0/1729154381 >> 172.21.15.26:0/1729154381 conn(0x55d6b8c0dda0 msgr2=0x55d6b8c131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.901+0000 7f7f5b131b80 1 -- 172.21.15.26:0/1729154381 shutdown_connections 2023-07-13T22:20:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.901+0000 7f7f5b131b80 1 -- 172.21.15.26:0/1729154381 wait complete. 2023-07-13T22:20:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.902+0000 7f7f5b131b80 1 -- start start 2023-07-13T22:20:00.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.902+0000 7f7f5b131b80 1 Processor -- start 2023-07-13T22:20:00.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8b09ac0 con 0x55d6b8c19c70 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8b097c0 con 0x55d6b8c184d0 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b89ed920 con 0x55d6b8c18e20 2023-07-13T22:20:00.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.903+0000 7f7f5b131b80 10 client.0 _ll_register_callbacks cb 0x55d6b8b07760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.907+0000 7f7f4559b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.907+0000 7f7f4559b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c22140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:45128/0 (socket says 172.21.15.26:45128) 2023-07-13T22:20:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.907+0000 7f7f4559b700 1 -- 172.21.15.26:0/3488383230 learned_addr learned my addr 172.21.15.26:0/3488383230 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.910+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 -- 172.21.15.26:0/3488383230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 msgr2=0x55d6b8c22680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c22680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 -- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 msgr2=0x55d6b8c22140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c22140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 -- 172.21.15.26:0/3488383230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f40117640 con 0x55d6b8c19c70 2023-07-13T22:20:00.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.911+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c296c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7f3805a990 tx=0x7f7f38004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed183f1bf151576e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.915+0000 7f7f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:20:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.915+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3804dab0 con 0x55d6b8c19c70 2023-07-13T22:20:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.915+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f3804dc10 con 0x55d6b8c19c70 2023-07-13T22:20:00.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.916+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6b8ac34f0 con 0x55d6b8c19c70 2023-07-13T22:20:00.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.919+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f3804ddd0 con 0x55d6b8c19c70 2023-07-13T22:20:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.920+0000 7f7f5b131b80 10 client.16083 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:00.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.920+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d6b8980c60 con 0x55d6b8c19c70 2023-07-13T22:20:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.921+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d6b8ac34f0 con 0x55d6b8c19c70 2023-07-13T22:20:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.921+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7f3806d070 con 0x55d6b8c19c70 2023-07-13T22:20:00.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 20 client.16083 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c2dfb0 0x55d6b8c303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 4 client.16083 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:00.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6b8b62eb0 con 0x55d6b8c2dfb0 2023-07-13T22:20:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.922+0000 7f7f4559b700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c2dfb0 0x55d6b8c303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.923+0000 7f7f4559b700 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c2dfb0 0x55d6b8c303c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56cbb546d4a71b27 server_cookie=cd79f94fcfd96462 in_seq=0 out_seq=0 2023-07-13T22:20:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.923+0000 7f7f28ff9700 10 client.16083 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.924+0000 7f7f28ff9700 1 -- 172.21.15.26:0/3488383230 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d6b8b62eb0 con 0x55d6b8c2dfb0 2023-07-13T22:20:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.924+0000 7f7f28ff9700 10 client.16083 handle_command_reply: tid=0 2023-07-13T22:20:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.924+0000 7f7f5b131b80 1 client.16083 shutdown 2023-07-13T22:20:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.925+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 msgr2=0x55d6b8c296c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.925+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c296c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7f3805a990 tx=0x7f7f38004b40 comp rx=0 tx=0).stop 2023-07-13T22:20:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 shutdown_connections 2023-07-13T22:20:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c296c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c2dfb0 0x55d6b8c303c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c22140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/3488383230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 >> 172.21.15.26:0/3488383230 conn(0x55d6b8c0dda0 msgr2=0x55d6b8c128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f28ff9700 0 client.16083 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.926+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 shutdown_connections 2023-07-13T22:20:00.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.927+0000 7f7f5b131b80 1 -- 172.21.15.26:0/3488383230 wait complete. 2023-07-13T22:20:00.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.928+0000 7f7f5b131b80 20 client.16083 trim_cache size 0 max 16384 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 Processor -- start 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 -- start start 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c1ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c29680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c21cf0 con 0x55d6b8c19c70 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c14a30 con 0x55d6b8c184d0 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.977+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c0d9a0 con 0x55d6b8c18e20 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.978+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.978+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52878/0 (socket says 172.21.15.26:52878) 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.978+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2383322168 learned_addr learned my addr 172.21.15.26:0/2383322168 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2383322168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 msgr2=0x55d6b8c29680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c29680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2383322168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 msgr2=0x55d6b8c1ffb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c1ffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2383322168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b89be250 con 0x55d6b8c19c70 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.979+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7f3806e800 tx=0x7f7f3806e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7332db0852ca685e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.989+0000 7f7f2bfff700 1 -- 172.21.15.26:0/2383322168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3806e8b0 con 0x55d6b8c19c70 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.989+0000 7f7f2bfff700 1 -- 172.21.15.26:0/2383322168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f38054040 con 0x55d6b8c19c70 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.989+0000 7f7f2bfff700 1 -- 172.21.15.26:0/2383322168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38078420 con 0x55d6b8c19c70 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.990+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2383322168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 msgr2=0x55d6b8c29e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.990+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7f3806e800 tx=0x7f7f3806e3a0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.991+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2383322168 shutdown_connections 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.991+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c19c70 0x55d6b8c29e30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.991+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c1ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.991+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2383322168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c18e20 0x55d6b8c29680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.991+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2383322168 >> 172.21.15.26:0/2383322168 conn(0x55d6b8c0dda0 msgr2=0x55d6b8c2ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.994+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2383322168 shutdown_connections 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.995+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2383322168 wait complete. 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.996+0000 7f7f5b131b80 1 -- start start 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.996+0000 7f7f5b131b80 1 Processor -- start 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c25c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c29620 con 0x55d6b8c28c70 2023-07-13T22:20:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c29790 con 0x55d6b8c184d0 2023-07-13T22:20:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8c29950 con 0x55d6b8c19c70 2023-07-13T22:20:00.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.997+0000 7f7f5b131b80 10 client.0 _ll_register_callbacks cb 0x7f7f3804e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.998+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.998+0000 7f7f45d9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52884/0 (socket says 172.21.15.26:52884) 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.998+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2914066288 learned_addr learned my addr 172.21.15.26:0/2914066288 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f4559b700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f44d9a700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c25c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2914066288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 msgr2=0x55d6b8c25c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c25c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 msgr2=0x55d6b8c25710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c25710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:00.999+0000 7f7f45d9c700 1 -- 172.21.15.26:0/2914066288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6b8980c60 con 0x55d6b8c28c70 2023-07-13T22:20:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.000+0000 7f7f45d9c700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f3804ee60 tx=0x7f7f38004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbdc9c3f71f7511 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.000+0000 7f7f28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:20:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.004+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3806ed00 con 0x55d6b8c28c70 2023-07-13T22:20:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.004+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7f3806ee60 con 0x55d6b8c28c70 2023-07-13T22:20:00.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.004+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f3c0038a0 con 0x55d6b8c28c70 2023-07-13T22:20:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.005+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f38081450 con 0x55d6b8c28c70 2023-07-13T22:20:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.005+0000 7f7f5b131b80 10 client.16089 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:00.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.005+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d6b89962b0 con 0x55d6b8c28c70 2023-07-13T22:20:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.006+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f3c0038a0 con 0x55d6b8c28c70 2023-07-13T22:20:00.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.006+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7f3805aa70 con 0x55d6b8c28c70 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 20 client.16089 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c83120 0x55d6b8c85530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 4 client.16089 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:20:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.007+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d6b8b25050 con 0x55d6b8c83120 2023-07-13T22:20:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.008+0000 7f7f4559b700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c83120 0x55d6b8c85530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.009+0000 7f7f4559b700 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c83120 0x55d6b8c85530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293e0b492bb64bb4 server_cookie=d6eec9e1d2586e49 in_seq=0 out_seq=0 2023-07-13T22:20:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.009+0000 7f7f28ff9700 10 client.16089 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.009+0000 7f7f28ff9700 1 -- 172.21.15.26:0/2914066288 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d6b8b25050 con 0x55d6b8c83120 2023-07-13T22:20:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.009+0000 7f7f28ff9700 10 client.16089 handle_command_reply: tid=0 2023-07-13T22:20:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.009+0000 7f7f5b131b80 1 client.16089 shutdown 2023-07-13T22:20:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 msgr2=0x55d6b8c290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f3804ee60 tx=0x7f7f38004e20 comp rx=0 tx=0).stop 2023-07-13T22:20:00.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 shutdown_connections 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d6b8c28c70 0x55d6b8c290e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d6b8c83120 0x55d6b8c85530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d6b8c184d0 0x55d6b8c25710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 --2- 172.21.15.26:0/2914066288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d6b8c19c70 0x55d6b8c25c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 >> 172.21.15.26:0/2914066288 conn(0x55d6b8c0dda0 msgr2=0x55d6b8c29e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:00.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.010+0000 7f7f28ff9700 0 client.16089 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.011+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 shutdown_connections 2023-07-13T22:20:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.011+0000 7f7f5b131b80 1 -- 172.21.15.26:0/2914066288 wait complete. 2023-07-13T22:20:00.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.012+0000 7f7f5b131b80 20 client.16089 trim_cache size 0 max 16384 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout: "b02ba038-22a6-4e0e-a1ab-b6b4554e5da4": { 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:20:00.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "b02ba038-22a6-4e0e-a1ab-b6b4554e5da4", 2023-07-13T22:20:01.000 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:20:01.000 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:20:01.000 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:20:01.000 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.013+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 msgr2=0x7f7f2c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.013+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 0x7f7f2c0a3800 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f3804e690 tx=0x7f7f38050040 comp rx=0 tx=0).stop 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.013+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 msgr2=0x7f7f40115160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.013+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 0x7f7f40115160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7f40072540 tx=0x7f7f3000b730 comp rx=0 tx=0).stop 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 shutdown_connections 2023-07-13T22:20:01.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7f2c0a1350 0x7f7f2c0a3800 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f40070ee0 0x7f7f40114c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f4010bd60 0x7f7f401161a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 --2- 172.21.15.26:0/530669949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f40071830 0x7f7f40115160 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.014+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 >> 172.21.15.26:0/530669949 conn(0x7f7f4006ca90 msgr2=0x7f7f4006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.015+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 shutdown_connections 2023-07-13T22:20:01.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:01.015+0000 7f7f477ff700 1 -- 172.21.15.26:0/530669949 wait complete. 2023-07-13T22:20:01.041 INFO:tasks.cephfs.filesystem:scrub status for tag:b02ba038-22a6-4e0e-a1ab-b6b4554e5da4 - {'path': '/', 'tag': 'b02ba038-22a6-4e0e-a1ab-b6b4554e5da4', 'options': 'recursive,force'} 2023-07-13T22:20:01.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: overall HEALTH_OK 2023-07-13T22:20:01.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:20:01.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:20:01.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:20:01.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:00 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: overall HEALTH_OK 2023-07-13T22:20:01.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:20:01.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:20:01.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:20:01.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:00 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: overall HEALTH_OK 2023-07-13T22:20:01.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:20:01.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:20:01.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:20:01.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:01.241 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:00 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:02.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:01 smithi117 ceph-mon[129074]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:20:02.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:01 smithi026 ceph-mon[95647]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:20:02.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:01 smithi121 ceph-mon[129187]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 283 MiB/s wr, 88 op/s 2023-07-13T22:20:03.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:02 smithi121 ceph-mon[129187]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:20:03.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:02 smithi117 ceph-mon[129074]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:20:03.471 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:02 smithi026 ceph-mon[95647]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 274 MiB/s wr, 88 op/s 2023-07-13T22:20:05.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:04 smithi121 ceph-mon[129187]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:20:05.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:04 smithi117 ceph-mon[129074]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:20:05.476 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:04 smithi026 ceph-mon[95647]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 304 MiB/s wr, 96 op/s 2023-07-13T22:20:07.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:07 smithi117 ceph-mon[129074]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 84 op/s 2023-07-13T22:20:07.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:07 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:20:07.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:07 smithi026 ceph-mon[95647]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 84 op/s 2023-07-13T22:20:07.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:07 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:20:07.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:07 smithi121 ceph-mon[129187]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 265 MiB/s wr, 84 op/s 2023-07-13T22:20:07.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:07 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:20:08.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:08 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:20:08.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:08 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:20:08.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:08 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:20:09.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:09 smithi117 ceph-mon[129074]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:20:09.475 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:09 smithi026 ceph-mon[95647]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:20:09.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:09 smithi121 ceph-mon[129187]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 91 op/s 2023-07-13T22:20:11.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:11 smithi117 ceph-mon[129074]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 87 op/s 2023-07-13T22:20:11.454 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:11 smithi026 ceph-mon[95647]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 87 op/s 2023-07-13T22:20:11.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:11 smithi121 ceph-mon[129187]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 280 MiB/s wr, 87 op/s 2023-07-13T22:20:13.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:13 smithi117 ceph-mon[129074]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 87 op/s 2023-07-13T22:20:13.484 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:13 smithi026 ceph-mon[95647]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 87 op/s 2023-07-13T22:20:13.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:13 smithi121 ceph-mon[129187]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 87 op/s 2023-07-13T22:20:15.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:15 smithi121 ceph-mon[129187]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 309 MiB/s wr, 96 op/s 2023-07-13T22:20:15.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:15 smithi117 ceph-mon[129074]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 309 MiB/s wr, 96 op/s 2023-07-13T22:20:15.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:15 smithi026 ceph-mon[95647]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 309 MiB/s wr, 96 op/s 2023-07-13T22:20:17.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:17 smithi121 ceph-mon[129187]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 265 MiB/s wr, 82 op/s 2023-07-13T22:20:17.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:17 smithi117 ceph-mon[129074]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 265 MiB/s wr, 82 op/s 2023-07-13T22:20:17.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:17 smithi026 ceph-mon[95647]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 265 MiB/s wr, 82 op/s 2023-07-13T22:20:19.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:19 smithi117 ceph-mon[129074]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 281 MiB/s wr, 91 op/s 2023-07-13T22:20:19.703 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:19 smithi026 ceph-mon[95647]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 281 MiB/s wr, 91 op/s 2023-07-13T22:20:19.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:19 smithi121 ceph-mon[129187]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 281 MiB/s wr, 91 op/s 2023-07-13T22:20:21.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:21 smithi117 ceph-mon[129074]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 89 op/s 2023-07-13T22:20:21.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:21 smithi026 ceph-mon[95647]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 89 op/s 2023-07-13T22:20:21.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:21 smithi121 ceph-mon[129187]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 89 op/s 2023-07-13T22:20:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:23 smithi117 ceph-mon[129074]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:20:23.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:23 smithi026 ceph-mon[95647]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:20:23.740 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:23 smithi121 ceph-mon[129187]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:20:25.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:25 smithi117 ceph-mon[129074]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 95 op/s 2023-07-13T22:20:25.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:25 smithi026 ceph-mon[95647]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 95 op/s 2023-07-13T22:20:25.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:25 smithi121 ceph-mon[129187]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 95 op/s 2023-07-13T22:20:27.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:27 smithi121 ceph-mon[129187]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:27.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:27 smithi117 ceph-mon[129074]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:27.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:27 smithi026 ceph-mon[95647]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:29.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:29 smithi117 ceph-mon[129074]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:20:29.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:29 smithi026 ceph-mon[95647]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:20:29.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:29 smithi121 ceph-mon[129187]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:20:31.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:20:31.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.545+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/1155349094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb8071b20 msgr2=0x7f2eb810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.545+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/1155349094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb8071b20 0x7f2eb810b530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009990 tx=0x7f2eac009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.550+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/1155349094 shutdown_connections 2023-07-13T22:20:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.550+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/1155349094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.550+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/1155349094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2eb80711d0 0x7f2eb80715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.550+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/1155349094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb8071b20 0x7f2eb810b530 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.550+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/1155349094 >> 172.21.15.26:0/1155349094 conn(0x7f2eb806c8b0 msgr2=0x7f2eb806ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.551+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/1155349094 shutdown_connections 2023-07-13T22:20:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.551+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/1155349094 wait complete. 2023-07-13T22:20:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.553+0000 7f2ebdb9d700 1 Processor -- start 2023-07-13T22:20:31.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.553+0000 7f2ebdb9d700 1 -- start start 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb80711d0 0x7f2eb81a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2eb8071b20 0x7f2eb81a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81ac000 con 0x7f2eb810bba0 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81ac140 con 0x7f2eb80711d0 2023-07-13T22:20:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.554+0000 7f2ebdb9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81ac2d0 con 0x7f2eb8071b20 2023-07-13T22:20:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.555+0000 7f2ebd39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.555+0000 7f2ebd39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37986/0 (socket says 172.21.15.26:37986) 2023-07-13T22:20:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.555+0000 7f2ebd39c700 1 -- 172.21.15.26:0/3089620296 learned_addr learned my addr 172.21.15.26:0/3089620296 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 -- 172.21.15.26:0/3089620296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2eb8071b20 msgr2=0x7f2eb81a49c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:20:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2eb8071b20 0x7f2eb81a49c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 -- 172.21.15.26:0/3089620296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb80711d0 msgr2=0x7f2eb81a4480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:20:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb80711d0 0x7f2eb81a4480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 -- 172.21.15.26:0/3089620296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0095f0 con 0x7f2eb810bba0 2023-07-13T22:20:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.556+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000ca40 tx=0x7f2eb000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.557+0000 7f2eb5ffb700 1 -- 172.21.15.26:0/3089620296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000d040 con 0x7f2eb810bba0 2023-07-13T22:20:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.557+0000 7f2eb5ffb700 1 -- 172.21.15.26:0/3089620296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb0012470 con 0x7f2eb810bba0 2023-07-13T22:20:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.557+0000 7f2eb5ffb700 1 -- 172.21.15.26:0/3089620296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0011610 con 0x7f2eb810bba0 2023-07-13T22:20:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.557+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb81ac5b0 con 0x7f2eb810bba0 2023-07-13T22:20:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.557+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81aca70 con 0x7f2eb810bba0 2023-07-13T22:20:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.563+0000 7f2eb5ffb700 1 -- 172.21.15.26:0/3089620296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2eb00125e0 con 0x7f2eb810bba0 2023-07-13T22:20:31.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.568+0000 7f2eb5ffb700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 0x7f2ea00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.568+0000 7f2eb5ffb700 1 -- 172.21.15.26:0/3089620296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2eb00ccae0 con 0x7f2eb810bba0 2023-07-13T22:20:31.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.569+0000 7f2ed2673b80 1 Processor -- start 2023-07-13T22:20:31.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 -- start start 2023-07-13T22:20:31.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf421720 0x55f7cf421b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf422e30 0x55f7cf4232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf326cb0 con 0x55f7cf422070 2023-07-13T22:20:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf379a40 con 0x55f7cf421720 2023-07-13T22:20:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.570+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf372ed0 con 0x55f7cf422e30 2023-07-13T22:20:31.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.571+0000 7f2eb7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.571+0000 7f2eb7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37990/0 (socket says 172.21.15.26:37990) 2023-07-13T22:20:31.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.571+0000 7f2eb7fff700 1 -- 172.21.15.26:0/1055697622 learned_addr learned my addr 172.21.15.26:0/1055697622 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:31.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf422e30 0x55f7cf4232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 -- 172.21.15.26:0/1055697622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf422e30 msgr2=0x55f7cf4232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf422e30 0x55f7cf4232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 -- 172.21.15.26:0/1055697622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf421720 msgr2=0x55f7cf421b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:20:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf421720 0x55f7cf421b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 -- 172.21.15.26:0/1055697622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7cf1a0de0 con 0x55f7cf422070 2023-07-13T22:20:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.572+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de84ca69c5fa3561 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.581+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 0x7f2ea00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.583+0000 7f2e9effd700 1 -- 172.21.15.26:0/1055697622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac01d070 con 0x55f7cf422070 2023-07-13T22:20:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.583+0000 7f2e9effd700 1 -- 172.21.15.26:0/1055697622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eac0040c0 con 0x55f7cf422070 2023-07-13T22:20:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.583+0000 7f2e9effd700 1 -- 172.21.15.26:0/1055697622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00be40 con 0x55f7cf422070 2023-07-13T22:20:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.584+0000 7f2ed2673b80 1 -- 172.21.15.26:0/1055697622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 msgr2=0x55f7cf4228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.584+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2eac000c00 tx=0x7f2eac005ad0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.585+0000 7f2ed2673b80 1 -- 172.21.15.26:0/1055697622 shutdown_connections 2023-07-13T22:20:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.585+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf422e30 0x55f7cf4232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.585+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf421720 0x55f7cf421b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.585+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/1055697622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf4228a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.585+0000 7f2ed2673b80 1 -- 172.21.15.26:0/1055697622 >> 172.21.15.26:0/1055697622 conn(0x55f7cf416ff0 msgr2=0x55f7cf41c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.595+0000 7f2ed2673b80 1 -- 172.21.15.26:0/1055697622 shutdown_connections 2023-07-13T22:20:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.598+0000 7f2ed2673b80 1 -- 172.21.15.26:0/1055697622 wait complete. 2023-07-13T22:20:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.599+0000 7f2ed2673b80 1 -- start start 2023-07-13T22:20:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.599+0000 7f2ed2673b80 1 Processor -- start 2023-07-13T22:20:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf43c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf43c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf4438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf312ac0 con 0x55f7cf422e30 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf3127c0 con 0x55f7cf422070 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf1f6920 con 0x55f7cf421720 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ed2673b80 10 client.0 _ll_register_callbacks cb 0x55f7cf310760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.622+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 0x7f2ea00a38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8072710 tx=0x7f2ea8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.623+0000 7f2ebcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf43c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.623+0000 7f2ebcb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf43c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:45722/0 (socket says 172.21.15.26:45722) 2023-07-13T22:20:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.623+0000 7f2ebcb9b700 1 -- 172.21.15.26:0/37011388 learned_addr learned my addr 172.21.15.26:0/37011388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:31.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.623+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf4438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.623+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf43c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 -- 172.21.15.26:0/37011388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 msgr2=0x55f7cf43c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf43c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 -- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 msgr2=0x55f7cf43c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf43c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 -- 172.21.15.26:0/37011388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb8117590 con 0x55f7cf422e30 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.624+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf4438c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000a390 tx=0x7f2eb0018340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b2876d2ae9ae9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.625+0000 7f2e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:20:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.625+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb009ded0 con 0x55f7cf422e30 2023-07-13T22:20:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.625+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2eb00a22f0 con 0x55f7cf422e30 2023-07-13T22:20:31.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.632+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7cf2cc4f0 con 0x55f7cf422e30 2023-07-13T22:20:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.634+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eb00b82a0 con 0x55f7cf422e30 2023-07-13T22:20:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.635+0000 7f2ed2673b80 10 client.16098 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.635+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7cf189c60 con 0x55f7cf422e30 2023-07-13T22:20:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.635+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7cf2cc4f0 con 0x55f7cf422e30 2023-07-13T22:20:31.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2eb00a1ef0 con 0x55f7cf422e30 2023-07-13T22:20:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 20 client.16098 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf448110 0x55f7cf44a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 4 client.16098 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.637+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7cf36beb0 con 0x55f7cf448110 2023-07-13T22:20:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.638+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf448110 0x55f7cf44a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.638+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf448110 0x55f7cf44a520 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c2ee7eca64a828 server_cookie=3698dbde00b2e379 in_seq=0 out_seq=0 2023-07-13T22:20:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.638+0000 7f2e97fff700 10 client.16098 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.639+0000 7f2e97fff700 1 -- 172.21.15.26:0/37011388 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7cf36beb0 con 0x55f7cf448110 2023-07-13T22:20:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.639+0000 7f2e97fff700 10 client.16098 handle_command_reply: tid=0 2023-07-13T22:20:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.640+0000 7f2ed2673b80 1 client.16098 shutdown 2023-07-13T22:20:31.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.640+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 msgr2=0x55f7cf4438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.640+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf4438c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000a390 tx=0x7f2eb0018340 comp rx=0 tx=0).stop 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 shutdown_connections 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf4438c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf448110 0x55f7cf44a520 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf43c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/37011388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf43c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 >> 172.21.15.26:0/37011388 conn(0x55f7cf416ff0 msgr2=0x55f7cf41bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.641+0000 7f2e97fff700 0 client.16098 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.642+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 shutdown_connections 2023-07-13T22:20:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.642+0000 7f2ed2673b80 1 -- 172.21.15.26:0/37011388 wait complete. 2023-07-13T22:20:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.642+0000 7f2ed2673b80 20 client.16098 trim_cache size 0 max 16384 2023-07-13T22:20:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 Processor -- start 2023-07-13T22:20:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 -- start start 2023-07-13T22:20:31.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf443880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf2cc4f0 con 0x55f7cf422e30 2023-07-13T22:20:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf416bc0 con 0x55f7cf422070 2023-07-13T22:20:31.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.682+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf416e00 con 0x55f7cf421720 2023-07-13T22:20:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.684+0000 7f2ebd39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.684+0000 7f2ebd39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38018/0 (socket says 172.21.15.26:38018) 2023-07-13T22:20:31.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.684+0000 7f2ebd39c700 1 -- 172.21.15.26:0/2622063477 learned_addr learned my addr 172.21.15.26:0/2622063477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 -- 172.21.15.26:0/2622063477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 msgr2=0x55f7cf4292e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:31.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4292e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 -- 172.21.15.26:0/2622063477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 msgr2=0x55f7cf443880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf443880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 -- 172.21.15.26:0/2622063477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7cf4234d0 con 0x55f7cf422e30 2023-07-13T22:20:31.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.686+0000 7f2ebd39c700 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0096910 tx=0x7f2eb0007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b692a8bfd659eb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.687+0000 7f2e9f7fe700 1 -- 172.21.15.26:0/2622063477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb009ded0 con 0x55f7cf422e30 2023-07-13T22:20:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.687+0000 7f2e9f7fe700 1 -- 172.21.15.26:0/2622063477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb00a22f0 con 0x55f7cf422e30 2023-07-13T22:20:31.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.687+0000 7f2e9f7fe700 1 -- 172.21.15.26:0/2622063477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb00adba0 con 0x55f7cf422e30 2023-07-13T22:20:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.688+0000 7f2ed2673b80 1 -- 172.21.15.26:0/2622063477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 msgr2=0x55f7cf43c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.688+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0096910 tx=0x7f2eb0007410 comp rx=0 tx=0).stop 2023-07-13T22:20:31.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 -- 172.21.15.26:0/2622063477 shutdown_connections 2023-07-13T22:20:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422e30 0x55f7cf43c340 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/2622063477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422070 0x55f7cf443880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 -- 172.21.15.26:0/2622063477 >> 172.21.15.26:0/2622063477 conn(0x55f7cf416ff0 msgr2=0x55f7cf43afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.689+0000 7f2ed2673b80 1 -- 172.21.15.26:0/2622063477 shutdown_connections 2023-07-13T22:20:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.690+0000 7f2ed2673b80 1 -- 172.21.15.26:0/2622063477 wait complete. 2023-07-13T22:20:31.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.690+0000 7f2ed2673b80 1 -- start start 2023-07-13T22:20:31.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 Processor -- start 2023-07-13T22:20:31.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422e30 0x55f7cf442d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf443250 con 0x55f7cf422070 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf4433c0 con 0x55f7cf422e30 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7cf443580 con 0x55f7cf421720 2023-07-13T22:20:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.691+0000 7f2ed2673b80 10 client.0 _ll_register_callbacks cb 0x55f7cf310760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.692+0000 7f2eb7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.692+0000 7f2eb7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38032/0 (socket says 172.21.15.26:38032) 2023-07-13T22:20:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.692+0000 7f2eb7fff700 1 -- 172.21.15.26:0/188115563 learned_addr learned my addr 172.21.15.26:0/188115563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.693+0000 7f2eb7fff700 1 -- 172.21.15.26:0/188115563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 msgr2=0x55f7cf4417c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.693+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4417c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.693+0000 7f2eb7fff700 1 -- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422e30 msgr2=0x55f7cf442d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.693+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422e30 0x55f7cf442d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.693+0000 7f2eb7fff700 1 -- 172.21.15.26:0/188115563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7cf189c60 con 0x55f7cf422070 2023-07-13T22:20:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.694+0000 7f2eb7fff700 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2eac003f50 tx=0x7f2eac00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e670564b18a51130 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.694+0000 7f2e97fff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:20:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.696+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac01d040 con 0x55f7cf422070 2023-07-13T22:20:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.696+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2eac003bd0 con 0x55f7cf422070 2023-07-13T22:20:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.696+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2eac0040c0 con 0x55f7cf422070 2023-07-13T22:20:31.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.706+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eac00f460 con 0x55f7cf422070 2023-07-13T22:20:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.718+0000 7f2ed2673b80 10 client.16104 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.718+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f7cf1c7250 con 0x55f7cf422070 2023-07-13T22:20:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.739+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eac0040c0 con 0x55f7cf422070 2023-07-13T22:20:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.739+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2eac032070 con 0x55f7cf422070 2023-07-13T22:20:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:31.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 20 client.16104 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf49cb90 0x55f7cf49efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 4 client.16104 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:20:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.745+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f7cf32e050 con 0x55f7cf49cb90 2023-07-13T22:20:31.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.746+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf49cb90 0x55f7cf49efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.748+0000 7f2ebcb9b700 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf49cb90 0x55f7cf49efa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09982a31a3cf50c server_cookie=30f63131a2574254 in_seq=0 out_seq=0 2023-07-13T22:20:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.748+0000 7f2e97fff700 10 client.16104 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.749+0000 7f2e97fff700 1 -- 172.21.15.26:0/188115563 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f7cf32e050 con 0x55f7cf49cb90 2023-07-13T22:20:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.749+0000 7f2e97fff700 10 client.16104 handle_command_reply: tid=0 2023-07-13T22:20:31.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.749+0000 7f2ed2673b80 1 client.16104 shutdown 2023-07-13T22:20:31.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.765+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 msgr2=0x55f7cf441d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.765+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2eac003f50 tx=0x7f2eac00ba10 comp rx=0 tx=0).stop 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 shutdown_connections 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55f7cf49cb90 0x55f7cf49efa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7cf422e30 0x55f7cf442d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7cf421720 0x55f7cf4417c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 --2- 172.21.15.26:0/188115563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f7cf422070 0x55f7cf441d00 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2eac003f50 tx=0x7f2eac00ba10 comp rx=0 tx=0).stop 2023-07-13T22:20:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 >> 172.21.15.26:0/188115563 conn(0x55f7cf416ff0 msgr2=0x55f7cf4292e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.766+0000 7f2e97fff700 0 client.16104 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:31.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.790+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 shutdown_connections 2023-07-13T22:20:31.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.794+0000 7f2ed2673b80 1 -- 172.21.15.26:0/188115563 wait complete. 2023-07-13T22:20:31.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.794+0000 7f2ed2673b80 20 client.16104 trim_cache size 0 max 16384 2023-07-13T22:20:31.779 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:20:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:20:31.780 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:20:31.780 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:20:31.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.814+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 msgr2=0x7f2ea00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.814+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 0x7f2ea00a38e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8072710 tx=0x7f2ea8009260 comp rx=0 tx=0).stop 2023-07-13T22:20:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.814+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 msgr2=0x7f2eb81aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.814+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000ca40 tx=0x7f2eb000ce00 comp rx=0 tx=0).stop 2023-07-13T22:20:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 shutdown_connections 2023-07-13T22:20:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2ea00a1430 0x7f2ea00a38e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eb810bba0 0x7f2eb81aba30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2eb80711d0 0x7f2eb81a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 --2- 172.21.15.26:0/3089620296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2eb8071b20 0x7f2eb81a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.820+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 >> 172.21.15.26:0/3089620296 conn(0x7f2eb806c8b0 msgr2=0x7f2eb806daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.840+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 shutdown_connections 2023-07-13T22:20:31.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:31.840+0000 7f2ebdb9d700 1 -- 172.21.15.26:0/3089620296 wait complete. 2023-07-13T22:20:31.911 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:31 smithi026 ceph-mon[95647]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 90 op/s 2023-07-13T22:20:31.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:20:31.913 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:20:31.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:31 smithi117 ceph-mon[129074]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 90 op/s 2023-07-13T22:20:31.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:31 smithi121 ceph-mon[129187]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 294 MiB/s wr, 90 op/s 2023-07-13T22:20:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.589+0000 7fabfdbff700 1 -- 172.21.15.26:0/1695056445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 msgr2=0x7fabf810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:32.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.589+0000 7fabfdbff700 1 --2- 172.21.15.26:0/1695056445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf810b7d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fabec009990 tx=0x7fabec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.600+0000 7fabfdbff700 1 -- 172.21.15.26:0/1695056445 shutdown_connections 2023-07-13T22:20:32.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.600+0000 7fabfdbff700 1 --2- 172.21.15.26:0/1695056445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.600+0000 7fabfdbff700 1 --2- 172.21.15.26:0/1695056445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8070ed0 0x7fabf80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.600+0000 7fabfdbff700 1 --2- 172.21.15.26:0/1695056445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf810b7d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.600+0000 7fabfdbff700 1 -- 172.21.15.26:0/1695056445 >> 172.21.15.26:0/1695056445 conn(0x7fabf806ca40 msgr2=0x7fabf806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:32.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.601+0000 7fabfdbff700 1 -- 172.21.15.26:0/1695056445 shutdown_connections 2023-07-13T22:20:32.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.601+0000 7fabfdbff700 1 -- 172.21.15.26:0/1695056445 wait complete. 2023-07-13T22:20:32.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.603+0000 7fabfdbff700 1 Processor -- start 2023-07-13T22:20:32.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.603+0000 7fabfdbff700 1 -- start start 2023-07-13T22:20:32.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8070ed0 0x7fabf81a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:32.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf81a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:32.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:32.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81ac3b0 con 0x7fabf8071820 2023-07-13T22:20:32.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81ac4f0 con 0x7fabf8070ed0 2023-07-13T22:20:32.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.604+0000 7fabfdbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf81ac680 con 0x7fabf810bd60 2023-07-13T22:20:32.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:32.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:45760/0 (socket says 172.21.15.26:45760) 2023-07-13T22:20:32.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 -- 172.21.15.26:0/4139436496 learned_addr learned my addr 172.21.15.26:0/4139436496 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:32.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf6ffd700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf81a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:32.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 -- 172.21.15.26:0/4139436496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8070ed0 msgr2=0x7fabf81a4830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:20:32.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8070ed0 0x7fabf81a4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 -- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 msgr2=0x7fabf81a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:32.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.605+0000 7fabf7fff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf81a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:32.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.606+0000 7fabf7fff700 1 -- 172.21.15.26:0/4139436496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0095f0 con 0x7fabf810bd60 2023-07-13T22:20:32.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.606+0000 7fabf7fff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b780 tx=0x7fabf000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:32.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.606+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0010860 con 0x7fabf810bd60 2023-07-13T22:20:32.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.606+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabf0010e80 con 0x7fabf810bd60 2023-07-13T22:20:32.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.606+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0007370 con 0x7fabf810bd60 2023-07-13T22:20:32.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.607+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf81ac960 con 0x7fabf810bd60 2023-07-13T22:20:32.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.607+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf81acea0 con 0x7fabf810bd60 2023-07-13T22:20:32.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.610+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf811a270 con 0x7fabf810bd60 2023-07-13T22:20:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.616+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fabf000f640 con 0x7fabf810bd60 2023-07-13T22:20:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.619+0000 7fabf4ff9700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 0x7fabe00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:32.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.619+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fabf00cc890 con 0x7fabf810bd60 2023-07-13T22:20:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.621+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fabf0096010 con 0x7fabf810bd60 2023-07-13T22:20:32.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.622+0000 7fabf77fe700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 0x7fabe00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:32.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:32.629+0000 7fabf77fe700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 0x7fabe00a3870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009710 tx=0x7fabe8006c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:33.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.297+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabf811a460 con 0x7fabf810bd60 2023-07-13T22:20:33.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.299+0000 7fabf4ff9700 1 -- 172.21.15.26:0/4139436496 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fabf0014020 con 0x7fabf810bd60 2023-07-13T22:20:33.291 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:20:33.291 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 msgr2=0x7fabe00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 0x7fabe00a3870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009710 tx=0x7fabe8006c60 comp rx=0 tx=0).stop 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 msgr2=0x7fabf81abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fabf000b780 tx=0x7fabf000ba90 comp rx=0 tx=0).stop 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 shutdown_connections 2023-07-13T22:20:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fabf810bd60 0x7fabf81abde0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabf8070ed0 0x7fabf81a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fabe00a13c0 0x7fabe00a3870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 --2- 172.21.15.26:0/4139436496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabf8071820 0x7fabf81a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:33.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.318+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 >> 172.21.15.26:0/4139436496 conn(0x7fabf806ca40 msgr2=0x7fabf806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:33.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.319+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 shutdown_connections 2023-07-13T22:20:33.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:33.319+0000 7fabfdbff700 1 -- 172.21.15.26:0/4139436496 wait complete. 2023-07-13T22:20:33.306 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:20:33.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:20:33.640 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:33 smithi026 ceph-mon[95647]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 88 op/s 2023-07-13T22:20:33.641 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:33 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/4139436496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:20:33.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:33 smithi117 ceph-mon[129074]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 88 op/s 2023-07-13T22:20:33.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:33 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/4139436496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:20:33.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:33 smithi121 ceph-mon[129187]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 282 MiB/s wr, 88 op/s 2023-07-13T22:20:33.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:33 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/4139436496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:20:34.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.237+0000 7efe6d3ff700 1 -- 172.21.15.26:0/917336245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6810bd60 msgr2=0x7efe6810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.237+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/917336245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6810bd60 0x7efe6810e1f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efe58009990 tx=0x7efe58009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 -- 172.21.15.26:0/917336245 shutdown_connections 2023-07-13T22:20:34.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/917336245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe6810bd60 0x7efe6810e1f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/917336245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 0x7efe680712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/917336245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe68071830 0x7efe6810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 -- 172.21.15.26:0/917336245 >> 172.21.15.26:0/917336245 conn(0x7efe6806ca90 msgr2=0x7efe6806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 -- 172.21.15.26:0/917336245 shutdown_connections 2023-07-13T22:20:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.238+0000 7efe6d3ff700 1 -- 172.21.15.26:0/917336245 wait complete. 2023-07-13T22:20:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.240+0000 7efe6d3ff700 1 Processor -- start 2023-07-13T22:20:34.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.240+0000 7efe6d3ff700 1 -- start start 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 0x7efe681a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6810bd60 0x7efe68078770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68078d40 con 0x7efe6810bd60 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68078eb0 con 0x7efe68070ee0 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe6d3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68079070 con 0x7efe68071830 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.241+0000 7efe5e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.242+0000 7efe5e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52118/0 (socket says 172.21.15.26:52118) 2023-07-13T22:20:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.242+0000 7efe5e7fc700 1 -- 172.21.15.26:0/3367171700 learned_addr learned my addr 172.21.15.26:0/3367171700 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.242+0000 7efe66ffd700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 0x7efe681a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.242+0000 7efe677fe700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6810bd60 0x7efe68078770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 -- 172.21.15.26:0/3367171700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 msgr2=0x7efe681a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 0x7efe681a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 -- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6810bd60 msgr2=0x7efe68078770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6810bd60 0x7efe68078770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 -- 172.21.15.26:0/3367171700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580095f0 con 0x7efe68071830 2023-07-13T22:20:34.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.243+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d980 tx=0x7efe5000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.255+0000 7efe64ff9700 1 -- 172.21.15.26:0/3367171700 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5000de60 con 0x7efe68071830 2023-07-13T22:20:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.256+0000 7efe64ff9700 1 -- 172.21.15.26:0/3367171700 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe500044e0 con 0x7efe68071830 2023-07-13T22:20:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.256+0000 7efe64ff9700 1 -- 172.21.15.26:0/3367171700 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe50003b10 con 0x7efe68071830 2023-07-13T22:20:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.256+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe680792f0 con 0x7efe68071830 2023-07-13T22:20:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.256+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe680797b0 con 0x7efe68071830 2023-07-13T22:20:34.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.265+0000 7efe80c47b80 1 Processor -- start 2023-07-13T22:20:34.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.265+0000 7efe80c47b80 1 -- start start 2023-07-13T22:20:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f1a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f1ac70 0x55ec06f1b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06e1ecb0 con 0x55ec06f19e20 2023-07-13T22:20:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06e71a40 con 0x55ec06f194d0 2023-07-13T22:20:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.266+0000 7efe80c47b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06e6aed0 con 0x55ec06f1ac70 2023-07-13T22:20:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.267+0000 7efe677fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.267+0000 7efe677fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55176/0 (socket says 172.21.15.26:55176) 2023-07-13T22:20:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.267+0000 7efe677fe700 1 -- 172.21.15.26:0/2999249558 learned_addr learned my addr 172.21.15.26:0/2999249558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f1a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe677fe700 1 -- 172.21.15.26:0/2999249558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f1ac70 msgr2=0x55ec06f1b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:20:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe677fe700 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f1ac70 0x55ec06f1b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe677fe700 1 -- 172.21.15.26:0/2999249558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 msgr2=0x55ec06f1a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe677fe700 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f1a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.268+0000 7efe677fe700 1 -- 172.21.15.26:0/2999249558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec06c98de0 con 0x55ec06f194d0 2023-07-13T22:20:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.269+0000 7efe677fe700 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b2e0 tx=0x7efe58005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd83ab0e289bb3df server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.270+0000 7efe5d7fa700 1 -- 172.21.15.26:0/2999249558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5800be60 con 0x55ec06f194d0 2023-07-13T22:20:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.270+0000 7efe5d7fa700 1 -- 172.21.15.26:0/2999249558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5801e040 con 0x55ec06f194d0 2023-07-13T22:20:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.270+0000 7efe5d7fa700 1 -- 172.21.15.26:0/2999249558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58004cf0 con 0x55ec06f194d0 2023-07-13T22:20:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 -- 172.21.15.26:0/2999249558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 msgr2=0x55ec06f198e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b2e0 tx=0x7efe58005ea0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 -- 172.21.15.26:0/2999249558 shutdown_connections 2023-07-13T22:20:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f1ac70 0x55ec06f1b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f198e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2999249558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f1a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.271+0000 7efe80c47b80 1 -- 172.21.15.26:0/2999249558 >> 172.21.15.26:0/2999249558 conn(0x55ec06f0eda0 msgr2=0x55ec06f141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.272+0000 7efe64ff9700 1 -- 172.21.15.26:0/3367171700 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efe50003c70 con 0x7efe68071830 2023-07-13T22:20:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.274+0000 7efe64ff9700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 0x7efe540a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.274+0000 7efe64ff9700 1 -- 172.21.15.26:0/3367171700 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe500cc5f0 con 0x7efe68071830 2023-07-13T22:20:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.277+0000 7efe80c47b80 1 -- 172.21.15.26:0/2999249558 shutdown_connections 2023-07-13T22:20:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.277+0000 7efe80c47b80 1 -- 172.21.15.26:0/2999249558 wait complete. 2023-07-13T22:20:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.278+0000 7efe80c47b80 1 -- start start 2023-07-13T22:20:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 Processor -- start 2023-07-13T22:20:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f23810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f2ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06e0aac0 con 0x55ec06f28840 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.279+0000 7efe80c47b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06e0a7c0 con 0x55ec06f194d0 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.280+0000 7efe80c47b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06cee920 con 0x55ec06f19e20 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.280+0000 7efe80c47b80 10 client.0 _ll_register_callbacks cb 0x55ec06e08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.282+0000 7efe66ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.282+0000 7efe66ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f232d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55178/0 (socket says 172.21.15.26:55178) 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.282+0000 7efe66ffd700 1 -- 172.21.15.26:0/2848988696 learned_addr learned my addr 172.21.15.26:0/2848988696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.282+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f23810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.283+0000 7efe5e7fc700 1 -- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 msgr2=0x55ec06f232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.283+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.283+0000 7efe5e7fc700 1 -- 172.21.15.26:0/2848988696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 msgr2=0x55ec06f2ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.283+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f2ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.283+0000 7efe5e7fc700 1 -- 172.21.15.26:0/2848988696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68117640 con 0x55ec06f19e20 2023-07-13T22:20:34.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.284+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f23810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ec06f1a610 tx=0x7efe500057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b06e2a7b5fa8e79 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.290+0000 7efe4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.290+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe500990a0 con 0x55ec06f19e20 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.290+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe500a1e00 con 0x55ec06f19e20 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.290+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec06dc44f0 con 0x55ec06f19e20 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.292+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe500b20f0 con 0x55ec06f19e20 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.292+0000 7efe80c47b80 10 client.26452 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.292+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ec06c81c60 con 0x55ec06f19e20 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.293+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec06dc44f0 con 0x55ec06f19e20 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.293+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe50004020 con 0x55ec06f19e20 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.294+0000 7efe80c47b80 10 client.26452 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.294+0000 7efe80c47b80 10 client.26452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.295+0000 7efe80c47b80 20 client.26452 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.295+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f40480 0x55ec06f42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.295+0000 7efe80c47b80 4 client.26452 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.295+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec06e63eb0 con 0x55ec06f40480 2023-07-13T22:20:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.296+0000 7efe66ffd700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 0x7efe540a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.297+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f40480 0x55ec06f42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.298+0000 7efe66ffd700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 0x7efe540a36e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe60007ac0 tx=0x7efe600074c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.298+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f40480 0x55ec06f42890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f6c3527347b5846 server_cookie=486b7fc171744351 in_seq=0 out_seq=0 2023-07-13T22:20:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.298+0000 7efe4e7fc700 10 client.26452 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:20:34.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.299+0000 7efe4e7fc700 1 -- 172.21.15.26:0/2848988696 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec06e63eb0 con 0x55ec06f40480 2023-07-13T22:20:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.299+0000 7efe4e7fc700 10 client.26452 handle_command_reply: tid=0 2023-07-13T22:20:34.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.300+0000 7efe80c47b80 1 client.26452 shutdown 2023-07-13T22:20:34.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.304+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 msgr2=0x55ec06f23810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.304+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f23810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ec06f1a610 tx=0x7efe500057e0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 shutdown_connections 2023-07-13T22:20:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f40480 0x55ec06f42890 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f2ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2848988696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f23810 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 >> 172.21.15.26:0/2848988696 conn(0x55ec06f0eda0 msgr2=0x55ec06f138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.316+0000 7efe4e7fc700 0 client.26452 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:20:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.317+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 shutdown_connections 2023-07-13T22:20:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.317+0000 7efe80c47b80 1 -- 172.21.15.26:0/2848988696 wait complete. 2023-07-13T22:20:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.317+0000 7efe80c47b80 20 client.26452 trim_cache size 0 max 16384 2023-07-13T22:20:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.345+0000 7efe80c47b80 1 Processor -- start 2023-07-13T22:20:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.345+0000 7efe80c47b80 1 -- start start 2023-07-13T22:20:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f20f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f400d0 con 0x55ec06f28840 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f155c0 con 0x55ec06f194d0 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.346+0000 7efe80c47b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f42ba0 con 0x55ec06f19e20 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55190/0 (socket says 172.21.15.26:55190) 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 -- 172.21.15.26:0/2752752595 learned_addr learned my addr 172.21.15.26:0/2752752595 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe5e7fc700 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 -- 172.21.15.26:0/2752752595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 msgr2=0x55ec06f500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 -- 172.21.15.26:0/2752752595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 msgr2=0x55ec06f20f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f20f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.347+0000 7efe66ffd700 1 -- 172.21.15.26:0/2752752595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec06cbf250 con 0x55ec06f194d0 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.348+0000 7efe66ffd700 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe6001a660 tx=0x7efe60020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e83daf0ab26ab8f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.367+0000 7efe5dffb700 1 -- 172.21.15.26:0/2752752595 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6002b070 con 0x55ec06f194d0 2023-07-13T22:20:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.368+0000 7efe5dffb700 1 -- 172.21.15.26:0/2752752595 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe6000f040 con 0x55ec06f194d0 2023-07-13T22:20:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.368+0000 7efe5dffb700 1 -- 172.21.15.26:0/2752752595 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe600272f0 con 0x55ec06f194d0 2023-07-13T22:20:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.369+0000 7efe80c47b80 1 -- 172.21.15.26:0/2752752595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 msgr2=0x55ec06f22980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.369+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efe6001a660 tx=0x7efe60020000 comp rx=0 tx=0).stop 2023-07-13T22:20:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.412+0000 7efe80c47b80 1 -- 172.21.15.26:0/2752752595 shutdown_connections 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.412+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f28840 0x55ec06f20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.412+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f194d0 0x55ec06f22980 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.412+0000 7efe80c47b80 1 --2- 172.21.15.26:0/2752752595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f19e20 0x55ec06f500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.412+0000 7efe80c47b80 1 -- 172.21.15.26:0/2752752595 >> 172.21.15.26:0/2752752595 conn(0x55ec06f0eda0 msgr2=0x55ec06f42490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.421+0000 7efe80c47b80 1 -- 172.21.15.26:0/2752752595 shutdown_connections 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.421+0000 7efe80c47b80 1 -- 172.21.15.26:0/2752752595 wait complete. 2023-07-13T22:20:34.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.422+0000 7efe80c47b80 1 -- start start 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.422+0000 7efe80c47b80 1 Processor -- start 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f194d0 0x55ec06f4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 0x55ec06f930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f935e0 con 0x55ec06f19e20 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f93750 con 0x55ec06f28840 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec06f93910 con 0x55ec06f194d0 2023-07-13T22:20:34.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.423+0000 7efe80c47b80 10 client.0 _ll_register_callbacks cb 0x7efe500994f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.426+0000 7efe5e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.426+0000 7efe5e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f4fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45252/0 (socket says 172.21.15.26:45252) 2023-07-13T22:20:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.426+0000 7efe5e7fc700 1 -- 172.21.15.26:0/432444903 learned_addr learned my addr 172.21.15.26:0/432444903 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:34.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.426+0000 7efe66ffd700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f194d0 0x55ec06f4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.426+0000 7efe677fe700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 0x55ec06f930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.427+0000 7efe677fe700 1 -- 172.21.15.26:0/432444903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f194d0 msgr2=0x55ec06f4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.427+0000 7efe677fe700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f194d0 0x55ec06f4f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.427+0000 7efe677fe700 1 -- 172.21.15.26:0/432444903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 msgr2=0x55ec06f4fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.427+0000 7efe677fe700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f4fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.427+0000 7efe677fe700 1 -- 172.21.15.26:0/432444903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe600098a0 con 0x55ec06f28840 2023-07-13T22:20:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.428+0000 7efe677fe700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 0x55ec06f930a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b680 tx=0x7efe58005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88710823480fbd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.428+0000 7efe4e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:20:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.441+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58004700 con 0x55ec06f28840 2023-07-13T22:20:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.441+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efe58004860 con 0x55ec06f28840 2023-07-13T22:20:34.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.442+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe5801e040 con 0x55ec06f28840 2023-07-13T22:20:34.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.449+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe58004a40 con 0x55ec06f28840 2023-07-13T22:20:34.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.452+0000 7efe80c47b80 10 client.26477 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:34.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.452+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ec06c972b0 con 0x55ec06f28840 2023-07-13T22:20:34.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.453+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe5801e040 con 0x55ec06f28840 2023-07-13T22:20:34.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.453+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efe58034070 con 0x55ec06f28840 2023-07-13T22:20:34.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.453+0000 7efe80c47b80 10 client.26477 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:34.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.453+0000 7efe80c47b80 10 client.26477 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:20:34.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.453+0000 7efe80c47b80 20 client.26477 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:34.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.454+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f94d00 0x55ec06f970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:34.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.454+0000 7efe80c47b80 4 client.26477 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:20:34.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.454+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec06e26050 con 0x55ec06f94d00 2023-07-13T22:20:34.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.456+0000 7efe66ffd700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f94d00 0x55ec06f970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.457+0000 7efe66ffd700 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f94d00 0x55ec06f970e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6047e5271f0cdd47 server_cookie=786fb202f862c1a6 in_seq=0 out_seq=0 2023-07-13T22:20:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.458+0000 7efe4e7fc700 10 client.26477 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:20:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.458+0000 7efe4e7fc700 1 -- 172.21.15.26:0/432444903 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec06e26050 con 0x55ec06f94d00 2023-07-13T22:20:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.458+0000 7efe4e7fc700 10 client.26477 handle_command_reply: tid=0 2023-07-13T22:20:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.458+0000 7efe80c47b80 1 client.26477 shutdown 2023-07-13T22:20:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 msgr2=0x55ec06f930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 0x55ec06f930a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efe5800b680 tx=0x7efe58005270 comp rx=0 tx=0).stop 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 shutdown_connections 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ec06f94d00 0x55ec06f970e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec06f28840 0x55ec06f930a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec06f194d0 0x55ec06f4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 --2- 172.21.15.26:0/432444903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec06f19e20 0x55ec06f4fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 >> 172.21.15.26:0/432444903 conn(0x55ec06f0eda0 msgr2=0x55ec06f20f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.460+0000 7efe4e7fc700 0 client.26477 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:20:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.462+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 shutdown_connections 2023-07-13T22:20:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.462+0000 7efe80c47b80 1 -- 172.21.15.26:0/432444903 wait complete. 2023-07-13T22:20:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.468+0000 7efe80c47b80 20 client.26477 trim_cache size 0 max 16384 2023-07-13T22:20:34.453 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:20:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 msgr2=0x7efe540a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 0x7efe540a36e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efe60007ac0 tx=0x7efe600074c0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 msgr2=0x7efe681a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efe5000d980 tx=0x7efe5000dd40 comp rx=0 tx=0).stop 2023-07-13T22:20:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 shutdown_connections 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efe540a1230 0x7efe540a36e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe6810bd60 0x7efe68078770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe68070ee0 0x7efe681a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 --2- 172.21.15.26:0/3367171700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe68071830 0x7efe681a4b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.469+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 >> 172.21.15.26:0/3367171700 conn(0x7efe6806ca90 msgr2=0x7efe6806dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.470+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 shutdown_connections 2023-07-13T22:20:34.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:34.470+0000 7efe6d3ff700 1 -- 172.21.15.26:0/3367171700 wait complete. 2023-07-13T22:20:34.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:20:35.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/2507612034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c810bd60 msgr2=0x7fe8c810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/2507612034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c810bd60 0x7fe8c810e1f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b0a0 tx=0x7fe8b800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/2507612034 shutdown_connections 2023-07-13T22:20:35.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/2507612034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8c8070ee0 0x7fe8c80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/2507612034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c810bd60 0x7fe8c810e1f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/2507612034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8071830 0x7fe8c810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.157+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/2507612034 >> 172.21.15.26:0/2507612034 conn(0x7fe8c806ca90 msgr2=0x7fe8c806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.171+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/2507612034 shutdown_connections 2023-07-13T22:20:35.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.171+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/2507612034 wait complete. 2023-07-13T22:20:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.173+0000 7fe8ce3ff700 1 Processor -- start 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 -- start start 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8071830 0x7fe8c8114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8c810bd60 0x7fe8c81a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8114650 con 0x7fe8c810bd60 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81147c0 con 0x7fe8c8071830 2023-07-13T22:20:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.174+0000 7fe8ce3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c81a3e00 con 0x7fe8c8070ee0 2023-07-13T22:20:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.183+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.187+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52170/0 (socket says 172.21.15.26:52170) 2023-07-13T22:20:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.187+0000 7fe8c7fff700 1 -- 172.21.15.26:0/1195359808 learned_addr learned my addr 172.21.15.26:0/1195359808 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.188+0000 7fe8c7fff700 1 -- 172.21.15.26:0/1195359808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8071830 msgr2=0x7fe8c8114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.188+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8071830 0x7fe8c8114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.188+0000 7fe8c7fff700 1 -- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8c810bd60 msgr2=0x7fe8c81a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.188+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8c810bd60 0x7fe8c81a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.188+0000 7fe8c7fff700 1 -- 172.21.15.26:0/1195359808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009d00 con 0x7fe8c8070ee0 2023-07-13T22:20:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.192+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000d950 tx=0x7fe8c000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.192+0000 7fe8c57fa700 1 -- 172.21.15.26:0/1195359808 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0009b40 con 0x7fe8c8070ee0 2023-07-13T22:20:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.193+0000 7fe8c57fa700 1 -- 172.21.15.26:0/1195359808 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c0009ca0 con 0x7fe8c8070ee0 2023-07-13T22:20:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.193+0000 7fe8c57fa700 1 -- 172.21.15.26:0/1195359808 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c000f620 con 0x7fe8c8070ee0 2023-07-13T22:20:35.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.194+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c81a40e0 con 0x7fe8c8070ee0 2023-07-13T22:20:35.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.194+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81a4570 con 0x7fe8c8070ee0 2023-07-13T22:20:35.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.202+0000 7fe8e1ca0b80 1 Processor -- start 2023-07-13T22:20:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 -- start start 2023-07-13T22:20:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa445208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa445216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44425cb0 con 0x55aa445204d0 2023-07-13T22:20:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44478a40 con 0x55aa44521c70 2023-07-13T22:20:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.203+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44471ed0 con 0x55aa44520e20 2023-07-13T22:20:35.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.204+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.204+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55222/0 (socket says 172.21.15.26:55222) 2023-07-13T22:20:35.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.204+0000 7fe8c77fe700 1 -- 172.21.15.26:0/1767846032 learned_addr learned my addr 172.21.15.26:0/1767846032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.205+0000 7fe8c77fe700 1 -- 172.21.15.26:0/1767846032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 msgr2=0x55aa445216e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:20:35.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.205+0000 7fe8c77fe700 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa445216e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.205+0000 7fe8c77fe700 1 -- 172.21.15.26:0/1767846032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 msgr2=0x55aa445208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.205+0000 7fe8c77fe700 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa445208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.205+0000 7fe8c77fe700 1 -- 172.21.15.26:0/1767846032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa4429fde0 con 0x55aa44521c70 2023-07-13T22:20:35.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.206+0000 7fe8c77fe700 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000bb70 tx=0x7fe8b000be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657993ee8bd97ec6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.207+0000 7fe8be7fc700 1 -- 172.21.15.26:0/1767846032 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000d890 con 0x55aa44521c70 2023-07-13T22:20:35.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.207+0000 7fe8be7fc700 1 -- 172.21.15.26:0/1767846032 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b0010460 con 0x55aa44521c70 2023-07-13T22:20:35.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.207+0000 7fe8be7fc700 1 -- 172.21.15.26:0/1767846032 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0018620 con 0x55aa44521c70 2023-07-13T22:20:35.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.208+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/1767846032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 msgr2=0x55aa445220e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.208+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000bb70 tx=0x7fe8b000be80 comp rx=0 tx=0).stop 2023-07-13T22:20:35.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.211+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/1767846032 shutdown_connections 2023-07-13T22:20:35.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.211+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa445208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.211+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa445220e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.211+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/1767846032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa445216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.211+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/1767846032 >> 172.21.15.26:0/1767846032 conn(0x55aa44515da0 msgr2=0x55aa4451b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.212+0000 7fe8c57fa700 1 -- 172.21.15.26:0/1195359808 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe8c0010460 con 0x7fe8c8070ee0 2023-07-13T22:20:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.214+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/1767846032 shutdown_connections 2023-07-13T22:20:35.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.215+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/1767846032 wait complete. 2023-07-13T22:20:35.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.215+0000 7fe8c57fa700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 0x7fe8b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.215+0000 7fe8c57fa700 1 -- 172.21.15.26:0/1195359808 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe8c00cc670 con 0x7fe8c8070ee0 2023-07-13T22:20:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.215+0000 7fe8e1ca0b80 1 -- start start 2023-07-13T22:20:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.216+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 0x7fe8b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.216+0000 7fe8e1ca0b80 1 Processor -- start 2023-07-13T22:20:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 0x7fe8b40a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b070 tx=0x7fe8b8006210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa4452a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa4452a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa44531770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44411ac0 con 0x55aa445204d0 2023-07-13T22:20:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa444117c0 con 0x55aa44521c70 2023-07-13T22:20:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa442f5920 con 0x55aa44520e20 2023-07-13T22:20:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.217+0000 7fe8e1ca0b80 10 client.0 _ll_register_callbacks cb 0x55aa4440f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.218+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa4452a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa44531770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa4452a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8c77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa4452a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52180/0 (socket says 172.21.15.26:52180) 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8c77fe700 1 -- 172.21.15.26:0/2864217174 learned_addr learned my addr 172.21.15.26:0/2864217174 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 -- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 msgr2=0x55aa4452a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa4452a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 -- 172.21.15.26:0/2864217174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 msgr2=0x55aa4452a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa4452a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.219+0000 7fe8cc99c700 1 -- 172.21.15.26:0/2864217174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b000b7a0 con 0x55aa44521c70 2023-07-13T22:20:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.220+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa44531770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007ab0 tx=0x7fe8b8009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3c36e5d981ebe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.220+0000 7fe8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:20:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.221+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800e040 con 0x55aa44521c70 2023-07-13T22:20:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.221+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe8b801b860 con 0x55aa44521c70 2023-07-13T22:20:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.221+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa443cb4f0 con 0x55aa44521c70 2023-07-13T22:20:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.222+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b801ba20 con 0x55aa44521c70 2023-07-13T22:20:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.222+0000 7fe8e1ca0b80 10 client.26486 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.222+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aa44288c60 con 0x55aa44521c70 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.222+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa443cb4f0 con 0x55aa44521c70 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe8b8005800 con 0x55aa44521c70 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 10 client.26486 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 10 client.26486 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 20 client.26486 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa44535fb0 0x55aa445383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 4 client.26486 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa4446aeb0 con 0x55aa44535fb0 2023-07-13T22:20:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.223+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa44535fb0 0x55aa445383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.224+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa44535fb0 0x55aa445383c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea4535e12297787a server_cookie=62a9a14652911dae in_seq=0 out_seq=0 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.224+0000 7fe8a77fe700 10 client.26486 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8a77fe700 1 -- 172.21.15.26:0/2864217174 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa4446aeb0 con 0x55aa44535fb0 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8a77fe700 10 client.26486 handle_command_reply: tid=0 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 client.26486 shutdown 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 msgr2=0x55aa44531770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa44531770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8007ab0 tx=0x7fe8b8009440 comp rx=0 tx=0).stop 2023-07-13T22:20:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 shutdown_connections 2023-07-13T22:20:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa44535fb0 0x55aa445383c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa445204d0 0x55aa4452a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa44531770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/2864217174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa44520e20 0x55aa4452a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 >> 172.21.15.26:0/2864217174 conn(0x55aa44515da0 msgr2=0x55aa4451a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.225+0000 7fe8a77fe700 0 client.26486 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.226+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 shutdown_connections 2023-07-13T22:20:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.226+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/2864217174 wait complete. 2023-07-13T22:20:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.226+0000 7fe8e1ca0b80 20 client.26486 trim_cache size 0 max 16384 2023-07-13T22:20:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.243+0000 7fe8e1ca0b80 1 Processor -- start 2023-07-13T22:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.243+0000 7fe8e1ca0b80 1 -- start start 2023-07-13T22:20:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44520e20 0x55aa44531730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa4453cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa445159e0 con 0x55aa44520e20 2023-07-13T22:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa4453d290 con 0x55aa44521c70 2023-07-13T22:20:35.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44535cb0 con 0x55aa445204d0 2023-07-13T22:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52190/0 (socket says 172.21.15.26:52190) 2023-07-13T22:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8c7fff700 1 -- 172.21.15.26:0/135690073 learned_addr learned my addr 172.21.15.26:0/135690073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.244+0000 7fe8c77fe700 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44520e20 0x55aa44531730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa4453cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 -- 172.21.15.26:0/135690073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 msgr2=0x55aa4453cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa4453cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 -- 172.21.15.26:0/135690073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44520e20 msgr2=0x55aa44531730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44520e20 0x55aa44531730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 -- 172.21.15.26:0/135690073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa442c6250 con 0x55aa445204d0 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00999c0 tx=0x7fe8c0010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64cac78466b7e7c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.245+0000 7fe8beffd700 1 -- 172.21.15.26:0/135690073 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c009da60 con 0x55aa445204d0 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8beffd700 1 -- 172.21.15.26:0/135690073 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c009ae90 con 0x55aa445204d0 2023-07-13T22:20:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8beffd700 1 -- 172.21.15.26:0/135690073 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0099ef0 con 0x55aa445204d0 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/135690073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 msgr2=0x55aa445208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00999c0 tx=0x7fe8c0010bd0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/135690073 shutdown_connections 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44521c70 0x55aa4453cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa445208a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/135690073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44520e20 0x55aa44531730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.246+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/135690073 >> 172.21.15.26:0/135690073 conn(0x55aa44515da0 msgr2=0x55aa4451b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.247+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/135690073 shutdown_connections 2023-07-13T22:20:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.247+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/135690073 wait complete. 2023-07-13T22:20:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.247+0000 7fe8e1ca0b80 1 -- start start 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 Processor -- start 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa4452d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44520e20 0x55aa4452de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 0x55aa44530c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa445311a0 con 0x55aa44521c70 2023-07-13T22:20:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa44531310 con 0x55aa44520e20 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa445314d0 con 0x55aa445204d0 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.248+0000 7fe8e1ca0b80 10 client.0 _ll_register_callbacks cb 0x7fe8b8007990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.249+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa4452d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.249+0000 7fe8c7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa4452d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52200/0 (socket says 172.21.15.26:52200) 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.249+0000 7fe8c7fff700 1 -- 172.21.15.26:0/3376332338 learned_addr learned my addr 172.21.15.26:0/3376332338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.249+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 0x55aa44530c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 -- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 msgr2=0x55aa4452d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa4452d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 -- 172.21.15.26:0/3376332338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44520e20 msgr2=0x55aa4452de50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44520e20 0x55aa4452de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 -- 172.21.15.26:0/3376332338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c81a4570 con 0x55aa44521c70 2023-07-13T22:20:35.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8cc99c700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 0x55aa44530c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005bd0 tx=0x7fe8b8027ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bf94be207eee39 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.250+0000 7fe8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:20:35.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800e070 con 0x55aa44521c70 2023-07-13T22:20:35.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe8b801bc30 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8b0010460 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8b801bdf0 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8e1ca0b80 10 client.16134 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.251+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55aa4429e2b0 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.252+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8b0010460 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe8b8039530 con 0x55aa44521c70 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 10 client.16134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:20:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 20 client.16134 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa4458ac80 0x55aa4458d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 4 client.16134 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:20:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa4442d050 con 0x55aa4458ac80 2023-07-13T22:20:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.253+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa4458ac80 0x55aa4458d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.254+0000 7fe8c7fff700 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa4458ac80 0x55aa4458d090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=258d2b809811fe76 server_cookie=31dfaa60ded7997f in_seq=0 out_seq=0 2023-07-13T22:20:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.254+0000 7fe8a77fe700 10 client.16134 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:20:35.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.254+0000 7fe8a77fe700 1 -- 172.21.15.26:0/3376332338 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa4442d050 con 0x55aa4458ac80 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.254+0000 7fe8a77fe700 10 client.16134 handle_command_reply: tid=0 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.254+0000 7fe8e1ca0b80 1 client.16134 shutdown 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 msgr2=0x55aa44530c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 0x55aa44530c60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005bd0 tx=0x7fe8b8027ae0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 shutdown_connections 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55aa4458ac80 0x55aa4458d090 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55aa44521c70 0x55aa44530c60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55aa445204d0 0x55aa4452d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 --2- 172.21.15.26:0/3376332338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aa44520e20 0x55aa4452de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 >> 172.21.15.26:0/3376332338 conn(0x55aa44515da0 msgr2=0x55aa44532060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8a77fe700 0 client.16134 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 shutdown_connections 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.255+0000 7fe8e1ca0b80 1 -- 172.21.15.26:0/3376332338 wait complete. 2023-07-13T22:20:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.256+0000 7fe8e1ca0b80 20 client.16134 trim_cache size 0 max 16384 2023-07-13T22:20:35.245 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:20:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 msgr2=0x7fe8b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 0x7fe8b40a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b070 tx=0x7fe8b8006210 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 msgr2=0x7fe8c8113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8c000d950 tx=0x7fe8c000dd10 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 shutdown_connections 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8b40a1130 0x7fe8b40a35e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8c810bd60 0x7fe8c81a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8c8070ee0 0x7fe8c8113b10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 --2- 172.21.15.26:0/1195359808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8c8071830 0x7fe8c8114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 >> 172.21.15.26:0/1195359808 conn(0x7fe8c806ca90 msgr2=0x7fe8c806dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.257+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 shutdown_connections 2023-07-13T22:20:35.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.258+0000 7fe8ce3ff700 1 -- 172.21.15.26:0/1195359808 wait complete. 2023-07-13T22:20:35.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:20:35.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.709+0000 7fb9b43ff700 1 -- 172.21.15.26:0/2180682312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 msgr2=0x7fb9ac107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.709+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/2180682312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac107e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009990 tx=0x7fb9a8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 -- 172.21.15.26:0/2180682312 shutdown_connections 2023-07-13T22:20:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/2180682312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac071620 0x7fb9ac110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/2180682312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac107e30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/2180682312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac070c70 0x7fb9ac0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 -- 172.21.15.26:0/2180682312 >> 172.21.15.26:0/2180682312 conn(0x7fb9ac06ca90 msgr2=0x7fb9ac06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 -- 172.21.15.26:0/2180682312 shutdown_connections 2023-07-13T22:20:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.710+0000 7fb9b43ff700 1 -- 172.21.15.26:0/2180682312 wait complete. 2023-07-13T22:20:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 Processor -- start 2023-07-13T22:20:35.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:35 smithi026 ceph-mon[95647]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 95 op/s 2023-07-13T22:20:35.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 -- start start 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 0x7fb9ac10e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac071620 0x7fb9ac10e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac10ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac10b4d0 con 0x7fb9ac071620 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac10b640 con 0x7fb9ac107a20 2023-07-13T22:20:35.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.712+0000 7fb9b43ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac10b800 con 0x7fb9ac070c70 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.713+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 0x7fb9ac10e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.713+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 0x7fb9ac10e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52226/0 (socket says 172.21.15.26:52226) 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.713+0000 7fb9b219b700 1 -- 172.21.15.26:0/1525034871 learned_addr learned my addr 172.21.15.26:0/1525034871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.713+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac10ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 -- 172.21.15.26:0/1525034871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 msgr2=0x7fb9ac10e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 0x7fb9ac10e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 -- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac071620 msgr2=0x7fb9ac10e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac071620 0x7fb9ac10e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 -- 172.21.15.26:0/1525034871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a80095f0 con 0x7fb9ac107a20 2023-07-13T22:20:35.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.714+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac10ad30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.717+0000 7fb9a37fe700 1 -- 172.21.15.26:0/1525034871 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4018070 con 0x7fb9ac107a20 2023-07-13T22:20:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.717+0000 7fb9a37fe700 1 -- 172.21.15.26:0/1525034871 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a400f040 con 0x7fb9ac107a20 2023-07-13T22:20:35.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.717+0000 7fb9a37fe700 1 -- 172.21.15.26:0/1525034871 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4007860 con 0x7fb9ac107a20 2023-07-13T22:20:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.718+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac19ff30 con 0x7fb9ac107a20 2023-07-13T22:20:35.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.718+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac1a03f0 con 0x7fb9ac107a20 2023-07-13T22:20:35.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.723+0000 7fb9a37fe700 1 -- 172.21.15.26:0/1525034871 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9a400c9f0 con 0x7fb9ac107a20 2023-07-13T22:20:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.725+0000 7fb9a37fe700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 0x7fb9980a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.726+0000 7fb9a37fe700 1 -- 172.21.15.26:0/1525034871 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9a40cbe50 con 0x7fb9ac107a20 2023-07-13T22:20:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.727+0000 7fb9c7db3b80 1 Processor -- start 2023-07-13T22:20:35.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.727+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 0x7fb9980a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.727+0000 7fb9c7db3b80 1 -- start start 2023-07-13T22:20:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.727+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599306114d0 0x5599306118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.727+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x5599306126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930516cb0 con 0x5599306114d0 2023-07-13T22:20:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930569a40 con 0x559930611e20 2023-07-13T22:20:35.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930562ed0 con 0x559930612c70 2023-07-13T22:20:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52238/0 (socket says 172.21.15.26:52238) 2023-07-13T22:20:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 -- 172.21.15.26:0/2348857175 learned_addr learned my addr 172.21.15.26:0/2348857175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b199a700 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x5599306126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 -- 172.21.15.26:0/2348857175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 msgr2=0x5599306126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x5599306126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 -- 172.21.15.26:0/2348857175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599306114d0 msgr2=0x5599306118e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599306114d0 0x5599306118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.728+0000 7fb9b299c700 1 -- 172.21.15.26:0/2348857175 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559930390de0 con 0x559930612c70 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.729+0000 7fb9b299c700 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40090a0 tx=0x7fb9a4098c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe81eb960d326af server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.729+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 0x7fb9980a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8006010 tx=0x7fb9a800b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.730+0000 7fb9a0bfe700 1 -- 172.21.15.26:0/2348857175 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a409d2a0 con 0x559930612c70 2023-07-13T22:20:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.730+0000 7fb9a0bfe700 1 -- 172.21.15.26:0/2348857175 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a40a16c0 con 0x559930612c70 2023-07-13T22:20:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.730+0000 7fb9a0bfe700 1 -- 172.21.15.26:0/2348857175 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4099730 con 0x559930612c70 2023-07-13T22:20:35.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.731+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2348857175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 msgr2=0x5599306130e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.731+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40090a0 tx=0x7fb9a4098c50 comp rx=0 tx=0).stop 2023-07-13T22:20:35.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2348857175 shutdown_connections 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599306114d0 0x5599306118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930612c70 0x5599306130e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2348857175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x5599306126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2348857175 >> 172.21.15.26:0/2348857175 conn(0x559930606da0 msgr2=0x55993060c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2348857175 shutdown_connections 2023-07-13T22:20:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.732+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2348857175 wait complete. 2023-07-13T22:20:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.733+0000 7fb9c7db3b80 1 -- start start 2023-07-13T22:20:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.734+0000 7fb9c7db3b80 1 Processor -- start 2023-07-13T22:20:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x55993061b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930622750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559930502ac0 con 0x559930612c70 2023-07-13T22:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599305027c0 con 0x559930611e20 2023-07-13T22:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599303e6920 con 0x5599306114d0 2023-07-13T22:20:35.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.735+0000 7fb9c7db3b80 10 client.0 _ll_register_callbacks cb 0x559930500760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.736+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.736+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52244/0 (socket says 172.21.15.26:52244) 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.736+0000 7fb9b219b700 1 -- 172.21.15.26:0/2437324443 learned_addr learned my addr 172.21.15.26:0/2437324443 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.736+0000 7fb9b199a700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x55993061b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.736+0000 7fb9b299c700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930622750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.737+0000 7fb9b219b700 1 -- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 msgr2=0x55993061b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.737+0000 7fb9b219b700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x55993061b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.737+0000 7fb9b219b700 1 -- 172.21.15.26:0/2437324443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 msgr2=0x559930622750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.737+0000 7fb9b219b700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930622750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.737+0000 7fb9b219b700 1 -- 172.21.15.26:0/2437324443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac1a03f0 con 0x5599306114d0 2023-07-13T22:20:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb9b219b700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005770 tx=0x7fb9a8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d4511022db4399 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb995ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a802b070 con 0x5599306114d0 2023-07-13T22:20:35.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9a8004ca0 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599304bc4f0 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a8004e60 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb9c7db3b80 10 client.26485 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.738+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x559930379c60 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599304bc4f0 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9a802f710 con 0x5599306114d0 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 10 client.26485 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 10 client.26485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:20:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 20 client.26485 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x559930626f60 0x559930629370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 4 client.26485 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55993055beb0 con 0x559930626f60 2023-07-13T22:20:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.739+0000 7fb9b199a700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x559930626f60 0x559930629370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.740+0000 7fb9b199a700 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x559930626f60 0x559930629370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=814691b0faf60050 server_cookie=845fb5e43dc0933b in_seq=0 out_seq=0 2023-07-13T22:20:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.740+0000 7fb995ffb700 10 client.26485 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.741+0000 7fb995ffb700 1 -- 172.21.15.26:0/2437324443 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55993055beb0 con 0x559930626f60 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.741+0000 7fb995ffb700 10 client.26485 handle_command_reply: tid=0 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.741+0000 7fb9c7db3b80 1 client.26485 shutdown 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.741+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 msgr2=0x55993061b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.741+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8005770 tx=0x7fb9a8009310 comp rx=0 tx=0).stop 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 shutdown_connections 2023-07-13T22:20:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930622750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x559930626f60 0x559930629370 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599306114d0 0x55993061b1d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/2437324443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559930611e20 0x55993061b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 >> 172.21.15.26:0/2437324443 conn(0x559930606da0 msgr2=0x55993060b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb995ffb700 0 client.26485 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 shutdown_connections 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/2437324443 wait complete. 2023-07-13T22:20:35.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.742+0000 7fb9c7db3b80 20 client.26485 trim_cache size 0 max 16384 2023-07-13T22:20:35.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.770+0000 7fb9c7db3b80 1 Processor -- start 2023-07-13T22:20:35.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.770+0000 7fb9c7db3b80 1 -- start start 2023-07-13T22:20:35.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x5599306226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930616140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993061a680 con 0x559930612c70 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993061ad40 con 0x5599306114d0 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599306121f0 con 0x559930611e20 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55306/0 (socket says 172.21.15.26:55306) 2023-07-13T22:20:35.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9b219b700 1 -- 172.21.15.26:0/1530651336 learned_addr learned my addr 172.21.15.26:0/1530651336 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.771+0000 7fb9b199a700 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x5599306226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930616140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 -- 172.21.15.26:0/1530651336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 msgr2=0x5599306226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x5599306226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 -- 172.21.15.26:0/1530651336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 msgr2=0x559930616140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930616140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 -- 172.21.15.26:0/1530651336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599303b7250 con 0x5599306114d0 2023-07-13T22:20:35.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802fa70 tx=0x7fb9a802fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7e0312f8005dd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9a13ff700 1 -- 172.21.15.26:0/1530651336 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a802b070 con 0x5599306114d0 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9a13ff700 1 -- 172.21.15.26:0/1530651336 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a801e040 con 0x5599306114d0 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.772+0000 7fb9a13ff700 1 -- 172.21.15.26:0/1530651336 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800f8f0 con 0x5599306114d0 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1530651336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 msgr2=0x559930615c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802fa70 tx=0x7fb9a802fc90 comp rx=0 tx=0).stop 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1530651336 shutdown_connections 2023-07-13T22:20:35.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x559930616140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930615c00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1530651336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x5599306226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1530651336 >> 172.21.15.26:0/1530651336 conn(0x559930606da0 msgr2=0x5599306190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.773+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1530651336 shutdown_connections 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.774+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1530651336 wait complete. 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.774+0000 7fb9c7db3b80 1 -- start start 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.774+0000 7fb9c7db3b80 1 Processor -- start 2023-07-13T22:20:35.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x559930622510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x55993061ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993061efb0 con 0x559930612c70 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993061f0f0 con 0x5599306114d0 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993061f2b0 con 0x559930611e20 2023-07-13T22:20:35.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9c7db3b80 10 client.0 _ll_register_callbacks cb 0x559930500760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.775+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55312/0 (socket says 172.21.15.26:55312) 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 -- 172.21.15.26:0/1229344687 learned_addr learned my addr 172.21.15.26:0/1229344687 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b299c700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x55993061ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 -- 172.21.15.26:0/1229344687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 msgr2=0x559930622510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x559930622510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 -- 172.21.15.26:0/1229344687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 msgr2=0x55993061ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x55993061ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 -- 172.21.15.26:0/1229344687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559930379c60 con 0x5599306114d0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb9b219b700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802faf0 tx=0x7fb9a8004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c83d96f1677c85 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.776+0000 7fb995ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a802b040 con 0x5599306114d0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9a802f820 con 0x5599306114d0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a40a16c0 con 0x5599306114d0 2023-07-13T22:20:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a800f650 con 0x5599306114d0 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb9c7db3b80 10 client.26501 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55993038f2b0 con 0x5599306114d0 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.777+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9a40a16c0 con 0x5599306114d0 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb9a800f7b0 con 0x5599306114d0 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 10 client.26501 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 10 client.26501 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 20 client.26501 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:35.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55993067bde0 0x55993067e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:35.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 4 client.26501 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:20:35.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55993051e050 con 0x55993067bde0 2023-07-13T22:20:35.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.778+0000 7fb9b199a700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55993067bde0 0x55993067e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9b199a700 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55993067bde0 0x55993067e1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbbee621b4f55db9 server_cookie=bfdaf73b01d1bbb0 in_seq=0 out_seq=0 2023-07-13T22:20:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb995ffb700 10 client.26501 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:20:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb995ffb700 1 -- 172.21.15.26:0/1229344687 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55993051e050 con 0x55993067bde0 2023-07-13T22:20:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb995ffb700 10 client.26501 handle_command_reply: tid=0 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 client.26501 shutdown 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 msgr2=0x559930621fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802faf0 tx=0x7fb9a8004e00 comp rx=0 tx=0).stop 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 shutdown_connections 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x559930612c70 0x55993061ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55993067bde0 0x55993067e1f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599306114d0 0x559930621fd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.779+0000 7fb9c7db3b80 1 --2- 172.21.15.26:0/1229344687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559930611e20 0x559930622510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.780+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 >> 172.21.15.26:0/1229344687 conn(0x559930606da0 msgr2=0x559930615c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.780+0000 7fb995ffb700 0 client.26501 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.780+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 shutdown_connections 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.780+0000 7fb9c7db3b80 1 -- 172.21.15.26:0/1229344687 wait complete. 2023-07-13T22:20:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.780+0000 7fb9c7db3b80 20 client.26501 trim_cache size 0 max 16384 2023-07-13T22:20:35.772 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:20:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 msgr2=0x7fb9980a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 0x7fb9980a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8006010 tx=0x7fb9a800b260 comp rx=0 tx=0).stop 2023-07-13T22:20:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 msgr2=0x7fb9ac10ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac10ad30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400efc0 tx=0x7fb9a400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 shutdown_connections 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9ac107a20 0x7fb9ac10ad30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9ac070c70 0x7fb9ac10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb9980a12a0 0x7fb9980a3750 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 --2- 172.21.15.26:0/1525034871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9ac071620 0x7fb9ac10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:35.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 >> 172.21.15.26:0/1525034871 conn(0x7fb9ac06ca90 msgr2=0x7fb9ac10ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:35.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 shutdown_connections 2023-07-13T22:20:35.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:35.781+0000 7fb9b43ff700 1 -- 172.21.15.26:0/1525034871 wait complete. 2023-07-13T22:20:35.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:20:35.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:35 smithi121 ceph-mon[129187]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 95 op/s 2023-07-13T22:20:36.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:35 smithi117 ceph-mon[129074]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 95 op/s 2023-07-13T22:20:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 -- 172.21.15.26:0/3265762964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 msgr2=0x7fdc24102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/3265762964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24102120 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009990 tx=0x7fdc20009ca0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 -- 172.21.15.26:0/3265762964 shutdown_connections 2023-07-13T22:20:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/3265762964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24103890 0x7fdc2410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/3265762964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24102120 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/3265762964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24102f00 0x7fdc24103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.309+0000 7fdc2bdff700 1 -- 172.21.15.26:0/3265762964 >> 172.21.15.26:0/3265762964 conn(0x7fdc240fd350 msgr2=0x7fdc240ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.310+0000 7fdc2bdff700 1 -- 172.21.15.26:0/3265762964 shutdown_connections 2023-07-13T22:20:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.310+0000 7fdc2bdff700 1 -- 172.21.15.26:0/3265762964 wait complete. 2023-07-13T22:20:36.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.311+0000 7fdc2bdff700 1 Processor -- start 2023-07-13T22:20:36.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 -- start start 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 0x7fdc24197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24103890 0x7fdc2419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419f580 con 0x7fdc24102f00 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419f6c0 con 0x7fdc24101d10 2023-07-13T22:20:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.312+0000 7fdc2bdff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2419f850 con 0x7fdc24103890 2023-07-13T22:20:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55332/0 (socket says 172.21.15.26:55332) 2023-07-13T22:20:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 -- 172.21.15.26:0/2870303978 learned_addr learned my addr 172.21.15.26:0/2870303978 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24103890 0x7fdc2419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc2939a700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 0x7fdc24197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 -- 172.21.15.26:0/2870303978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24103890 msgr2=0x7fdc2419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24103890 0x7fdc2419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 -- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 msgr2=0x7fdc24197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 0x7fdc24197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 -- 172.21.15.26:0/2870303978 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200095f0 con 0x7fdc24101d10 2023-07-13T22:20:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc2939a700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 0x7fdc24197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:20:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.313+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc200057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.314+0000 7fdc1affd700 1 -- 172.21.15.26:0/2870303978 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2001d070 con 0x7fdc24101d10 2023-07-13T22:20:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.314+0000 7fdc1affd700 1 -- 172.21.15.26:0/2870303978 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc20003fe0 con 0x7fdc24101d10 2023-07-13T22:20:36.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.314+0000 7fdc1affd700 1 -- 172.21.15.26:0/2870303978 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20022df0 con 0x7fdc24101d10 2023-07-13T22:20:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.314+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2419fad0 con 0x7fdc24101d10 2023-07-13T22:20:36.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.314+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2419fea0 con 0x7fdc24101d10 2023-07-13T22:20:36.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.319+0000 7fdc1affd700 1 -- 172.21.15.26:0/2870303978 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdc20022470 con 0x7fdc24101d10 2023-07-13T22:20:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.320+0000 7fdc3f74db80 1 Processor -- start 2023-07-13T22:20:36.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.321+0000 7fdc3f74db80 1 -- start start 2023-07-13T22:20:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.321+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 0x5572f8b090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.321+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b09ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 0x5572f8b0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8a05cb0 con 0x5572f8b0a480 2023-07-13T22:20:36.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8a58a40 con 0x5572f8b08ce0 2023-07-13T22:20:36.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8a51ed0 con 0x5572f8b09630 2023-07-13T22:20:36.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc2a39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 0x5572f8b090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc2a39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 0x5572f8b090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55348/0 (socket says 172.21.15.26:55348) 2023-07-13T22:20:36.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc2a39c700 1 -- 172.21.15.26:0/910989670 learned_addr learned my addr 172.21.15.26:0/910989670 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.322+0000 7fdc2939a700 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 0x5572f8b0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b09ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 -- 172.21.15.26:0/910989670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 msgr2=0x5572f8b09ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b09ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 -- 172.21.15.26:0/910989670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 msgr2=0x5572f8b090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 0x5572f8b090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 -- 172.21.15.26:0/910989670 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572f887fde0 con 0x5572f8b0a480 2023-07-13T22:20:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc1affd700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 0x7fdc100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.323+0000 7fdc2939a700 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 0x5572f8b0a8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009c10 tx=0x7fdc14009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77543bd918fb1a5f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.324+0000 7fdc1affd700 1 -- 172.21.15.26:0/2870303978 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc200cdd40 con 0x7fdc24101d10 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.324+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 0x7fdc100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.324+0000 7fdc0ffff700 1 -- 172.21.15.26:0/910989670 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400eaa0 con 0x5572f8b0a480 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.324+0000 7fdc0ffff700 1 -- 172.21.15.26:0/910989670 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc14004cf0 con 0x5572f8b0a480 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.324+0000 7fdc0ffff700 1 -- 172.21.15.26:0/910989670 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140056c0 con 0x5572f8b0a480 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.325+0000 7fdc3f74db80 1 -- 172.21.15.26:0/910989670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 msgr2=0x5572f8b0a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.325+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 0x5572f8b0a8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009c10 tx=0x7fdc14009f20 comp rx=0 tx=0).stop 2023-07-13T22:20:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.325+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 0x7fdc100a3680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c011500 tx=0x7fdc1c009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 -- 172.21.15.26:0/910989670 shutdown_connections 2023-07-13T22:20:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b0a480 0x5572f8b0a8f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b08ce0 0x5572f8b090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/910989670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b09ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 -- 172.21.15.26:0/910989670 >> 172.21.15.26:0/910989670 conn(0x5572f8af5da0 msgr2=0x5572f8afb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 -- 172.21.15.26:0/910989670 shutdown_connections 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.326+0000 7fdc3f74db80 1 -- 172.21.15.26:0/910989670 wait complete. 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.327+0000 7fdc3f74db80 1 -- start start 2023-07-13T22:20:36.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.327+0000 7fdc3f74db80 1 Processor -- start 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b0a480 0x5572f8b19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f89f1ac0 con 0x5572f8b08ce0 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f89f17c0 con 0x5572f8b09630 2023-07-13T22:20:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f88d5920 con 0x5572f8b0a480 2023-07-13T22:20:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.328+0000 7fdc3f74db80 10 client.0 _ll_register_callbacks cb 0x5572f89ef760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55352/0 (socket says 172.21.15.26:55352) 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 -- 172.21.15.26:0/2826623410 learned_addr learned my addr 172.21.15.26:0/2826623410 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b0a480 0x5572f8b19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 -- 172.21.15.26:0/2826623410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b0a480 msgr2=0x5572f8b19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b0a480 0x5572f8b19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 -- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 msgr2=0x5572f8b129d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b129d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.329+0000 7fdc2939a700 1 -- 172.21.15.26:0/2826623410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc14009840 con 0x5572f8b09630 2023-07-13T22:20:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.330+0000 7fdc2939a700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000c00 tx=0x7fdc140056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45eca6994fa9c18c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.330+0000 7fdc0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.330+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14020070 con 0x5572f8b09630 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.330+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc140049e0 con 0x5572f8b09630 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.330+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572f89ab4f0 con 0x5572f8b09630 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1400e8a0 con 0x5572f8b09630 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 10 client.26513 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5572f8868c60 con 0x5572f8b09630 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572f89ab4f0 con 0x5572f8b09630 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc1400c070 con 0x5572f8b09630 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 10 client.26513 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 10 client.26513 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 20 client.26513 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b01480 0x5572f8b03890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 4 client.26513 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.331+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572f8a4aeb0 con 0x5572f8b01480 2023-07-13T22:20:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.332+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b01480 0x5572f8b03890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.334+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b01480 0x5572f8b03890 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a0fe057286502c6 server_cookie=d454935027d7ef69 in_seq=0 out_seq=0 2023-07-13T22:20:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.334+0000 7fdc0d7fa700 10 client.26513 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:20:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.335+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/2826623410 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5572f8a4aeb0 con 0x5572f8b01480 2023-07-13T22:20:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.335+0000 7fdc0d7fa700 10 client.26513 handle_command_reply: tid=0 2023-07-13T22:20:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.335+0000 7fdc3f74db80 1 client.26513 shutdown 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 msgr2=0x5572f8b12f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000c00 tx=0x7fdc140056c0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 shutdown_connections 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b01480 0x5572f8b03890 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b129d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b0a480 0x5572f8b19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/2826623410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b09630 0x5572f8b12f10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 >> 172.21.15.26:0/2826623410 conn(0x5572f8af5da0 msgr2=0x5572f8afa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc0d7fa700 0 client.26513 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:20:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 shutdown_connections 2023-07-13T22:20:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.336+0000 7fdc3f74db80 1 -- 172.21.15.26:0/2826623410 wait complete. 2023-07-13T22:20:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.337+0000 7fdc3f74db80 20 client.26513 trim_cache size 0 max 16384 2023-07-13T22:20:36.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 Processor -- start 2023-07-13T22:20:36.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 -- start start 2023-07-13T22:20:36.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b0a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8b12540 con 0x5572f8b08ce0 2023-07-13T22:20:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8afcd00 con 0x5572f8b0a480 2023-07-13T22:20:36.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.365+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8afc4c0 con 0x5572f8b09630 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.366+0000 7fdc2a39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.366+0000 7fdc2a39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55366/0 (socket says 172.21.15.26:55366) 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.366+0000 7fdc2a39c700 1 -- 172.21.15.26:0/4135759004 learned_addr learned my addr 172.21.15.26:0/4135759004 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 -- 172.21.15.26:0/4135759004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 msgr2=0x5572f8b081f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b081f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 -- 172.21.15.26:0/4135759004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 msgr2=0x5572f8b0a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b0a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 -- 172.21.15.26:0/4135759004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572f88a6250 con 0x5572f8b0a480 2023-07-13T22:20:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.367+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c004a60 tx=0x7fdc1c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d4f5211d0081b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.378+0000 7fdc18bff700 1 -- 172.21.15.26:0/4135759004 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c017e30 con 0x5572f8b0a480 2023-07-13T22:20:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.378+0000 7fdc18bff700 1 -- 172.21.15.26:0/4135759004 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1c01b070 con 0x5572f8b0a480 2023-07-13T22:20:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.378+0000 7fdc18bff700 1 -- 172.21.15.26:0/4135759004 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0238f0 con 0x5572f8b0a480 2023-07-13T22:20:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 -- 172.21.15.26:0/4135759004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 msgr2=0x5572f8b03490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c004a60 tx=0x7fdc1c014000 comp rx=0 tx=0).stop 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 -- 172.21.15.26:0/4135759004 shutdown_connections 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b0a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b03490 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/4135759004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 -- 172.21.15.26:0/4135759004 >> 172.21.15.26:0/4135759004 conn(0x5572f8af5da0 msgr2=0x5572f8b108e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.380+0000 7fdc3f74db80 1 -- 172.21.15.26:0/4135759004 shutdown_connections 2023-07-13T22:20:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.381+0000 7fdc3f74db80 1 -- 172.21.15.26:0/4135759004 wait complete. 2023-07-13T22:20:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.382+0000 7fdc3f74db80 1 -- start start 2023-07-13T22:20:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.382+0000 7fdc3f74db80 1 Processor -- start 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b08480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b06fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8b076e0 con 0x5572f8b08ce0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8b07850 con 0x5572f8b0a480 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572f8b7a5f0 con 0x5572f8b09630 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.383+0000 7fdc3f74db80 10 client.0 _ll_register_callbacks cb 0x7fdc1c007a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.384+0000 7fdc2939a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.384+0000 7fdc2939a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52314/0 (socket says 172.21.15.26:52314) 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.384+0000 7fdc2939a700 1 -- 172.21.15.26:0/1012782811 learned_addr learned my addr 172.21.15.26:0/1012782811 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.384+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b08480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2a39c700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b06fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2939a700 1 -- 172.21.15.26:0/1012782811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 msgr2=0x5572f8b06fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2939a700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b06fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2939a700 1 -- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 msgr2=0x5572f8b08480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2939a700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b08480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.385+0000 7fdc2939a700 1 -- 172.21.15.26:0/1012782811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c00f040 con 0x5572f8b09630 2023-07-13T22:20:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.386+0000 7fdc2939a700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc14011c50 tx=0x7fdc1401a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b3aa218e00b32e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.386+0000 7fdc0d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.387+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14020040 con 0x5572f8b09630 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.387+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdc14000e40 con 0x5572f8b09630 2023-07-13T22:20:36.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.387+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc14004cf0 con 0x5572f8b09630 2023-07-13T22:20:36.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.389+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1400eab0 con 0x5572f8b09630 2023-07-13T22:20:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.389+0000 7fdc3f74db80 10 client.26500 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.390+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5572f887e2b0 con 0x5572f8b09630 2023-07-13T22:20:36.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.393+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc14004cf0 con 0x5572f8b09630 2023-07-13T22:20:36.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.393+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdc1400c070 con 0x5572f8b09630 2023-07-13T22:20:36.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.414+0000 7fdc3f74db80 10 client.26500 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:36.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.414+0000 7fdc3f74db80 10 client.26500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:20:36.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.414+0000 7fdc3f74db80 20 client.26500 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:36.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.415+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b7bb30 0x5572f8b7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.415+0000 7fdc3f74db80 4 client.26500 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:20:36.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.415+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5572f8a0d050 con 0x5572f8b7bb30 2023-07-13T22:20:36.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.417+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b7bb30 0x5572f8b7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.418+0000 7fdc29b9b700 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b7bb30 0x5572f8b7df40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28d71077db98963b server_cookie=a0e11d2d33032838 in_seq=0 out_seq=0 2023-07-13T22:20:36.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.419+0000 7fdc0d7fa700 10 client.26500 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:20:36.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.420+0000 7fdc0d7fa700 1 -- 172.21.15.26:0/1012782811 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5572f8a0d050 con 0x5572f8b7bb30 2023-07-13T22:20:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.420+0000 7fdc0d7fa700 10 client.26500 handle_command_reply: tid=0 2023-07-13T22:20:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.420+0000 7fdc3f74db80 1 client.26500 shutdown 2023-07-13T22:20:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 msgr2=0x5572f8b06a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc14011c50 tx=0x7fdc1401a400 comp rx=0 tx=0).stop 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 shutdown_connections 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5572f8b7bb30 0x5572f8b7df40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5572f8b08ce0 0x5572f8b08480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5572f8b0a480 0x5572f8b06fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 --2- 172.21.15.26:0/1012782811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5572f8b09630 0x5572f8b06a90 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc14011c50 tx=0x7fdc1401a400 comp rx=0 tx=0).stop 2023-07-13T22:20:36.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 >> 172.21.15.26:0/1012782811 conn(0x5572f8af5da0 msgr2=0x5572f8b03490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.421+0000 7fdc0d7fa700 0 client.26500 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:20:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.425+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 shutdown_connections 2023-07-13T22:20:36.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.425+0000 7fdc3f74db80 1 -- 172.21.15.26:0/1012782811 wait complete. 2023-07-13T22:20:36.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.425+0000 7fdc3f74db80 20 client.26500 trim_cache size 0 max 16384 2023-07-13T22:20:36.411 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:20:36.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 msgr2=0x7fdc100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 0x7fdc100a3680 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c011500 tx=0x7fdc1c009040 comp rx=0 tx=0).stop 2023-07-13T22:20:36.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 msgr2=0x7fdc24197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc200057f0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 shutdown_connections 2023-07-13T22:20:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc24103890 0x7fdc2419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc24101d10 0x7fdc24197a00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fdc100a11d0 0x7fdc100a3680 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 --2- 172.21.15.26:0/2870303978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc24102f00 0x7fdc24197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.427+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 >> 172.21.15.26:0/2870303978 conn(0x7fdc240fd350 msgr2=0x7fdc240ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.428+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 shutdown_connections 2023-07-13T22:20:36.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.428+0000 7fdc2bdff700 1 -- 172.21.15.26:0/2870303978 wait complete. 2023-07-13T22:20:36.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:20:36.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 -- 172.21.15.26:0/3571846806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0107d20 msgr2=0x7f45c010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 --2- 172.21.15.26:0/3571846806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0107d20 0x7f45c010e5e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45b00098f0 tx=0x7f45b0009c00 comp rx=0 tx=0).stop 2023-07-13T22:20:36.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 -- 172.21.15.26:0/3571846806 shutdown_connections 2023-07-13T22:20:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 --2- 172.21.15.26:0/3571846806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0107d20 0x7f45c010e5e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 --2- 172.21.15.26:0/3571846806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c01061a0 0x7f45c01065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 --2- 172.21.15.26:0/3571846806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 0x7f45c01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.952+0000 7f45c57ff700 1 -- 172.21.15.26:0/3571846806 >> 172.21.15.26:0/3571846806 conn(0x7f45c00751c0 msgr2=0x7f45c0077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.953+0000 7f45c57ff700 1 -- 172.21.15.26:0/3571846806 shutdown_connections 2023-07-13T22:20:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.953+0000 7f45c57ff700 1 -- 172.21.15.26:0/3571846806 wait complete. 2023-07-13T22:20:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.955+0000 7f45c57ff700 1 Processor -- start 2023-07-13T22:20:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.955+0000 7f45c57ff700 1 -- start start 2023-07-13T22:20:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.955+0000 7f45c57ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.956+0000 7f45c57ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 0x7f45c007e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.956+0000 7f45c57ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c0107d20 0x7f45c01abfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.956+0000 7f45c57ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0081190 con 0x7f45c0107390 2023-07-13T22:20:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.956+0000 7f45c57ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00812d0 con 0x7f45c0107d20 2023-07-13T22:20:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.956+0000 7f45c57ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c0069f30 con 0x7f45c01061a0 2023-07-13T22:20:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.957+0000 7f45beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.957+0000 7f45beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52344/0 (socket says 172.21.15.26:52344) 2023-07-13T22:20:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.957+0000 7f45beffd700 1 -- 172.21.15.26:0/2459509605 learned_addr learned my addr 172.21.15.26:0/2459509605 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.957+0000 7f45be7fc700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 0x7f45c007e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.957+0000 7f45bf7fe700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c0107d20 0x7f45c01abfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 -- 172.21.15.26:0/2459509605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c0107d20 msgr2=0x7f45c01abfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c0107d20 0x7f45c01abfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 -- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 msgr2=0x7f45c007e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 0x7f45c007e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 -- 172.21.15.26:0/2459509605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0009520 con 0x7f45c01061a0 2023-07-13T22:20:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.958+0000 7f45beffd700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f45b800efb0 tx=0x7f45b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.959+0000 7f45a7fff700 1 -- 172.21.15.26:0/2459509605 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45c01061a0 2023-07-13T22:20:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.959+0000 7f45a7fff700 1 -- 172.21.15.26:0/2459509605 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b800f040 con 0x7f45c01061a0 2023-07-13T22:20:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.959+0000 7f45a7fff700 1 -- 172.21.15.26:0/2459509605 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8004310 con 0x7f45c01061a0 2023-07-13T22:20:36.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.959+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c006a1b0 con 0x7f45c01061a0 2023-07-13T22:20:36.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.959+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c006a670 con 0x7f45c01061a0 2023-07-13T22:20:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.966+0000 7f45a7fff700 1 -- 172.21.15.26:0/2459509605 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f45b80092e0 con 0x7f45c01061a0 2023-07-13T22:20:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.969+0000 7f45d90aeb80 1 Processor -- start 2023-07-13T22:20:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.969+0000 7f45d90aeb80 1 -- start start 2023-07-13T22:20:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43d4d0 0x55e4af43d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43de20 0x55e4af43e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af342cb0 con 0x55e4af43de20 2023-07-13T22:20:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af395a40 con 0x55e4af43ec70 2023-07-13T22:20:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.970+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af38eed0 con 0x55e4af43d4d0 2023-07-13T22:20:36.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55414/0 (socket says 172.21.15.26:55414) 2023-07-13T22:20:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45bf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43d4d0 0x55e4af43d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 -- 172.21.15.26:0/2103648052 learned_addr learned my addr 172.21.15.26:0/2103648052 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 -- 172.21.15.26:0/2103648052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43d4d0 msgr2=0x55e4af43d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43d4d0 0x55e4af43d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 -- 172.21.15.26:0/2103648052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43de20 msgr2=0x55e4af43e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43de20 0x55e4af43e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.971+0000 7f45be7fc700 1 -- 172.21.15.26:0/2103648052 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4af1bcde0 con 0x55e4af43ec70 2023-07-13T22:20:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.972+0000 7f45be7fc700 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f45b400bd70 tx=0x7f45b40106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a7912983bb8aa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.975+0000 7f45a7fff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 0x7f45a80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.976+0000 7f45a7fff700 1 -- 172.21.15.26:0/2459509605 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45b80cc190 con 0x7f45c01061a0 2023-07-13T22:20:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.978+0000 7f45a57fa700 1 -- 172.21.15.26:0/2103648052 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4010e10 con 0x55e4af43ec70 2023-07-13T22:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.978+0000 7f45a57fa700 1 -- 172.21.15.26:0/2103648052 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b4015070 con 0x55e4af43ec70 2023-07-13T22:20:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.978+0000 7f45a57fa700 1 -- 172.21.15.26:0/2103648052 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4011680 con 0x55e4af43ec70 2023-07-13T22:20:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.986+0000 7f45d90aeb80 1 -- 172.21.15.26:0/2103648052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 msgr2=0x55e4af43f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:36.986+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f45b400bd70 tx=0x7f45b40106b0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.032+0000 7f45bf7fe700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 0x7f45a80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 -- 172.21.15.26:0/2103648052 shutdown_connections 2023-07-13T22:20:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af43f0e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43d4d0 0x55e4af43d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/2103648052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43de20 0x55e4af43e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 -- 172.21.15.26:0/2103648052 >> 172.21.15.26:0/2103648052 conn(0x55e4af432da0 msgr2=0x55e4af4381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 -- 172.21.15.26:0/2103648052 shutdown_connections 2023-07-13T22:20:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.033+0000 7f45d90aeb80 1 -- 172.21.15.26:0/2103648052 wait complete. 2023-07-13T22:20:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.034+0000 7f45d90aeb80 1 -- start start 2023-07-13T22:20:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.035+0000 7f45d90aeb80 1 Processor -- start 2023-07-13T22:20:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af4471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af32eac0 con 0x55e4af43d4d0 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af32e7c0 con 0x55e4af43ec70 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af212920 con 0x55e4af43de20 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45d90aeb80 10 client.0 _ll_register_callbacks cb 0x55e4af32c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.036+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52358/0 (socket says 172.21.15.26:52358) 2023-07-13T22:20:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.037+0000 7f45be7fc700 1 -- 172.21.15.26:0/1299664349 learned_addr learned my addr 172.21.15.26:0/1299664349 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.037+0000 7f45bf7fe700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.038+0000 7f45be7fc700 1 -- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 msgr2=0x55e4af44e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.038+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.038+0000 7f45be7fc700 1 -- 172.21.15.26:0/1299664349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 msgr2=0x55e4af4471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.038+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af4471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.038+0000 7f45be7fc700 1 -- 172.21.15.26:0/1299664349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b400b9a0 con 0x55e4af43de20 2023-07-13T22:20:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.040+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45b400b7e0 tx=0x7f45b4010e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec41199f9af1cb14 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.042+0000 7f45a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.042+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4020070 con 0x55e4af43de20 2023-07-13T22:20:37.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.042+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45b4017070 con 0x55e4af43de20 2023-07-13T22:20:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.045+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4af2e84f0 con 0x55e4af43de20 2023-07-13T22:20:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.045+0000 7f45bf7fe700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 0x7f45a80a3800 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009810 tx=0x7f45b000f720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.046+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45b40052d0 con 0x55e4af43de20 2023-07-13T22:20:37.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.046+0000 7f45d90aeb80 10 client.26512 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.046+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e4af1a5c60 con 0x55e4af43de20 2023-07-13T22:20:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.046+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4af2e84f0 con 0x55e4af43de20 2023-07-13T22:20:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.047+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45b4004ac0 con 0x55e4af43de20 2023-07-13T22:20:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.047+0000 7f45d90aeb80 10 client.26512 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.048+0000 7f45d90aeb80 10 client.26512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.048+0000 7f45d90aeb80 20 client.26512 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.048+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af452de0 0x55e4af4551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.048+0000 7f45d90aeb80 4 client.26512 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.048+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4af387eb0 con 0x55e4af452de0 2023-07-13T22:20:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.049+0000 7f45beffd700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af452de0 0x55e4af4551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.049+0000 7f45beffd700 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af452de0 0x55e4af4551f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762039f722fbe7cf server_cookie=e9d608c59e5f5b95 in_seq=0 out_seq=0 2023-07-13T22:20:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.049+0000 7f45a2ff5700 10 client.26512 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:37.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.050+0000 7f45a2ff5700 1 -- 172.21.15.26:0/1299664349 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e4af387eb0 con 0x55e4af452de0 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.050+0000 7f45a2ff5700 10 client.26512 handle_command_reply: tid=0 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.050+0000 7f45d90aeb80 1 client.26512 shutdown 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 msgr2=0x55e4af447730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f45b400b7e0 tx=0x7f45b4010e40 comp rx=0 tx=0).stop 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 shutdown_connections 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af4471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af452de0 0x55e4af4551f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1299664349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af447730 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 >> 172.21.15.26:0/1299664349 conn(0x55e4af432da0 msgr2=0x55e4af4378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45a2ff5700 0 client.26512 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 shutdown_connections 2023-07-13T22:20:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.051+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1299664349 wait complete. 2023-07-13T22:20:37.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.052+0000 7f45d90aeb80 20 client.26512 trim_cache size 0 max 16384 2023-07-13T22:20:37.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.080+0000 7f45d90aeb80 1 Processor -- start 2023-07-13T22:20:37.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 -- start start 2023-07-13T22:20:37.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 0x55e4af43f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43ec70 0x55e4af459b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af439070 con 0x55e4af43ec70 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af452bc0 con 0x55e4af43d4d0 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.081+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af444ef0 con 0x55e4af43de20 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.082+0000 7f45beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 0x55e4af43f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.082+0000 7f45beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 0x55e4af43f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55442/0 (socket says 172.21.15.26:55442) 2023-07-13T22:20:37.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.082+0000 7f45beffd700 1 -- 172.21.15.26:0/1833610171 learned_addr learned my addr 172.21.15.26:0/1833610171 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:37.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.082+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 -- 172.21.15.26:0/1833610171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 msgr2=0x55e4af43f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 0x55e4af43f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 -- 172.21.15.26:0/1833610171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43ec70 msgr2=0x55e4af459b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43ec70 0x55e4af459b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 -- 172.21.15.26:0/1833610171 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4af1e3250 con 0x55e4af43de20 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.083+0000 7f45be7fc700 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44e730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f45b40106d0 tx=0x7f45b4005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b179bad617ce4718 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.084+0000 7f45a5ffb700 1 -- 172.21.15.26:0/1833610171 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4020070 con 0x55e4af43de20 2023-07-13T22:20:37.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.084+0000 7f45a5ffb700 1 -- 172.21.15.26:0/1833610171 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b402cc70 con 0x55e4af43de20 2023-07-13T22:20:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.084+0000 7f45a5ffb700 1 -- 172.21.15.26:0/1833610171 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4004260 con 0x55e4af43de20 2023-07-13T22:20:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.085+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1833610171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 msgr2=0x55e4af44e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.085+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44e730 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f45b40106d0 tx=0x7f45b4005b00 comp rx=0 tx=0).stop 2023-07-13T22:20:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.086+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1833610171 shutdown_connections 2023-07-13T22:20:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.086+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43ec70 0x55e4af459b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.086+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43d4d0 0x55e4af43f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.086+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/1833610171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44e730 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.086+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1833610171 >> 172.21.15.26:0/1833610171 conn(0x55e4af432da0 msgr2=0x55e4af4381a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.087+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1833610171 shutdown_connections 2023-07-13T22:20:37.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.087+0000 7f45d90aeb80 1 -- 172.21.15.26:0/1833610171 wait complete. 2023-07-13T22:20:37.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.088+0000 7f45d90aeb80 1 -- start start 2023-07-13T22:20:37.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.088+0000 7f45d90aeb80 1 Processor -- start 2023-07-13T22:20:37.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af44c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af44dd70 con 0x55e4af43d4d0 2023-07-13T22:20:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af44dee0 con 0x55e4af43ec70 2023-07-13T22:20:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4af44e0a0 con 0x55e4af43de20 2023-07-13T22:20:37.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.089+0000 7f45d90aeb80 10 client.0 _ll_register_callbacks cb 0x7f45b0009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.090+0000 7f45bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.090+0000 7f45bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55450/0 (socket says 172.21.15.26:55450) 2023-07-13T22:20:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.090+0000 7f45bf7fe700 1 -- 172.21.15.26:0/668355143 learned_addr learned my addr 172.21.15.26:0/668355143 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:37.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.090+0000 7f45be7fc700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.091+0000 7f45be7fc700 1 -- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 msgr2=0x55e4af44d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.091+0000 7f45be7fc700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.091+0000 7f45be7fc700 1 -- 172.21.15.26:0/668355143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 msgr2=0x55e4af44c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.091+0000 7f45be7fc700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af44c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.091+0000 7f45be7fc700 1 -- 172.21.15.26:0/668355143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4af1a5c60 con 0x55e4af43de20 2023-07-13T22:20:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.092+0000 7f45be7fc700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44c820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005bf0 tx=0x7f45b40057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f02fc035dd2cfa1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.093+0000 7f45a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:37.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.093+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b4020040 con 0x55e4af43de20 2023-07-13T22:20:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.093+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f45b4000e40 con 0x55e4af43de20 2023-07-13T22:20:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.093+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45b4015070 con 0x55e4af43de20 2023-07-13T22:20:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.094+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45b4004bb0 con 0x55e4af43de20 2023-07-13T22:20:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.094+0000 7f45d90aeb80 10 client.26518 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.094+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e4af1bb2b0 con 0x55e4af43de20 2023-07-13T22:20:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.096+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45b4015070 con 0x55e4af43de20 2023-07-13T22:20:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.096+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45b4003680 con 0x55e4af43de20 2023-07-13T22:20:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 10 client.26518 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 20 client.26518 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af4a7550 0x55e4af4a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 4 client.26518 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:20:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.097+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4af34a050 con 0x55e4af4a7550 2023-07-13T22:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.098+0000 7f45beffd700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af4a7550 0x55e4af4a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.098+0000 7f45beffd700 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af4a7550 0x55e4af4a9930 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba4221ff2e865e9a server_cookie=1e22ea70140fb696 in_seq=0 out_seq=0 2023-07-13T22:20:37.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45a2ff5700 10 client.26518 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45a2ff5700 1 -- 172.21.15.26:0/668355143 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4af34a050 con 0x55e4af4a7550 2023-07-13T22:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45a2ff5700 10 client.26518 handle_command_reply: tid=0 2023-07-13T22:20:37.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 client.26518 shutdown 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 msgr2=0x55e4af44c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44c820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005bf0 tx=0x7f45b40057a0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 shutdown_connections 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e4af43d4d0 0x55e4af44c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e4af4a7550 0x55e4af4a9930 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e4af43ec70 0x55e4af44d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 --2- 172.21.15.26:0/668355143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4af43de20 0x55e4af44c820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.099+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 >> 172.21.15.26:0/668355143 conn(0x55e4af432da0 msgr2=0x55e4af445140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.100+0000 7f45a2ff5700 0 client.26518 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.100+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 shutdown_connections 2023-07-13T22:20:37.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.100+0000 7f45d90aeb80 1 -- 172.21.15.26:0/668355143 wait complete. 2023-07-13T22:20:37.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.103+0000 7f45d90aeb80 20 client.26518 trim_cache size 0 max 16384 2023-07-13T22:20:37.089 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.106+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 msgr2=0x7f45a80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.106+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 0x7f45a80a3800 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009810 tx=0x7f45b000f720 comp rx=0 tx=0).stop 2023-07-13T22:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.106+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 msgr2=0x7f45c007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.106+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f45b800efb0 tx=0x7f45b800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 shutdown_connections 2023-07-13T22:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45c0107d20 0x7f45c01abfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c01061a0 0x7f45c007db70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f45a80a1350 0x7f45a80a3800 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 --2- 172.21.15.26:0/2459509605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45c0107390 0x7f45c007e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:37.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.107+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 >> 172.21.15.26:0/2459509605 conn(0x7f45c00751c0 msgr2=0x7f45c0077400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:37.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.108+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 shutdown_connections 2023-07-13T22:20:37.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:37.108+0000 7f45c57ff700 1 -- 172.21.15.26:0/2459509605 wait complete. 2023-07-13T22:20:38.108 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:37 smithi117 ceph-mon[129074]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 84 op/s 2023-07-13T22:20:38.149 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:20:38.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:20:38.200 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:37 smithi026 ceph-mon[95647]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 84 op/s 2023-07-13T22:20:38.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:37 smithi121 ceph-mon[129187]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 275 MiB/s wr, 84 op/s 2023-07-13T22:20:38.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 -- 172.21.15.26:0/2789850277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814077210 msgr2=0x7fe814079690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 --2- 172.21.15.26:0/2789850277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814077210 0x7fe814079690 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0080b0 tx=0x7fe80c00d650 comp rx=0 tx=0).stop 2023-07-13T22:20:38.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 -- 172.21.15.26:0/2789850277 shutdown_connections 2023-07-13T22:20:38.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 --2- 172.21.15.26:0/2789850277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe814071200 0x7fe814071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 --2- 172.21.15.26:0/2789850277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814077210 0x7fe814079690 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 --2- 172.21.15.26:0/2789850277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814074850 0x7fe814076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.815+0000 7fe8189ff700 1 -- 172.21.15.26:0/2789850277 >> 172.21.15.26:0/2789850277 conn(0x7fe81406ca90 msgr2=0x7fe81406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.816+0000 7fe8189ff700 1 -- 172.21.15.26:0/2789850277 shutdown_connections 2023-07-13T22:20:38.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.828+0000 7fe8189ff700 1 -- 172.21.15.26:0/2789850277 wait complete. 2023-07-13T22:20:38.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.830+0000 7fe8189ff700 1 Processor -- start 2023-07-13T22:20:38.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 -- start start 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 0x7fe814082480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 0x7fe8140829c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe814077210 0x7fe8141bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8141bd2b0 con 0x7fe814077210 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8141ae230 con 0x7fe814071200 2023-07-13T22:20:38.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.831+0000 7fe8189ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8141ae3f0 con 0x7fe814074850 2023-07-13T22:20:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.832+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 0x7fe8140829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.833+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 0x7fe8140829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52398/0 (socket says 172.21.15.26:52398) 2023-07-13T22:20:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.833+0000 7fe80bfff700 1 -- 172.21.15.26:0/1307231636 learned_addr learned my addr 172.21.15.26:0/1307231636 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:38.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.833+0000 7fe81259c700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 0x7fe814082480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.834+0000 7fe81259c700 1 -- 172.21.15.26:0/1307231636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 msgr2=0x7fe8140829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.834+0000 7fe81259c700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 0x7fe8140829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.834+0000 7fe81259c700 1 -- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe814077210 msgr2=0x7fe8141bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.834+0000 7fe81259c700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe814077210 0x7fe8141bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.834+0000 7fe81259c700 1 -- 172.21.15.26:0/1307231636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c007ce0 con 0x7fe814071200 2023-07-13T22:20:38.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.835+0000 7fe81259c700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 0x7fe814082480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b450 tx=0x7fe80400b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.840+0000 7fe80b7fe700 1 -- 172.21.15.26:0/1307231636 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804011830 con 0x7fe814071200 2023-07-13T22:20:38.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.840+0000 7fe80b7fe700 1 -- 172.21.15.26:0/1307231636 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe804011e50 con 0x7fe814071200 2023-07-13T22:20:38.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.840+0000 7fe80b7fe700 1 -- 172.21.15.26:0/1307231636 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804010da0 con 0x7fe814071200 2023-07-13T22:20:38.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.840+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8141ae6d0 con 0x7fe814071200 2023-07-13T22:20:38.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.841+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8141aeb90 con 0x7fe814071200 2023-07-13T22:20:38.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.848+0000 7fe80b7fe700 1 -- 172.21.15.26:0/1307231636 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe804011990 con 0x7fe814071200 2023-07-13T22:20:38.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.864+0000 7fe82c332b80 1 Processor -- start 2023-07-13T22:20:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 -- start start 2023-07-13T22:20:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577aa8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577aa9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5779acd30 con 0x55c577aa8cf0 2023-07-13T22:20:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5779ffac0 con 0x55c577aa7550 2023-07-13T22:20:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.865+0000 7fe82c332b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5779f8f50 con 0x55c577aa7ea0 2023-07-13T22:20:38.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.866+0000 7fe80b7fe700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 0x7fe8000a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.867+0000 7fe80b7fe700 1 -- 172.21.15.26:0/1307231636 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe8040cc200 con 0x7fe814071200 2023-07-13T22:20:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.868+0000 7fe812d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.868+0000 7fe812d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55468/0 (socket says 172.21.15.26:55468) 2023-07-13T22:20:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.868+0000 7fe812d9d700 1 -- 172.21.15.26:0/2739861558 learned_addr learned my addr 172.21.15.26:0/2739861558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.869+0000 7fe812d9d700 1 -- 172.21.15.26:0/2739861558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 msgr2=0x55c577aa8760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.869+0000 7fe812d9d700 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577aa8760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.869+0000 7fe812d9d700 1 -- 172.21.15.26:0/2739861558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 msgr2=0x55c577aa9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.869+0000 7fe812d9d700 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577aa9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.869+0000 7fe812d9d700 1 -- 172.21.15.26:0/2739861558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c577826e60 con 0x55c577aa7550 2023-07-13T22:20:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.871+0000 7fe812d9d700 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe80c018f50 tx=0x7fe80c00f490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477b07629e58d9e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.871+0000 7fe812d9d700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 0x7fe8000a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.871+0000 7fe808ff9700 1 -- 172.21.15.26:0/2739861558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00fd00 con 0x55c577aa7550 2023-07-13T22:20:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.871+0000 7fe808ff9700 1 -- 172.21.15.26:0/2739861558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80c00fe60 con 0x55c577aa7550 2023-07-13T22:20:38.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.871+0000 7fe808ff9700 1 -- 172.21.15.26:0/2739861558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c005370 con 0x55c577aa7550 2023-07-13T22:20:38.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.872+0000 7fe82c332b80 1 -- 172.21.15.26:0/2739861558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 msgr2=0x55c577aa7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.872+0000 7fe82c332b80 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe80c018f50 tx=0x7fe80c00f490 comp rx=0 tx=0).stop 2023-07-13T22:20:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.874+0000 7fe812d9d700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 0x7fe8000a3800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe80c018f50 tx=0x7fe80c00f490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.884+0000 7fe82c332b80 1 -- 172.21.15.26:0/2739861558 shutdown_connections 2023-07-13T22:20:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.884+0000 7fe82c332b80 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577aa9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.884+0000 7fe82c332b80 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aa7960 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.884+0000 7fe82c332b80 1 --2- 172.21.15.26:0/2739861558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577aa8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.884+0000 7fe82c332b80 1 -- 172.21.15.26:0/2739861558 >> 172.21.15.26:0/2739861558 conn(0x55c577a9ce20 msgr2=0x55c577aa2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.885+0000 7fe82c332b80 1 -- 172.21.15.26:0/2739861558 shutdown_connections 2023-07-13T22:20:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.885+0000 7fe82c332b80 1 -- 172.21.15.26:0/2739861558 wait complete. 2023-07-13T22:20:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.886+0000 7fe82c332b80 1 -- start start 2023-07-13T22:20:38.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.886+0000 7fe82c332b80 1 Processor -- start 2023-07-13T22:20:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7ea0 0x55c577ab1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577ab8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577998b40 con 0x55c577aa8cf0 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577998840 con 0x55c577aa7ea0 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c57787c9a0 con 0x55c577aa7550 2023-07-13T22:20:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.887+0000 7fe82c332b80 10 client.0 _ll_register_callbacks cb 0x55c5779967e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.888+0000 7fe81259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.888+0000 7fe81259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52414/0 (socket says 172.21.15.26:52414) 2023-07-13T22:20:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.888+0000 7fe81259c700 1 -- 172.21.15.26:0/28214634 learned_addr learned my addr 172.21.15.26:0/28214634 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.893+0000 7fe81259c700 1 -- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7ea0 msgr2=0x55c577ab1740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:20:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.893+0000 7fe81259c700 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7ea0 0x55c577ab1740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.893+0000 7fe81259c700 1 -- 172.21.15.26:0/28214634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 msgr2=0x55c577ab8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.893+0000 7fe81259c700 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577ab8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.893+0000 7fe81259c700 1 -- 172.21.15.26:0/28214634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81407a0c0 con 0x55c577aa7550 2023-07-13T22:20:38.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.904+0000 7fe7f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.904+0000 7fe81259c700 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe8040a14e0 tx=0x7fe804098f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e1f932f3dae2c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.904+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8040995c0 con 0x55c577aa7550 2023-07-13T22:20:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.904+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe80409d5f0 con 0x55c577aa7550 2023-07-13T22:20:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.911+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c577952570 con 0x55c577aa7550 2023-07-13T22:20:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.918+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8040a1a10 con 0x55c577aa7550 2023-07-13T22:20:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.922+0000 7fe82c332b80 10 client.26527 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.922+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c57780fce0 con 0x55c577aa7550 2023-07-13T22:20:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.940+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c577952570 con 0x55c577aa7550 2023-07-13T22:20:38.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.940+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe804098bc0 con 0x55c577aa7550 2023-07-13T22:20:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 10 client.26527 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 10 client.26527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 20 client.26527 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577abcf60 0x55c577abf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 4 client.26527 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5779f1f30 con 0x55c577abcf60 2023-07-13T22:20:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.943+0000 7fe80bfff700 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577abcf60 0x55c577abf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.946+0000 7fe80bfff700 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577abcf60 0x55c577abf370 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aee558b8eab51111 server_cookie=4ea91438c23a191d in_seq=0 out_seq=0 2023-07-13T22:20:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.946+0000 7fe7f9ffb700 10 client.26527 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.947+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/28214634 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5779f1f30 con 0x55c577abcf60 2023-07-13T22:20:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.947+0000 7fe7f9ffb700 10 client.26527 handle_command_reply: tid=0 2023-07-13T22:20:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.947+0000 7fe82c332b80 1 client.26527 shutdown 2023-07-13T22:20:38.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 msgr2=0x55c577ab1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe8040a14e0 tx=0x7fe804098f20 comp rx=0 tx=0).stop 2023-07-13T22:20:38.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 shutdown_connections 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577ab8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577abcf60 0x55c577abf370 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7550 0x55c577ab1200 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 --2- 172.21.15.26:0/28214634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7ea0 0x55c577ab1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 >> 172.21.15.26:0/28214634 conn(0x55c577a9ce20 msgr2=0x55c577aa1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.948+0000 7fe7f9ffb700 0 client.26527 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.949+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 shutdown_connections 2023-07-13T22:20:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.949+0000 7fe82c332b80 1 -- 172.21.15.26:0/28214634 wait complete. 2023-07-13T22:20:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.950+0000 7fe82c332b80 20 client.26527 trim_cache size 0 max 16384 2023-07-13T22:20:38.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 Processor -- start 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 -- start start 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa7ea0 0x55c577ab8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa8cf0 0x55c577aabbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577abcaa0 con 0x55c577aa7ea0 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577ab0670 con 0x55c577aa7550 2023-07-13T22:20:38.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.990+0000 7fe82c332b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577ab2380 con 0x55c577aa8cf0 2023-07-13T22:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.992+0000 7fe81259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.992+0000 7fe81259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55486/0 (socket says 172.21.15.26:55486) 2023-07-13T22:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.992+0000 7fe81259c700 1 -- 172.21.15.26:0/1843785694 learned_addr learned my addr 172.21.15.26:0/1843785694 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:38.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.992+0000 7fe812d9d700 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa8cf0 0x55c577aabbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.992+0000 7fe80bfff700 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa7ea0 0x55c577ab8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.993+0000 7fe81259c700 1 -- 172.21.15.26:0/1843785694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa8cf0 msgr2=0x55c577aabbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.993+0000 7fe81259c700 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa8cf0 0x55c577aabbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.993+0000 7fe81259c700 1 -- 172.21.15.26:0/1843785694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa7ea0 msgr2=0x55c577ab8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.993+0000 7fe81259c700 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa7ea0 0x55c577ab8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.993+0000 7fe81259c700 1 -- 172.21.15.26:0/1843785694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c577a9e660 con 0x55c577aa7550 2023-07-13T22:20:38.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.994+0000 7fe81259c700 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8040992c0 tx=0x7fe804097670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a9b4c7589058d97 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.994+0000 7fe8097fa700 1 -- 172.21.15.26:0/1843785694 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804010f00 con 0x55c577aa7550 2023-07-13T22:20:38.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.994+0000 7fe8097fa700 1 -- 172.21.15.26:0/1843785694 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8040a17f0 con 0x55c577aa7550 2023-07-13T22:20:38.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.994+0000 7fe8097fa700 1 -- 172.21.15.26:0/1843785694 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804095430 con 0x55c577aa7550 2023-07-13T22:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 -- 172.21.15.26:0/1843785694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 msgr2=0x55c577aaf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe8040992c0 tx=0x7fe804097670 comp rx=0 tx=0).stop 2023-07-13T22:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 -- 172.21.15.26:0/1843785694 shutdown_connections 2023-07-13T22:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa8cf0 0x55c577aabbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577aaf0a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 --2- 172.21.15.26:0/1843785694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa7ea0 0x55c577ab8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.995+0000 7fe82c332b80 1 -- 172.21.15.26:0/1843785694 >> 172.21.15.26:0/1843785694 conn(0x55c577a9ce20 msgr2=0x55c577aaff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.996+0000 7fe82c332b80 1 -- 172.21.15.26:0/1843785694 shutdown_connections 2023-07-13T22:20:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.996+0000 7fe82c332b80 1 -- 172.21.15.26:0/1843785694 wait complete. 2023-07-13T22:20:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.996+0000 7fe82c332b80 1 -- start start 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 Processor -- start 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577ab5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577ab5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577b10e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577ab6170 con 0x55c577aa8cf0 2023-07-13T22:20:38.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577b11350 con 0x55c577aa7550 2023-07-13T22:20:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c577b11490 con 0x55c577aa7ea0 2023-07-13T22:20:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.997+0000 7fe82c332b80 10 client.0 _ll_register_callbacks cb 0x7fe804095920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.998+0000 7fe81259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577ab5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.998+0000 7fe81259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577ab5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55494/0 (socket says 172.21.15.26:55494) 2023-07-13T22:20:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.998+0000 7fe81259c700 1 -- 172.21.15.26:0/180873294 learned_addr learned my addr 172.21.15.26:0/180873294 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.998+0000 7fe80bfff700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577ab5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe812d9d700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577b10e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe80bfff700 1 -- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 msgr2=0x55c577ab5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe80bfff700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577ab5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe80bfff700 1 -- 172.21.15.26:0/180873294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 msgr2=0x55c577b10e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:38.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe80bfff700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577b10e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:38.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:38.999+0000 7fe80bfff700 1 -- 172.21.15.26:0/180873294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c57780fce0 con 0x55c577aa7ea0 2023-07-13T22:20:38.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.000+0000 7fe80bfff700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577ab5a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c577abd330 tx=0x7fe7fc0186e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5777cebf176ee087 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:38.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.000+0000 7fe7f9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.002+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc009d50 con 0x55c577aa7ea0 2023-07-13T22:20:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.003+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe7fc005a20 con 0x55c577aa7ea0 2023-07-13T22:20:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.003+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe80c00fe60 con 0x55c577aa7ea0 2023-07-13T22:20:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.004+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7fc009eb0 con 0x55c577aa7ea0 2023-07-13T22:20:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.004+0000 7fe82c332b80 10 client.26533 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:38.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.004+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c57784d2d0 con 0x55c577aa7ea0 2023-07-13T22:20:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.007+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe80c00fe60 con 0x55c577aa7ea0 2023-07-13T22:20:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.007+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe7fc015470 con 0x55c577aa7ea0 2023-07-13T22:20:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 10 client.26533 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 10 client.26533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 20 client.26533 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577b12e80 0x55c577b15290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 4 client.26533 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:20:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.008+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c5779b40d0 con 0x55c577b12e80 2023-07-13T22:20:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.009+0000 7fe81259c700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577b12e80 0x55c577b15290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:38.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.011+0000 7fe81259c700 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577b12e80 0x55c577b15290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f59477db7643c5fd server_cookie=226cf772b61fc535 in_seq=0 out_seq=0 2023-07-13T22:20:38.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.012+0000 7fe7f9ffb700 10 client.26533 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.015+0000 7fe7f9ffb700 1 -- 172.21.15.26:0/180873294 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5779b40d0 con 0x55c577b12e80 2023-07-13T22:20:39.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.015+0000 7fe7f9ffb700 10 client.26533 handle_command_reply: tid=0 2023-07-13T22:20:39.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.015+0000 7fe82c332b80 1 client.26533 shutdown 2023-07-13T22:20:39.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.016+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 msgr2=0x55c577ab5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.016+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577ab5a90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c577abd330 tx=0x7fe7fc0186e0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 shutdown_connections 2023-07-13T22:20:39.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c577aa8cf0 0x55c577b10e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c577b12e80 0x55c577b15290 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c577aa7550 0x55c577ab5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 --2- 172.21.15.26:0/180873294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c577aa7ea0 0x55c577ab5a90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 >> 172.21.15.26:0/180873294 conn(0x55c577a9ce20 msgr2=0x55c577aaf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe7f9ffb700 0 client.26533 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.018+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 shutdown_connections 2023-07-13T22:20:39.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.019+0000 7fe82c332b80 1 -- 172.21.15.26:0/180873294 wait complete. 2023-07-13T22:20:39.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.019+0000 7fe82c332b80 20 client.26533 trim_cache size 0 max 16384 2023-07-13T22:20:39.005 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:20:39.006 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:20:39.006 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "a071322e-d125-4a0f-adb6-997271a68f59", 2023-07-13T22:20:39.006 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:20:39.006 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:20:39.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.021+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 msgr2=0x7fe8000a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.021+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 0x7fe8000a3800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe80c018f50 tx=0x7fe80c00f490 comp rx=0 tx=0).stop 2023-07-13T22:20:39.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.021+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 msgr2=0x7fe814082480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.021+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 0x7fe814082480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b450 tx=0x7fe80400b760 comp rx=0 tx=0).stop 2023-07-13T22:20:39.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 shutdown_connections 2023-07-13T22:20:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe8000a1350 0x7fe8000a3800 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe814077210 0x7fe8141bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe814071200 0x7fe814082480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 --2- 172.21.15.26:0/1307231636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe814074850 0x7fe8140829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.022+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 >> 172.21.15.26:0/1307231636 conn(0x7fe81406ca90 msgr2=0x7fe814076050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.023+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 shutdown_connections 2023-07-13T22:20:39.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.023+0000 7fe8189ff700 1 -- 172.21.15.26:0/1307231636 wait complete. 2023-07-13T22:20:39.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:20:39.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.738+0000 7f77d65ff700 1 -- 172.21.15.26:0/3897541515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 msgr2=0x7f77d010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.738+0000 7f77d65ff700 1 --2- 172.21.15.26:0/3897541515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d010e1f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f77c801c0d0 tx=0x7f77c801c3e0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 -- 172.21.15.26:0/3897541515 shutdown_connections 2023-07-13T22:20:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 --2- 172.21.15.26:0/3897541515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d0070ee0 0x7f77d00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 --2- 172.21.15.26:0/3897541515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d010e1f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 --2- 172.21.15.26:0/3897541515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 0x7f77d010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 -- 172.21.15.26:0/3897541515 >> 172.21.15.26:0/3897541515 conn(0x7f77d006ca90 msgr2=0x7f77d006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 -- 172.21.15.26:0/3897541515 shutdown_connections 2023-07-13T22:20:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.740+0000 7f77d65ff700 1 -- 172.21.15.26:0/3897541515 wait complete. 2023-07-13T22:20:39.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.742+0000 7f77d65ff700 1 Processor -- start 2023-07-13T22:20:39.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.742+0000 7f77d65ff700 1 -- start start 2023-07-13T22:20:39.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d0070ee0 0x7f77d019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 0x7f77d01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01a39b0 con 0x7f77d0070ee0 2023-07-13T22:20:39.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01a3b20 con 0x7f77d0071830 2023-07-13T22:20:39.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.750+0000 7f77d65ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01a3ce0 con 0x7f77d010bd60 2023-07-13T22:20:39.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.751+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.751+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52440/0 (socket says 172.21.15.26:52440) 2023-07-13T22:20:39.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.751+0000 7f77d4b9c700 1 -- 172.21.15.26:0/2406057385 learned_addr learned my addr 172.21.15.26:0/2406057385 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:39.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.751+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 0x7f77d01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.753+0000 7f77d4b9c700 1 -- 172.21.15.26:0/2406057385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 msgr2=0x7f77d01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.753+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 0x7f77d01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.753+0000 7f77d4b9c700 1 -- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d0070ee0 msgr2=0x7f77d019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.753+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d0070ee0 0x7f77d019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.753+0000 7f77d4b9c700 1 -- 172.21.15.26:0/2406057385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c801ad10 con 0x7f77d010bd60 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.756+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77c801abc0 tx=0x7f77c8006250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.756+0000 7f77cd7fa700 1 -- 172.21.15.26:0/2406057385 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c801f040 con 0x7f77d010bd60 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.756+0000 7f77cd7fa700 1 -- 172.21.15.26:0/2406057385 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c8003e10 con 0x7f77d010bd60 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.756+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d01a3f60 con 0x7f77d010bd60 2023-07-13T22:20:39.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.756+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d01a4330 con 0x7f77d010bd60 2023-07-13T22:20:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.760+0000 7f77cd7fa700 1 -- 172.21.15.26:0/2406057385 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c802c8c0 con 0x7f77d010bd60 2023-07-13T22:20:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.765+0000 7f77e9e42b80 1 Processor -- start 2023-07-13T22:20:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.767+0000 7f77e9e42b80 1 -- start start 2023-07-13T22:20:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fa4d0 0x555b881fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fae20 0x555b881fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b880ffcb0 con 0x555b881fbc70 2023-07-13T22:20:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b88152a40 con 0x555b881fa4d0 2023-07-13T22:20:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.768+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8814bed0 con 0x555b881fae20 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.771+0000 7f77cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fae20 0x555b881fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.772+0000 7f77cf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fa4d0 0x555b881fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.772+0000 7f77cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.772+0000 7f77cf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45512/0 (socket says 172.21.15.26:45512) 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.772+0000 7f77cf7fe700 1 -- 172.21.15.26:0/2784565660 learned_addr learned my addr 172.21.15.26:0/2784565660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:39.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 -- 172.21.15.26:0/2784565660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fae20 msgr2=0x555b881fb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fae20 0x555b881fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 -- 172.21.15.26:0/2784565660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fa4d0 msgr2=0x555b881fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fa4d0 0x555b881fa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 -- 172.21.15.26:0/2784565660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b87f79de0 con 0x555b881fbc70 2023-07-13T22:20:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.773+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f77c400bef0 tx=0x7f77c400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f258de9b87b2bd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.774+0000 7f77b67fc700 1 -- 172.21.15.26:0/2784565660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004a30 con 0x555b881fbc70 2023-07-13T22:20:39.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.774+0000 7f77b67fc700 1 -- 172.21.15.26:0/2784565660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c40054e0 con 0x555b881fbc70 2023-07-13T22:20:39.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.774+0000 7f77b67fc700 1 -- 172.21.15.26:0/2784565660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400f750 con 0x555b881fbc70 2023-07-13T22:20:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.777+0000 7f77cd7fa700 1 -- 172.21.15.26:0/2406057385 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f77c8005110 con 0x7f77d010bd60 2023-07-13T22:20:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.779+0000 7f77cd7fa700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 0x7f77b80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.780+0000 7f77cd7fa700 1 -- 172.21.15.26:0/2406057385 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77c80de7c0 con 0x7f77d010bd60 2023-07-13T22:20:39.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 -- 172.21.15.26:0/2784565660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 msgr2=0x555b881fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f77c400bef0 tx=0x7f77c400b410 comp rx=0 tx=0).stop 2023-07-13T22:20:39.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 -- 172.21.15.26:0/2784565660 shutdown_connections 2023-07-13T22:20:39.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fbc70 0x555b881fc0e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fa4d0 0x555b881fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/2784565660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fae20 0x555b881fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.782+0000 7f77e9e42b80 1 -- 172.21.15.26:0/2784565660 >> 172.21.15.26:0/2784565660 conn(0x555b881efda0 msgr2=0x555b881f51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.783+0000 7f77e9e42b80 1 -- 172.21.15.26:0/2784565660 shutdown_connections 2023-07-13T22:20:39.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.783+0000 7f77e9e42b80 1 -- 172.21.15.26:0/2784565660 wait complete. 2023-07-13T22:20:39.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.784+0000 7f77cffff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 0x7f77b80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.784+0000 7f77e9e42b80 1 -- start start 2023-07-13T22:20:39.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.785+0000 7f77e9e42b80 1 Processor -- start 2023-07-13T22:20:39.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fa4d0 0x555b88204250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fae20 0x555b88204790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b880ebac0 con 0x555b881fa4d0 2023-07-13T22:20:39.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b880eb7c0 con 0x555b881fae20 2023-07-13T22:20:39.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b87fcf920 con 0x555b881fbc70 2023-07-13T22:20:39.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.786+0000 7f77e9e42b80 10 client.0 _ll_register_callbacks cb 0x555b880e9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:39.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.788+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.789+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52464/0 (socket says 172.21.15.26:52464) 2023-07-13T22:20:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.789+0000 7f77d4b9c700 1 -- 172.21.15.26:0/828642656 learned_addr learned my addr 172.21.15.26:0/828642656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.789+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fae20 0x555b88204790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.789+0000 7f77cffff700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fa4d0 0x555b88204250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.790+0000 7f77d4b9c700 1 -- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fae20 msgr2=0x555b88204790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.790+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fae20 0x555b88204790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.790+0000 7f77d4b9c700 1 -- 172.21.15.26:0/828642656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fa4d0 msgr2=0x555b88204250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.790+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fa4d0 0x555b88204250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.790+0000 7f77d4b9c700 1 -- 172.21.15.26:0/828642656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40123f0 con 0x555b881fbc70 2023-07-13T22:20:39.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.791+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f77c80dc270 tx=0x7f77c80afc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25894a021a8f1461 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.792+0000 7f77cffff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 0x7f77b80a3800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77d0117230 tx=0x7f77c000e750 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.793+0000 7f77af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:20:39.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.793+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80afea0 con 0x555b881fbc70 2023-07-13T22:20:39.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.793+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77c80ab300 con 0x555b881fbc70 2023-07-13T22:20:39.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.793+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b880a54f0 con 0x555b881fbc70 2023-07-13T22:20:39.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.794+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c80b3b40 con 0x555b881fbc70 2023-07-13T22:20:39.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.794+0000 7f77e9e42b80 10 client.26542 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:39.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.794+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555b87f62c60 con 0x555b881fbc70 2023-07-13T22:20:39.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.794+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b880a54f0 con 0x555b881fbc70 2023-07-13T22:20:39.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f77c8033ba0 con 0x555b881fbc70 2023-07-13T22:20:39.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 10 client.26542 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:39.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 20 client.26542 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b8820fd40 0x555b88212150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 4 client.26542 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:20:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.796+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b88144eb0 con 0x555b8820fd40 2023-07-13T22:20:39.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.797+0000 7f77cffff700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b8820fd40 0x555b88212150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.808+0000 7f77cffff700 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b8820fd40 0x555b88212150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ffe3502b49df977 server_cookie=f232fabd990f3dd9 in_seq=0 out_seq=0 2023-07-13T22:20:39.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.808+0000 7f77af7fe700 10 client.26542 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.810+0000 7f77af7fe700 1 -- 172.21.15.26:0/828642656 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b88144eb0 con 0x555b8820fd40 2023-07-13T22:20:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.810+0000 7f77af7fe700 10 client.26542 handle_command_reply: tid=0 2023-07-13T22:20:39.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.810+0000 7f77e9e42b80 1 client.26542 shutdown 2023-07-13T22:20:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.813+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 msgr2=0x555b8820b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.813+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f77c80dc270 tx=0x7f77c80afc40 comp rx=0 tx=0).stop 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 shutdown_connections 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fa4d0 0x555b88204250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b8820fd40 0x555b88212150 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fbc70 0x555b8820b7d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/828642656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fae20 0x555b88204790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.816+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 >> 172.21.15.26:0/828642656 conn(0x555b881efda0 msgr2=0x555b881f51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.817+0000 7f77af7fe700 0 client.26542 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.817+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 shutdown_connections 2023-07-13T22:20:39.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.817+0000 7f77e9e42b80 1 -- 172.21.15.26:0/828642656 wait complete. 2023-07-13T22:20:39.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.818+0000 7f77e9e42b80 20 client.26542 trim_cache size 0 max 16384 2023-07-13T22:20:39.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.865+0000 7f77e9e42b80 1 Processor -- start 2023-07-13T22:20:39.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 -- start start 2023-07-13T22:20:39.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b882020b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b8820b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b881f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b880a54f0 con 0x555b881fae20 2023-07-13T22:20:39.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b881fb570 con 0x555b881fbc70 2023-07-13T22:20:39.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.866+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b88203750 con 0x555b881fa4d0 2023-07-13T22:20:39.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.867+0000 7f77cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b882020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.867+0000 7f77cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b882020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:52480/0 (socket says 172.21.15.26:52480) 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.867+0000 7f77cffff700 1 -- 172.21.15.26:0/3008968493 learned_addr learned my addr 172.21.15.26:0/3008968493 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.867+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b881f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77cf7fe700 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b8820b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 -- 172.21.15.26:0/3008968493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 msgr2=0x555b882020b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b882020b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 -- 172.21.15.26:0/3008968493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 msgr2=0x555b8820b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b8820b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 -- 172.21.15.26:0/3008968493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b881fc310 con 0x555b881fbc70 2023-07-13T22:20:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.868+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b881f4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f77c8018be0 tx=0x7f77c80184f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61fbf23dbfcb2b07 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.869+0000 7f77b6ffd700 1 -- 172.21.15.26:0/3008968493 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80afea0 con 0x555b881fbc70 2023-07-13T22:20:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.869+0000 7f77b6ffd700 1 -- 172.21.15.26:0/3008968493 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c80aad60 con 0x555b881fbc70 2023-07-13T22:20:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.869+0000 7f77b6ffd700 1 -- 172.21.15.26:0/3008968493 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80ab100 con 0x555b881fbc70 2023-07-13T22:20:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 -- 172.21.15.26:0/3008968493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 msgr2=0x555b881f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b881f4fd0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f77c8018be0 tx=0x7f77c80184f0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 -- 172.21.15.26:0/3008968493 shutdown_connections 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b881f4fd0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b882020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/3008968493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b8820b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.870+0000 7f77e9e42b80 1 -- 172.21.15.26:0/3008968493 >> 172.21.15.26:0/3008968493 conn(0x555b881efda0 msgr2=0x555b88202ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.871+0000 7f77e9e42b80 1 -- 172.21.15.26:0/3008968493 shutdown_connections 2023-07-13T22:20:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.871+0000 7f77e9e42b80 1 -- 172.21.15.26:0/3008968493 wait complete. 2023-07-13T22:20:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.871+0000 7f77e9e42b80 1 -- start start 2023-07-13T22:20:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.872+0000 7f77e9e42b80 1 Processor -- start 2023-07-13T22:20:39.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b88209930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b88209e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8820ab70 con 0x555b881fae20 2023-07-13T22:20:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8820ace0 con 0x555b881fbc70 2023-07-13T22:20:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8820aea0 con 0x555b881fa4d0 2023-07-13T22:20:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.874+0000 7f77e9e42b80 10 client.0 _ll_register_callbacks cb 0x7f77c80ab570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:20:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.875+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.875+0000 7f77cffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b88209930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.875+0000 7f77d4b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55548/0 (socket says 172.21.15.26:55548) 2023-07-13T22:20:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.875+0000 7f77d4b9c700 1 -- 172.21.15.26:0/1040154069 learned_addr learned my addr 172.21.15.26:0/1040154069 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 -- 172.21.15.26:0/1040154069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 msgr2=0x555b88209930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b88209930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 -- 172.21.15.26:0/1040154069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 msgr2=0x555b88209e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b88209e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 -- 172.21.15.26:0/1040154069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b87f62c60 con 0x555b881fbc70 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77d4b9c700 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77c80a80c0 tx=0x7f77c80afc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc141e40e94f7406 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:20:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80afea0 con 0x555b881fbc70 2023-07-13T22:20:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.876+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f77c80aaed0 con 0x555b881fbc70 2023-07-13T22:20:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.877+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77c40054e0 con 0x555b881fbc70 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.877+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77c80c40c0 con 0x555b881fbc70 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.877+0000 7f77e9e42b80 10 client.26564 fetch_fsmap learned FSMap version 29 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.877+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555b87fa0250 con 0x555b881fbc70 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.877+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c40054e0 con 0x555b881fbc70 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f77c80cb230 con 0x555b881fbc70 2023-07-13T22:20:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 10 client.26564 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 10 client.26564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 20 client.26564 populate_metadata read hostname 'smithi026' 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b88264020 0x555b88266400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 4 client.26564 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555b88107050 con 0x555b88264020 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.878+0000 7f77cffff700 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b88264020 0x555b88266400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77cffff700 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b88264020 0x555b88266400 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6bda4c71e50f1a server_cookie=3b80e3fba7ec9b2e in_seq=0 out_seq=0 2023-07-13T22:20:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77af7fe700 10 client.26564 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77af7fe700 1 -- 172.21.15.26:0/1040154069 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555b88107050 con 0x555b88264020 2023-07-13T22:20:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77af7fe700 10 client.26564 handle_command_reply: tid=0 2023-07-13T22:20:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77e9e42b80 1 client.26564 shutdown 2023-07-13T22:20:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 msgr2=0x555b8820a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.879+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f77c80a80c0 tx=0x7f77c80afc40 comp rx=0 tx=0).stop 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 shutdown_connections 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555b88264020 0x555b88266400 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b881fbc70 0x555b8820a4a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b881fa4d0 0x555b88209930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 --2- 172.21.15.26:0/1040154069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b881fae20 0x555b88209e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77af7fe700 0 client.26564 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 >> 172.21.15.26:0/1040154069 conn(0x555b881efda0 msgr2=0x555b882020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 shutdown_connections 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.880+0000 7f77e9e42b80 1 -- 172.21.15.26:0/1040154069 wait complete. 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.881+0000 7f77e9e42b80 20 client.26564 trim_cache size 0 max 16384 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:20:39.869 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: "a071322e-d125-4a0f-adb6-997271a68f59": { 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "a071322e-d125-4a0f-adb6-997271a68f59", 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:20:39.870 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:20:39.871 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:20:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 msgr2=0x7f77b80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 0x7f77b80a3800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77d0117230 tx=0x7f77c000e750 comp rx=0 tx=0).stop 2023-07-13T22:20:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 msgr2=0x7f77d01a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:20:39.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f77c801abc0 tx=0x7f77c8006250 comp rx=0 tx=0).stop 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 shutdown_connections 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f77b80a1350 0x7f77b80a3800 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77d0070ee0 0x7f77d019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d010bd60 0x7f77d01a31e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 --2- 172.21.15.26:0/2406057385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77d0071830 0x7f77d01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:20:39.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 >> 172.21.15.26:0/2406057385 conn(0x7f77d006ca90 msgr2=0x7f77d006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:20:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.882+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 shutdown_connections 2023-07-13T22:20:39.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:20:39.883+0000 7f77d65ff700 1 -- 172.21.15.26:0/2406057385 wait complete. 2023-07-13T22:20:39.919 INFO:tasks.cephfs.filesystem:scrub status for tag:a071322e-d125-4a0f-adb6-997271a68f59 - {'path': '/', 'tag': 'a071322e-d125-4a0f-adb6-997271a68f59', 'options': 'recursive,force'} 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:20:40.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:39 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:20:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:20:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:20:40.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:20:40.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:39 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 89 op/s 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:40.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:39 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:20:42.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:41 smithi121 ceph-mon[129187]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 86 op/s 2023-07-13T22:20:42.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:41 smithi117 ceph-mon[129074]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 86 op/s 2023-07-13T22:20:42.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:41 smithi026 ceph-mon[95647]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 86 op/s 2023-07-13T22:20:44.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:44 smithi121 ceph-mon[129187]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 266 MiB/s wr, 85 op/s 2023-07-13T22:20:44.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:44 smithi117 ceph-mon[129074]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 266 MiB/s wr, 85 op/s 2023-07-13T22:20:44.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:44 smithi026 ceph-mon[95647]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 266 MiB/s wr, 85 op/s 2023-07-13T22:20:45.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:45 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:20:45.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:45 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:20:45.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:45 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:20:46.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:46 smithi117 ceph-mon[129074]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:20:46.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:46 smithi026 ceph-mon[95647]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:20:46.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:46 smithi121 ceph-mon[129187]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:20:47.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:47 smithi117 ceph-mon[129074]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:47.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:47 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:20:47.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:20:47.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:20:47.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:20:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:47 smithi026 ceph-mon[95647]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:47 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:20:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:20:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:20:47.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:20:47.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:47 smithi121 ceph-mon[129187]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 269 MiB/s wr, 84 op/s 2023-07-13T22:20:47.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:47 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:20:47.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:20:47.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:20:47.491 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:20:48.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:48 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:20:48.449 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:48 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:20:48.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:48 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:20:49.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:49 smithi121 ceph-mon[129187]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:20:49.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:49 smithi117 ceph-mon[129074]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:20:49.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:49 smithi026 ceph-mon[95647]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:20:51.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:51 smithi117 ceph-mon[129074]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:20:51.448 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:51 smithi026 ceph-mon[95647]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:20:51.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:51 smithi121 ceph-mon[129187]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 297 MiB/s wr, 94 op/s 2023-07-13T22:20:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:53 smithi117 ceph-mon[129074]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:20:53.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:53 smithi026 ceph-mon[95647]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:20:53.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:53 smithi121 ceph-mon[129187]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:20:55.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:55 smithi117 ceph-mon[129074]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 310 MiB/s wr, 98 op/s 2023-07-13T22:20:55.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:55 smithi026 ceph-mon[95647]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 310 MiB/s wr, 98 op/s 2023-07-13T22:20:55.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:55 smithi121 ceph-mon[129187]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 310 MiB/s wr, 98 op/s 2023-07-13T22:20:57.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:57 smithi121 ceph-mon[129187]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:20:57.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:57 smithi117 ceph-mon[129074]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:20:57.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:57 smithi026 ceph-mon[95647]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 274 MiB/s wr, 87 op/s 2023-07-13T22:20:59.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:20:59 smithi117 ceph-mon[129074]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:20:59.714 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:20:59 smithi026 ceph-mon[95647]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:20:59.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:20:59 smithi121 ceph-mon[129187]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:21:01.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:01 smithi117 ceph-mon[129074]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:21:01.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:01 smithi026 ceph-mon[95647]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:21:01.490 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:01 smithi121 ceph-mon[129187]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 298 MiB/s wr, 92 op/s 2023-07-13T22:21:03.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:03 smithi117 ceph-mon[129074]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:21:03.447 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:03 smithi026 ceph-mon[95647]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:21:03.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:03 smithi121 ceph-mon[129187]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 88 op/s 2023-07-13T22:21:05.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:05 smithi117 ceph-mon[129074]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:21:05.455 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:05 smithi026 ceph-mon[95647]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:21:05.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:05 smithi121 ceph-mon[129187]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 98 op/s 2023-07-13T22:21:07.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:07 smithi117 ceph-mon[129074]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 86 op/s 2023-07-13T22:21:07.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:07 smithi026 ceph-mon[95647]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 86 op/s 2023-07-13T22:21:07.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:07 smithi121 ceph-mon[129187]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 277 MiB/s wr, 86 op/s 2023-07-13T22:21:09.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:09 smithi117 ceph-mon[129074]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 93 op/s 2023-07-13T22:21:09.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:09 smithi026 ceph-mon[95647]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 93 op/s 2023-07-13T22:21:09.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:09 smithi121 ceph-mon[129187]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 290 MiB/s wr, 93 op/s 2023-07-13T22:21:09.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:21:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.558+0000 7fbec55ff700 1 -- 172.21.15.26:0/3790253410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0070ee0 msgr2=0x7fbec00712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.558+0000 7fbec55ff700 1 --2- 172.21.15.26:0/3790253410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0070ee0 0x7fbec00712f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009f90 tx=0x7fbeb800b330 comp rx=0 tx=0).stop 2023-07-13T22:21:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 -- 172.21.15.26:0/3790253410 shutdown_connections 2023-07-13T22:21:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 --2- 172.21.15.26:0/3790253410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec010bd60 0x7fbec010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 --2- 172.21.15.26:0/3790253410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0070ee0 0x7fbec00712f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 --2- 172.21.15.26:0/3790253410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec0071830 0x7fbec010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 -- 172.21.15.26:0/3790253410 >> 172.21.15.26:0/3790253410 conn(0x7fbec006ca90 msgr2=0x7fbec006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 -- 172.21.15.26:0/3790253410 shutdown_connections 2023-07-13T22:21:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.559+0000 7fbec55ff700 1 -- 172.21.15.26:0/3790253410 wait complete. 2023-07-13T22:21:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.561+0000 7fbec55ff700 1 Processor -- start 2023-07-13T22:21:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.561+0000 7fbec55ff700 1 -- start start 2023-07-13T22:21:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec0070ee0 0x7fbec01a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec010bd60 0x7fbec01abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec01ac3a0 con 0x7fbec010bd60 2023-07-13T22:21:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec010ec40 con 0x7fbec0070ee0 2023-07-13T22:21:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbec55ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec010edd0 con 0x7fbec0071830 2023-07-13T22:21:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbebe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbebe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38854/0 (socket says 172.21.15.26:38854) 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.562+0000 7fbebe7fc700 1 -- 172.21.15.26:0/4287686272 learned_addr learned my addr 172.21.15.26:0/4287686272 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.563+0000 7fbebe7fc700 1 -- 172.21.15.26:0/4287686272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec0070ee0 msgr2=0x7fbec01a4820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.563+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec0070ee0 0x7fbec01a4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.563+0000 7fbebe7fc700 1 -- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec010bd60 msgr2=0x7fbec01abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.563+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec010bd60 0x7fbec01abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.563+0000 7fbebe7fc700 1 -- 172.21.15.26:0/4287686272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8009c40 con 0x7fbec0071830 2023-07-13T22:21:10.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.564+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400d980 tx=0x7fbeb400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.565+0000 7fbea7fff700 1 -- 172.21.15.26:0/4287686272 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb400de60 con 0x7fbec0071830 2023-07-13T22:21:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.565+0000 7fbea7fff700 1 -- 172.21.15.26:0/4287686272 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb40044e0 con 0x7fbec0071830 2023-07-13T22:21:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.565+0000 7fbea7fff700 1 -- 172.21.15.26:0/4287686272 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4003b10 con 0x7fbec0071830 2023-07-13T22:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.566+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec010f0b0 con 0x7fbec0071830 2023-07-13T22:21:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.566+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec010f540 con 0x7fbec0071830 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.573+0000 7fbed8e72b80 1 Processor -- start 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 -- start start 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b18f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b094cb0 con 0x55e99b190c70 2023-07-13T22:21:10.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b0e7a40 con 0x55e99b18fe20 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.574+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b0e0ed0 con 0x55e99b18f4d0 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34962/0 (socket says 172.21.15.26:34962) 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 -- 172.21.15.26:0/3741252475 learned_addr learned my addr 172.21.15.26:0/3741252475 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b18f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 -- 172.21.15.26:0/3741252475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 msgr2=0x55e99b18f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b18f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 -- 172.21.15.26:0/3741252475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 msgr2=0x55e99b1910e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1910e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.575+0000 7fbebeffd700 1 -- 172.21.15.26:0/3741252475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e99af0ede0 con 0x55e99b18fe20 2023-07-13T22:21:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.576+0000 7fbebeffd700 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009380 tx=0x7fbeb80093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b18da5aeced02ef server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.576+0000 7fbea57fa700 1 -- 172.21.15.26:0/3741252475 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800e040 con 0x55e99b18fe20 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.576+0000 7fbea57fa700 1 -- 172.21.15.26:0/3741252475 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb8007660 con 0x55e99b18fe20 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.576+0000 7fbea57fa700 1 -- 172.21.15.26:0/3741252475 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800bd20 con 0x55e99b18fe20 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.577+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3741252475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 msgr2=0x55e99b1906e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.577+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009380 tx=0x7fbeb80093b0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3741252475 shutdown_connections 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b18f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3741252475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1906e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3741252475 >> 172.21.15.26:0/3741252475 conn(0x55e99b184da0 msgr2=0x55e99b18a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.578+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3741252475 shutdown_connections 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.579+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3741252475 wait complete. 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.580+0000 7fbed8e72b80 1 -- start start 2023-07-13T22:21:10.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.580+0000 7fbed8e72b80 1 Processor -- start 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18fe20 0x55e99b1bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b080ac0 con 0x55e99b190c70 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b0807c0 con 0x55e99b18f4d0 2023-07-13T22:21:10.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.581+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99af64920 con 0x55e99b18fe20 2023-07-13T22:21:10.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.582+0000 7fbed8e72b80 10 client.0 _ll_register_callbacks cb 0x55e99b07e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.582+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18fe20 0x55e99b1bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.582+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.583+0000 7fbebf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34972/0 (socket says 172.21.15.26:34972) 2023-07-13T22:21:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.583+0000 7fbebf7fe700 1 -- 172.21.15.26:0/3785840179 learned_addr learned my addr 172.21.15.26:0/3785840179 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.586+0000 7fbebf7fe700 1 -- 172.21.15.26:0/3785840179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18fe20 msgr2=0x55e99b1bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.586+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18fe20 0x55e99b1bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.586+0000 7fbebf7fe700 1 -- 172.21.15.26:0/3785840179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 msgr2=0x55e99b1c26e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:10.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.586+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1c26e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.587+0000 7fbebf7fe700 1 -- 172.21.15.26:0/3785840179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec0117640 con 0x55e99b18f4d0 2023-07-13T22:21:10.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.587+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00116f0 tx=0x7fbeb0011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7279a25955ff50 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.603+0000 7fbea2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:10.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.603+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004440 con 0x55e99b18f4d0 2023-07-13T22:21:10.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.603+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbeb0003680 con 0x55e99b18f4d0 2023-07-13T22:21:10.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.603+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e99b03a4f0 con 0x55e99b18f4d0 2023-07-13T22:21:10.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.604+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb00045a0 con 0x55e99b18f4d0 2023-07-13T22:21:10.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.604+0000 7fbed8e72b80 10 client.26573 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:10.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.604+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e99aef7c60 con 0x55e99b18f4d0 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.605+0000 7fbea7fff700 1 -- 172.21.15.26:0/4287686272 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbeb4009b90 con 0x7fbec0071830 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.608+0000 7fbea7fff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 0x7fbea80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.608+0000 7fbea7fff700 1 -- 172.21.15.26:0/4287686272 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbeb40cb7d0 con 0x7fbec0071830 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.609+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e99b03a4f0 con 0x55e99b18f4d0 2023-07-13T22:21:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.609+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeb0024b60 con 0x55e99b18f4d0 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 10 client.26573 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 10 client.26573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 20 client.26573 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b1c6f10 0x55e99b1c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 4 client.26573 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.610+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e99b0d9eb0 con 0x55e99b1c6f10 2023-07-13T22:21:10.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.611+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b1c6f10 0x55e99b1c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.612+0000 7fbebeffd700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 0x7fbea80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.613+0000 7fbebeffd700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 0x7fbea80a3710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8003f60 tx=0x7fbeb8005040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.613+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b1c6f10 0x55e99b1c9320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ffbbe4ad159b30 server_cookie=90404b0e99007d33 in_seq=0 out_seq=0 2023-07-13T22:21:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.613+0000 7fbea2ff5700 10 client.26573 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.614+0000 7fbea2ff5700 1 -- 172.21.15.26:0/3785840179 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e99b0d9eb0 con 0x55e99b1c6f10 2023-07-13T22:21:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.614+0000 7fbea2ff5700 10 client.26573 handle_command_reply: tid=0 2023-07-13T22:21:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.614+0000 7fbed8e72b80 1 client.26573 shutdown 2023-07-13T22:21:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 msgr2=0x55e99b1bb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00116f0 tx=0x7fbeb0011a00 comp rx=0 tx=0).stop 2023-07-13T22:21:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 shutdown_connections 2023-07-13T22:21:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b190c70 0x55e99b1c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b1c6f10 0x55e99b1c9320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18f4d0 0x55e99b1bb160 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3785840179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18fe20 0x55e99b1bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 >> 172.21.15.26:0/3785840179 conn(0x55e99b184da0 msgr2=0x55e99b1898b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.615+0000 7fbea2ff5700 0 client.26573 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.618+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 shutdown_connections 2023-07-13T22:21:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.620+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3785840179 wait complete. 2023-07-13T22:21:10.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.621+0000 7fbed8e72b80 20 client.26573 trim_cache size 0 max 16384 2023-07-13T22:21:10.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 Processor -- start 2023-07-13T22:21:10.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 -- start start 2023-07-13T22:21:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b184da0 0x55e99b193ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b18b680 con 0x55e99b184da0 2023-07-13T22:21:10.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b1c43d0 con 0x55e99b18fe20 2023-07-13T22:21:10.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.659+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b1ba610 con 0x55e99b18f4d0 2023-07-13T22:21:10.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.660+0000 7fbebe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.661+0000 7fbebe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38890/0 (socket says 172.21.15.26:38890) 2023-07-13T22:21:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.661+0000 7fbebe7fc700 1 -- 172.21.15.26:0/3131704743 learned_addr learned my addr 172.21.15.26:0/3131704743 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.661+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.661+0000 7fbebeffd700 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b184da0 0x55e99b193ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 -- 172.21.15.26:0/3131704743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 msgr2=0x55e99b1c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 -- 172.21.15.26:0/3131704743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b184da0 msgr2=0x55e99b193ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b184da0 0x55e99b193ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 -- 172.21.15.26:0/3131704743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e99af35250 con 0x55e99b18f4d0 2023-07-13T22:21:10.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.662+0000 7fbebe7fc700 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009520 tx=0x7fbeb40c91d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dfcd8904de94e27 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.663+0000 7fbea5ffb700 1 -- 172.21.15.26:0/3131704743 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb409cd50 con 0x55e99b18f4d0 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.663+0000 7fbea5ffb700 1 -- 172.21.15.26:0/3131704743 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb40a10e0 con 0x55e99b18f4d0 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.663+0000 7fbea5ffb700 1 -- 172.21.15.26:0/3131704743 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4098210 con 0x55e99b18f4d0 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3131704743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 msgr2=0x55e99b194010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009520 tx=0x7fbeb40c91d0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3131704743 shutdown_connections 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b184da0 0x55e99b193ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b18fe20 0x55e99b1c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/3131704743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b194010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.664+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3131704743 >> 172.21.15.26:0/3131704743 conn(0x55e99b186600 msgr2=0x55e99b196ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.665+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3131704743 shutdown_connections 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.665+0000 7fbed8e72b80 1 -- 172.21.15.26:0/3131704743 wait complete. 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.665+0000 7fbed8e72b80 1 -- start start 2023-07-13T22:21:10.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.666+0000 7fbed8e72b80 1 Processor -- start 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 0x55e99b1bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b1bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 0x55e99b21ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b1c0280 con 0x55e99b18fe20 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b21b2f0 con 0x55e99b184da0 2023-07-13T22:21:10.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e99b21b4b0 con 0x55e99b18f4d0 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.667+0000 7fbed8e72b80 10 client.0 _ll_register_callbacks cb 0x7fbeb0009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.668+0000 7fbebe7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b1bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.668+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 0x55e99b1bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.668+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 0x55e99b1bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34992/0 (socket says 172.21.15.26:34992) 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.668+0000 7fbebeffd700 1 -- 172.21.15.26:0/2238902425 learned_addr learned my addr 172.21.15.26:0/2238902425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:10.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.669+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 0x55e99b21ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.670+0000 7fbebf7fe700 1 -- 172.21.15.26:0/2238902425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 msgr2=0x55e99b1bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.670+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b1bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.670+0000 7fbebf7fe700 1 -- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 msgr2=0x55e99b1bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.670+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 0x55e99b1bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.670+0000 7fbebf7fe700 1 -- 172.21.15.26:0/2238902425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec010f540 con 0x55e99b18fe20 2023-07-13T22:21:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.671+0000 7fbebf7fe700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 0x55e99b21ad80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005b40 tx=0x7fbeb001aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc059de6396f662f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.672+0000 7fbea2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.672+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0003880 con 0x55e99b18fe20 2023-07-13T22:21:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.672+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbeb0000e40 con 0x55e99b18fe20 2023-07-13T22:21:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.672+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbeb8007660 con 0x55e99b18fe20 2023-07-13T22:21:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.673+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeb001a630 con 0x55e99b18fe20 2023-07-13T22:21:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.673+0000 7fbed8e72b80 10 client.16182 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:10.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.673+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e99af0d2b0 con 0x55e99b18fe20 2023-07-13T22:21:10.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.674+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbeb8007660 con 0x55e99b18fe20 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.674+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbeb00245d0 con 0x55e99b18fe20 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 20 client.16182 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b190c70 0x55e99b21edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 4 client.16182 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:21:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.675+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e99b09c050 con 0x55e99b190c70 2023-07-13T22:21:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.677+0000 7fbebeffd700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b190c70 0x55e99b21edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.682+0000 7fbebeffd700 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b190c70 0x55e99b21edb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8633a497b8296f7e server_cookie=4fccf143f0b4e21c in_seq=0 out_seq=0 2023-07-13T22:21:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.683+0000 7fbea2ff5700 10 client.16182 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:10.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.683+0000 7fbea2ff5700 1 -- 172.21.15.26:0/2238902425 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e99b09c050 con 0x55e99b190c70 2023-07-13T22:21:10.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.683+0000 7fbea2ff5700 10 client.16182 handle_command_reply: tid=0 2023-07-13T22:21:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.683+0000 7fbed8e72b80 1 client.16182 shutdown 2023-07-13T22:21:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.684+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 msgr2=0x55e99b21ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.684+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 0x55e99b21ad80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0005b40 tx=0x7fbeb001aa80 comp rx=0 tx=0).stop 2023-07-13T22:21:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 shutdown_connections 2023-07-13T22:21:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e99b18fe20 0x55e99b21ad80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55e99b190c70 0x55e99b21edb0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e99b184da0 0x55e99b1bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 --2- 172.21.15.26:0/2238902425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e99b18f4d0 0x55e99b1bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 >> 172.21.15.26:0/2238902425 conn(0x55e99b186600 msgr2=0x55e99b193ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.687+0000 7fbea2ff5700 0 client.16182 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.689+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 shutdown_connections 2023-07-13T22:21:10.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.689+0000 7fbed8e72b80 1 -- 172.21.15.26:0/2238902425 wait complete. 2023-07-13T22:21:10.676 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:10.676 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:21:10.676 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:21:10.676 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:10.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.690+0000 7fbed8e72b80 20 client.16182 trim_cache size 0 max 16384 2023-07-13T22:21:10.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.730+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 msgr2=0x7fbea80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.730+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 0x7fbea80a3710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8003f60 tx=0x7fbeb8005040 comp rx=0 tx=0).stop 2023-07-13T22:21:10.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.730+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 msgr2=0x7fbec01a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:10.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.730+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400d980 tx=0x7fbeb400dd40 comp rx=0 tx=0).stop 2023-07-13T22:21:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 shutdown_connections 2023-07-13T22:21:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbea80a1260 0x7fbea80a3710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbec010bd60 0x7fbec01abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbec0070ee0 0x7fbec01a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 --2- 172.21.15.26:0/4287686272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbec0071830 0x7fbec01a4d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.746+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 >> 172.21.15.26:0/4287686272 conn(0x7fbec006ca90 msgr2=0x7fbec006e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.754+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 shutdown_connections 2023-07-13T22:21:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:10.754+0000 7fbec55ff700 1 -- 172.21.15.26:0/4287686272 wait complete. 2023-07-13T22:21:10.852 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:21:10.853 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:21:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 -- 172.21.15.26:0/1899420074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 msgr2=0x7fc3e81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/1899420074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e81077e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009990 tx=0x7fc3e4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 -- 172.21.15.26:0/1899420074 shutdown_connections 2023-07-13T22:21:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/1899420074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e81061a0 0x7fc3e81065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/1899420074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e8107d20 0x7fc3e810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/1899420074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e81077e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.400+0000 7fc3f03ff700 1 -- 172.21.15.26:0/1899420074 >> 172.21.15.26:0/1899420074 conn(0x7fc3e80751f0 msgr2=0x7fc3e8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.401+0000 7fc3f03ff700 1 -- 172.21.15.26:0/1899420074 shutdown_connections 2023-07-13T22:21:11.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.401+0000 7fc3f03ff700 1 -- 172.21.15.26:0/1899420074 wait complete. 2023-07-13T22:21:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.402+0000 7fc3f03ff700 1 Processor -- start 2023-07-13T22:21:11.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 -- start start 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e81061a0 0x7fc3e806ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e8107d20 0x7fc3e819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8072390 con 0x7fc3e8107d20 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8072500 con 0x7fc3e81061a0 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.403+0000 7fc3f03ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e819b520 con 0x7fc3e8107390 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.405+0000 7fc3ed99a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.405+0000 7fc3ee19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e81061a0 0x7fc3e806ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.405+0000 7fc3ed99a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38928/0 (socket says 172.21.15.26:38928) 2023-07-13T22:21:11.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.405+0000 7fc3ed99a700 1 -- 172.21.15.26:0/901963208 learned_addr learned my addr 172.21.15.26:0/901963208 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:11.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.406+0000 7fc3ed99a700 1 -- 172.21.15.26:0/901963208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e81061a0 msgr2=0x7fc3e806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.406+0000 7fc3ed99a700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e81061a0 0x7fc3e806ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.406+0000 7fc3ed99a700 1 -- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e8107d20 msgr2=0x7fc3e819afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.406+0000 7fc3ed99a700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e8107d20 0x7fc3e819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.406+0000 7fc3ed99a700 1 -- 172.21.15.26:0/901963208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0009c40 con 0x7fc3e8107390 2023-07-13T22:21:11.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.407+0000 7fc3ed99a700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400b680 tx=0x7fc3e40057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.407+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e401d070 con 0x7fc3e8107390 2023-07-13T22:21:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.407+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e400fbc0 con 0x7fc3e8107390 2023-07-13T22:21:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.407+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4022ea0 con 0x7fc3e8107390 2023-07-13T22:21:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.407+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e40095f0 con 0x7fc3e8107390 2023-07-13T22:21:11.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.408+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e819b980 con 0x7fc3e8107390 2023-07-13T22:21:11.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.410+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc005320 con 0x7fc3e8107390 2023-07-13T22:21:11.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.414+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc3e4022470 con 0x7fc3e8107390 2023-07-13T22:21:11.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.417+0000 7fc3df7fe700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 0x7fc3d40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:11.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.417+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc3e40cda50 con 0x7fc3e8107390 2023-07-13T22:21:11.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.417+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3e409a880 con 0x7fc3e8107390 2023-07-13T22:21:11.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.418+0000 7fc3ee19b700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 0x7fc3d40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:11.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.420+0000 7fc3ee19b700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 0x7fc3d40a3800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8070250 tx=0x7fc3d800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:11.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:11 smithi117 ceph-mon[129074]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 290 MiB/s wr, 90 op/s 2023-07-13T22:21:11.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:11 smithi026 ceph-mon[95647]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 290 MiB/s wr, 90 op/s 2023-07-13T22:21:11.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:11 smithi121 ceph-mon[129187]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 290 MiB/s wr, 90 op/s 2023-07-13T22:21:11.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.949+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3cc005190 con 0x7fc3e8107390 2023-07-13T22:21:11.936 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:21:11.937 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:21:11.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.950+0000 7fc3df7fe700 1 -- 172.21.15.26:0/901963208 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fc3e4027070 con 0x7fc3e8107390 2023-07-13T22:21:11.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.955+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 msgr2=0x7fc3d40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.955+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 0x7fc3d40a3800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8070250 tx=0x7fc3d800b410 comp rx=0 tx=0).stop 2023-07-13T22:21:11.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.955+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 msgr2=0x7fc3e806f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:11.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.955+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400b680 tx=0x7fc3e40057b0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 shutdown_connections 2023-07-13T22:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc3d40a1350 0x7fc3d40a3800 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3e8107d20 0x7fc3e819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3e81061a0 0x7fc3e806ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 --2- 172.21.15.26:0/901963208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3e8107390 0x7fc3e806f240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.959+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 >> 172.21.15.26:0/901963208 conn(0x7fc3e80751f0 msgr2=0x7fc3e8077410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.960+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 shutdown_connections 2023-07-13T22:21:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:11.960+0000 7fc3f03ff700 1 -- 172.21.15.26:0/901963208 wait complete. 2023-07-13T22:21:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:21:11.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:21:12.392 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:12 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/901963208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:12.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.525+0000 7f66c55ff700 1 -- 172.21.15.26:0/97123227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0070ee0 msgr2=0x7f66c00712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.525+0000 7f66c55ff700 1 --2- 172.21.15.26:0/97123227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0070ee0 0x7f66c00712f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f66b800b0a0 tx=0x7f66b800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.526+0000 7f66c55ff700 1 -- 172.21.15.26:0/97123227 shutdown_connections 2023-07-13T22:21:12.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.526+0000 7f66c55ff700 1 --2- 172.21.15.26:0/97123227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c010bd60 0x7f66c010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.526+0000 7f66c55ff700 1 --2- 172.21.15.26:0/97123227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0070ee0 0x7f66c00712f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.526+0000 7f66c55ff700 1 --2- 172.21.15.26:0/97123227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c0071830 0x7f66c010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.526+0000 7f66c55ff700 1 -- 172.21.15.26:0/97123227 >> 172.21.15.26:0/97123227 conn(0x7f66c006ca90 msgr2=0x7f66c006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.527+0000 7f66c55ff700 1 -- 172.21.15.26:0/97123227 shutdown_connections 2023-07-13T22:21:12.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.527+0000 7f66c55ff700 1 -- 172.21.15.26:0/97123227 wait complete. 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.528+0000 7f66c55ff700 1 Processor -- start 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 -- start start 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0071830 0x7f66c0113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c010bd60 0x7f66c01b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0117090 con 0x7f66c010bd60 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c0117200 con 0x7f66c0071830 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.529+0000 7f66c55ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01b4e70 con 0x7f66c0070ee0 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.530+0000 7f66beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.532+0000 7f66beffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38940/0 (socket says 172.21.15.26:38940) 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.532+0000 7f66beffd700 1 -- 172.21.15.26:0/1018661078 learned_addr learned my addr 172.21.15.26:0/1018661078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.533+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0071830 0x7f66c0113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.533+0000 7f66beffd700 1 -- 172.21.15.26:0/1018661078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0071830 msgr2=0x7f66c0113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.533+0000 7f66beffd700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0071830 0x7f66c0113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.533+0000 7f66beffd700 1 -- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c010bd60 msgr2=0x7f66c01b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.533+0000 7f66beffd700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c010bd60 0x7f66c01b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.534+0000 7f66beffd700 1 -- 172.21.15.26:0/1018661078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8009d00 con 0x7f66c0070ee0 2023-07-13T22:21:12.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.534+0000 7f66beffd700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66b8006070 tx=0x7f66b80126d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.557+0000 7f66affff700 1 -- 172.21.15.26:0/1018661078 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b800e040 con 0x7f66c0070ee0 2023-07-13T22:21:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.557+0000 7f66affff700 1 -- 172.21.15.26:0/1018661078 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b8007e50 con 0x7f66c0070ee0 2023-07-13T22:21:12.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.557+0000 7f66affff700 1 -- 172.21.15.26:0/1018661078 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80055f0 con 0x7f66c0070ee0 2023-07-13T22:21:12.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.574+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c01b5010 con 0x7f66c0070ee0 2023-07-13T22:21:12.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.575+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c01b5400 con 0x7f66c0070ee0 2023-07-13T22:21:12.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.593+0000 7f66d8fb7b80 1 Processor -- start 2023-07-13T22:21:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.593+0000 7f66d8fb7b80 1 -- start start 2023-07-13T22:21:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab7d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab7e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aa81cb0 con 0x55a40ab7ce20 2023-07-13T22:21:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aad4a40 con 0x55a40ab7c4d0 2023-07-13T22:21:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aacded0 con 0x55a40ab7dc70 2023-07-13T22:21:12.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab7c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35042/0 (socket says 172.21.15.26:35042) 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.594+0000 7f66bf7fe700 1 -- 172.21.15.26:0/3704569441 learned_addr learned my addr 172.21.15.26:0/3704569441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab7e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 -- 172.21.15.26:0/3704569441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 msgr2=0x55a40ab7c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab7c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 -- 172.21.15.26:0/3704569441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 msgr2=0x55a40ab7d6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:12.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab7d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.595+0000 7f66be7fc700 1 -- 172.21.15.26:0/3704569441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40a8fbde0 con 0x55a40ab7dc70 2023-07-13T22:21:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.596+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab7e0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a8b0 tx=0x7f66b400abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca2aa51042e1408 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.597+0000 7f66ad7fa700 1 -- 172.21.15.26:0/3704569441 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011800 con 0x55a40ab7dc70 2023-07-13T22:21:12.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.597+0000 7f66ad7fa700 1 -- 172.21.15.26:0/3704569441 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b4011e20 con 0x55a40ab7dc70 2023-07-13T22:21:12.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.597+0000 7f66ad7fa700 1 -- 172.21.15.26:0/3704569441 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4018360 con 0x55a40ab7dc70 2023-07-13T22:21:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.599+0000 7f66affff700 1 -- 172.21.15.26:0/1018661078 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f66b8004cf0 con 0x7f66c0070ee0 2023-07-13T22:21:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.599+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3704569441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 msgr2=0x55a40ab7e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.599+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab7e0e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a8b0 tx=0x7f66b400abc0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.600+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3704569441 shutdown_connections 2023-07-13T22:21:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.600+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab7e0e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.600+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab7c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.600+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3704569441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab7d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.600+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3704569441 >> 172.21.15.26:0/3704569441 conn(0x55a40ab71da0 msgr2=0x55a40ab771d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.602+0000 7f66affff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 0x7f66a80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.603+0000 7f66affff700 1 -- 172.21.15.26:0/1018661078 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f66b80cd270 con 0x7f66c0070ee0 2023-07-13T22:21:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.607+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3704569441 shutdown_connections 2023-07-13T22:21:12.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.607+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3704569441 wait complete. 2023-07-13T22:21:12.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.608+0000 7f66d8fb7b80 1 -- start start 2023-07-13T22:21:12.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.609+0000 7f66d8fb7b80 1 Processor -- start 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab8d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aa6dac0 con 0x55a40ab7ce20 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aa6d7c0 con 0x55a40ab7c4d0 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40a951920 con 0x55a40ab7dc70 2023-07-13T22:21:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.610+0000 7f66d8fb7b80 10 client.0 _ll_register_callbacks cb 0x55a40aa6b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:12.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.613+0000 7f66be7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.613+0000 7f66be7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58192/0 (socket says 172.21.15.26:58192) 2023-07-13T22:21:12.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.613+0000 7f66be7fc700 1 -- 172.21.15.26:0/3755993122 learned_addr learned my addr 172.21.15.26:0/3755993122 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:12.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.614+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 0x7f66a80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66bf7fe700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab8d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66beffd700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66be7fc700 1 -- 172.21.15.26:0/3755993122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 msgr2=0x55a40ab8d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab8d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66be7fc700 1 -- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 msgr2=0x55a40ab86210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab86210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.615+0000 7f66be7fc700 1 -- 172.21.15.26:0/3755993122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b400a510 con 0x55a40ab7ce20 2023-07-13T22:21:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.616+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 0x7f66a80a3710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a120 tx=0x7f66b4022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.616+0000 7f66be7fc700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f66b4004790 tx=0x7f66b4011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5a4a09b1ffa34c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.620+0000 7f66a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:12.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.620+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4006c50 con 0x55a40ab7ce20 2023-07-13T22:21:12.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.620+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f66b4029470 con 0x55a40ab7ce20 2023-07-13T22:21:12.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.620+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a40aa274f0 con 0x55a40ab7ce20 2023-07-13T22:21:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.622+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66b4005a50 con 0x55a40ab7ce20 2023-07-13T22:21:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.622+0000 7f66d8fb7b80 10 client.16188 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.622+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a40a8e4c60 con 0x55a40ab7ce20 2023-07-13T22:21:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.622+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a40aa274f0 con 0x55a40ab7ce20 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.623+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f66b402f070 con 0x55a40ab7ce20 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 10 client.16188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 20 client.16188 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab91ee0 0x55a40ab942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 4 client.16188 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a40aac6eb0 con 0x55a40ab91ee0 2023-07-13T22:21:12.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.624+0000 7f66beffd700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab91ee0 0x55a40ab942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.625+0000 7f66a27fc700 10 client.16188 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:12.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.625+0000 7f66beffd700 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab91ee0 0x55a40ab942f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70d75933d0e72106 server_cookie=cf38b07a68b517f5 in_seq=0 out_seq=0 2023-07-13T22:21:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.625+0000 7f66a27fc700 1 -- 172.21.15.26:0/3755993122 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a40aac6eb0 con 0x55a40ab91ee0 2023-07-13T22:21:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.625+0000 7f66a27fc700 10 client.16188 handle_command_reply: tid=0 2023-07-13T22:21:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.625+0000 7f66d8fb7b80 1 client.16188 shutdown 2023-07-13T22:21:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 msgr2=0x55a40ab86750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f66b4004790 tx=0x7f66b4011be0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 shutdown_connections 2023-07-13T22:21:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab91ee0 0x55a40ab942f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7dc70 0x55a40ab8d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab86210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/3755993122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7ce20 0x55a40ab86750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 >> 172.21.15.26:0/3755993122 conn(0x55a40ab71da0 msgr2=0x55a40ab768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66a27fc700 0 client.16188 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.626+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 shutdown_connections 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.627+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/3755993122 wait complete. 2023-07-13T22:21:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.627+0000 7f66d8fb7b80 20 client.16188 trim_cache size 0 max 16384 2023-07-13T22:21:12.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.657+0000 7f66d8fb7b80 1 Processor -- start 2023-07-13T22:21:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 -- start start 2023-07-13T22:21:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab71da0 0x55a40ab858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40aa274f0 con 0x55a40ab71da0 2023-07-13T22:21:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ab87400 con 0x55a40ab7c4d0 2023-07-13T22:21:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.658+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ab719c0 con 0x55a40ab7ce20 2023-07-13T22:21:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.660+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.660+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38958/0 (socket says 172.21.15.26:38958) 2023-07-13T22:21:12.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.660+0000 7f66bf7fe700 1 -- 172.21.15.26:0/190519285 learned_addr learned my addr 172.21.15.26:0/190519285 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.660+0000 7f66be7fc700 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66beffd700 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab71da0 0x55a40ab858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66bf7fe700 1 -- 172.21.15.26:0/190519285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 msgr2=0x55a40ab8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66bf7fe700 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66bf7fe700 1 -- 172.21.15.26:0/190519285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab71da0 msgr2=0x55a40ab858c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66bf7fe700 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab71da0 0x55a40ab858c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.663+0000 7f66bf7fe700 1 -- 172.21.15.26:0/190519285 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a40a922250 con 0x55a40ab7ce20 2023-07-13T22:21:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.665+0000 7f66bf7fe700 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ab90 tx=0x7f66b000aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f717bd7212d1e88b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.665+0000 7f66adffb700 1 -- 172.21.15.26:0/190519285 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0012d90 con 0x55a40ab7ce20 2023-07-13T22:21:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.665+0000 7f66adffb700 1 -- 172.21.15.26:0/190519285 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b000f8d0 con 0x55a40ab7ce20 2023-07-13T22:21:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.665+0000 7f66adffb700 1 -- 172.21.15.26:0/190519285 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b001a850 con 0x55a40ab7ce20 2023-07-13T22:21:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.666+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/190519285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 msgr2=0x55a40ab840d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.666+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f66b000ab90 tx=0x7f66b000aea0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/190519285 shutdown_connections 2023-07-13T22:21:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab71da0 0x55a40ab858c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab840d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/190519285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab7c4d0 0x55a40ab8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/190519285 >> 172.21.15.26:0/190519285 conn(0x55a40ab73690 msgr2=0x55a40ab84eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/190519285 shutdown_connections 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.667+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/190519285 wait complete. 2023-07-13T22:21:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.668+0000 7f66d8fb7b80 1 -- start start 2023-07-13T22:21:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.668+0000 7f66d8fb7b80 1 Processor -- start 2023-07-13T22:21:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 0x55a40ab89860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 0x55a40ab89da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab8cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ab8d300 con 0x55a40ab7c4d0 2023-07-13T22:21:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ab8d470 con 0x55a40ab71da0 2023-07-13T22:21:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a40ab8d630 con 0x55a40ab7ce20 2023-07-13T22:21:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.669+0000 7f66d8fb7b80 10 client.0 _ll_register_callbacks cb 0x7f66b0005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.671+0000 7f66beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 0x55a40ab89860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.671+0000 7f66beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 0x55a40ab89860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:35064/0 (socket says 172.21.15.26:35064) 2023-07-13T22:21:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.671+0000 7f66beffd700 1 -- 172.21.15.26:0/1282560324 learned_addr learned my addr 172.21.15.26:0/1282560324 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.671+0000 7f66bf7fe700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab8cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 0x55a40ab89da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 -- 172.21.15.26:0/1282560324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 msgr2=0x55a40ab8cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab8cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 -- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 msgr2=0x55a40ab89860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 0x55a40ab89860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.672+0000 7f66be7fc700 1 -- 172.21.15.26:0/1282560324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b000a840 con 0x55a40ab7c4d0 2023-07-13T22:21:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.673+0000 7f66be7fc700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 0x55a40ab89da0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b400ac00 tx=0x7f66b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2edddaef1366ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.673+0000 7f66a27fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.673+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4029880 con 0x55a40ab7c4d0 2023-07-13T22:21:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.673+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f66b40299e0 con 0x55a40ab7c4d0 2023-07-13T22:21:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.674+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66b4011e20 con 0x55a40ab7c4d0 2023-07-13T22:21:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.674+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66b4006c50 con 0x55a40ab7c4d0 2023-07-13T22:21:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.674+0000 7f66d8fb7b80 10 client.16194 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.674+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a40a8fa2b0 con 0x55a40ab7c4d0 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.674+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b4049030 con 0x55a40ab7c4d0 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f66b402f050 con 0x55a40ab7c4d0 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 10 client.16194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 20 client.16194 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab7dc70 0x55a40abe8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 4 client.16194 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:12.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a40aa89050 con 0x55a40ab7dc70 2023-07-13T22:21:12.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.675+0000 7f66beffd700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab7dc70 0x55a40abe8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66beffd700 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab7dc70 0x55a40abe8c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=61b42c5b70c90fc4 server_cookie=155ba5539217dc32 in_seq=0 out_seq=0 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66a27fc700 10 client.16194 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66a27fc700 1 -- 172.21.15.26:0/1282560324 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a40aa89050 con 0x55a40ab7dc70 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66a27fc700 10 client.16194 handle_command_reply: tid=0 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66d8fb7b80 1 client.16194 shutdown 2023-07-13T22:21:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.676+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 msgr2=0x55a40ab89da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 0x55a40ab89da0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f66b400ac00 tx=0x7f66b400cc40 comp rx=0 tx=0).stop 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 shutdown_connections 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55a40ab7dc70 0x55a40abe8c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a40ab7ce20 0x55a40ab8cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a40ab71da0 0x55a40ab89860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 --2- 172.21.15.26:0/1282560324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a40ab7c4d0 0x55a40ab89da0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 >> 172.21.15.26:0/1282560324 conn(0x55a40ab73690 msgr2=0x55a40ab93ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66a27fc700 0 client.16194 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 shutdown_connections 2023-07-13T22:21:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.677+0000 7f66d8fb7b80 1 -- 172.21.15.26:0/1282560324 wait complete. 2023-07-13T22:21:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.678+0000 7f66d8fb7b80 20 client.16194 trim_cache size 0 max 16384 2023-07-13T22:21:12.665 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.678+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 msgr2=0x7f66a80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.678+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 0x7f66a80a3710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f66b400a120 tx=0x7f66b4022000 comp rx=0 tx=0).stop 2023-07-13T22:21:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.678+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 msgr2=0x7f66c0113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.678+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f66b8006070 tx=0x7f66b80126d0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 shutdown_connections 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f66a80a1260 0x7f66a80a3710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c010bd60 0x7f66c01b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66c0070ee0 0x7f66c0113a70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 --2- 172.21.15.26:0/1018661078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66c0071830 0x7f66c0113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 >> 172.21.15.26:0/1018661078 conn(0x7f66c006ca90 msgr2=0x7f66c006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 shutdown_connections 2023-07-13T22:21:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:12.679+0000 7f66c55ff700 1 -- 172.21.15.26:0/1018661078 wait complete. 2023-07-13T22:21:12.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:12 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/901963208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:12.693 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:21:12.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:12 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/901963208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.162+0000 7f1bef1ff700 1 -- 172.21.15.26:0/2282063655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8070ee0 msgr2=0x7f1be80712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.162+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/2282063655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8070ee0 0x7f1be80712f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1be000b0a0 tx=0x7f1be000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.165+0000 7f1bef1ff700 1 -- 172.21.15.26:0/2282063655 shutdown_connections 2023-07-13T22:21:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.165+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/2282063655 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.165+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/2282063655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8070ee0 0x7f1be80712f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.165+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/2282063655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be8071830 0x7f1be810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.165+0000 7f1bef1ff700 1 -- 172.21.15.26:0/2282063655 >> 172.21.15.26:0/2282063655 conn(0x7f1be806ca90 msgr2=0x7f1be806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.166+0000 7f1bef1ff700 1 -- 172.21.15.26:0/2282063655 shutdown_connections 2023-07-13T22:21:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.166+0000 7f1bef1ff700 1 -- 172.21.15.26:0/2282063655 wait complete. 2023-07-13T22:21:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.167+0000 7f1bef1ff700 1 Processor -- start 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 -- start start 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be8070ee0 0x7f1be8113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8071830 0x7f1be8113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8117090 con 0x7f1be8070ee0 2023-07-13T22:21:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be8117200 con 0x7f1be810bd60 2023-07-13T22:21:13.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.168+0000 7f1bef1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be81b4e70 con 0x7f1be8071830 2023-07-13T22:21:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.170+0000 7f1bed79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.171+0000 7f1be7fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8071830 0x7f1be8113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.171+0000 7f1bed79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:33996/0 (socket says 172.21.15.26:33996) 2023-07-13T22:21:13.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.171+0000 7f1bed79c700 1 -- 172.21.15.26:0/1706475357 learned_addr learned my addr 172.21.15.26:0/1706475357 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.173+0000 7f1bed79c700 1 -- 172.21.15.26:0/1706475357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8071830 msgr2=0x7f1be8113fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.173+0000 7f1bed79c700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8071830 0x7f1be8113fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.173+0000 7f1bed79c700 1 -- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be8070ee0 msgr2=0x7f1be8113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.173+0000 7f1bed79c700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be8070ee0 0x7f1be8113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.173+0000 7f1bed79c700 1 -- 172.21.15.26:0/1706475357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be0009d00 con 0x7f1be810bd60 2023-07-13T22:21:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1bed79c700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d980 tx=0x7f1bd800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1be5ffb700 1 -- 172.21.15.26:0/1706475357 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004a30 con 0x7f1be810bd60 2023-07-13T22:21:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1be5ffb700 1 -- 172.21.15.26:0/1706475357 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd800b5c0 con 0x7f1be810bd60 2023-07-13T22:21:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1be5ffb700 1 -- 172.21.15.26:0/1706475357 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8009af0 con 0x7f1be810bd60 2023-07-13T22:21:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be81b5070 con 0x7f1be810bd60 2023-07-13T22:21:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.174+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be81b55b0 con 0x7f1be810bd60 2023-07-13T22:21:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.193+0000 7f1c02a72b80 1 Processor -- start 2023-07-13T22:21:13.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.197+0000 7f1be5ffb700 1 -- 172.21.15.26:0/1706475357 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1bd8009c50 con 0x7f1be810bd60 2023-07-13T22:21:13.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.200+0000 7f1be5ffb700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 0x7f1bd00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.200+0000 7f1be5ffb700 1 -- 172.21.15.26:0/1706475357 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1bd80cc670 con 0x7f1be810bd60 2023-07-13T22:21:13.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.204+0000 7f1be7fff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 0x7f1bd00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.206+0000 7f1c02a72b80 1 -- start start 2023-07-13T22:21:13.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cbb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40cbc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40bbfcb0 con 0x55cb40cbbc70 2023-07-13T22:21:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40c12a40 con 0x55cb40cba4d0 2023-07-13T22:21:13.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.207+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40c0bed0 con 0x55cb40cbae20 2023-07-13T22:21:13.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.210+0000 7f1bed79c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40cbc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.213+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.213+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34010/0 (socket says 172.21.15.26:34010) 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.213+0000 7f1becf9b700 1 -- 172.21.15.26:0/4073720527 learned_addr learned my addr 172.21.15.26:0/4073720527 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.213+0000 7f1be7fff700 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cbb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.214+0000 7f1becf9b700 1 -- 172.21.15.26:0/4073720527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 msgr2=0x55cb40cbb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.214+0000 7f1becf9b700 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cbb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.214+0000 7f1becf9b700 1 -- 172.21.15.26:0/4073720527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 msgr2=0x55cb40cbc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.214+0000 7f1becf9b700 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40cbc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.214+0000 7f1becf9b700 1 -- 172.21.15.26:0/4073720527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb40a39de0 con 0x55cb40cba4d0 2023-07-13T22:21:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.215+0000 7f1becf9b700 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1be0007bd0 tx=0x7f1be0005a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f972a35144b9bc11 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.216+0000 7f1bd6ffd700 1 -- 172.21.15.26:0/4073720527 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000e040 con 0x55cb40cba4d0 2023-07-13T22:21:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.216+0000 7f1bd6ffd700 1 -- 172.21.15.26:0/4073720527 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be00093d0 con 0x55cb40cba4d0 2023-07-13T22:21:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.216+0000 7f1bd6ffd700 1 -- 172.21.15.26:0/4073720527 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be00052f0 con 0x55cb40cba4d0 2023-07-13T22:21:13.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 -- 172.21.15.26:0/4073720527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 msgr2=0x55cb40cba8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1be0007bd0 tx=0x7f1be0005a60 comp rx=0 tx=0).stop 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 -- 172.21.15.26:0/4073720527 shutdown_connections 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40cbc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cba8e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/4073720527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cbb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.217+0000 7f1c02a72b80 1 -- 172.21.15.26:0/4073720527 >> 172.21.15.26:0/4073720527 conn(0x55cb40cafda0 msgr2=0x55cb40cb51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.224+0000 7f1be7fff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 0x7f1bd00a3800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc012270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.226+0000 7f1c02a72b80 1 -- 172.21.15.26:0/4073720527 shutdown_connections 2023-07-13T22:21:13.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.226+0000 7f1c02a72b80 1 -- 172.21.15.26:0/4073720527 wait complete. 2023-07-13T22:21:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.227+0000 7f1c02a72b80 1 -- start start 2023-07-13T22:21:13.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.227+0000 7f1c02a72b80 1 Processor -- start 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cc4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40ccb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40babac0 con 0x55cb40cbbc70 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40bab7c0 con 0x55cb40cba4d0 2023-07-13T22:21:13.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40a8f920 con 0x55cb40cbae20 2023-07-13T22:21:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.228+0000 7f1c02a72b80 10 client.0 _ll_register_callbacks cb 0x55cb40ba9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.229+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.229+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34026/0 (socket says 172.21.15.26:34026) 2023-07-13T22:21:13.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.229+0000 7f1becf9b700 1 -- 172.21.15.26:0/1204167963 learned_addr learned my addr 172.21.15.26:0/1204167963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.229+0000 7f1be7fff700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 -- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 msgr2=0x55cb40cc4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cc4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 -- 172.21.15.26:0/1204167963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 msgr2=0x55cb40ccb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40ccb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 -- 172.21.15.26:0/1204167963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be8117640 con 0x55cb40cba4d0 2023-07-13T22:21:13.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.230+0000 7f1becf9b700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1be0003c30 tx=0x7f1be0003c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f50216f6e460a17 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.235+0000 7f1bcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:13.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.235+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000e040 con 0x55cb40cba4d0 2023-07-13T22:21:13.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.235+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1be00052e0 con 0x55cb40cba4d0 2023-07-13T22:21:13.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.235+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb40b654f0 con 0x55cb40cba4d0 2023-07-13T22:21:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.236+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be00054a0 con 0x55cb40cba4d0 2023-07-13T22:21:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.236+0000 7f1c02a72b80 10 client.26609 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:13.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.236+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cb40a22c60 con 0x55cb40cba4d0 2023-07-13T22:21:13.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cb40b654f0 con 0x55cb40cba4d0 2023-07-13T22:21:13.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1be0019070 con 0x55cb40cba4d0 2023-07-13T22:21:13.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 10 client.26609 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:13.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 10 client.26609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:21:13.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 20 client.26609 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:13.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40ccfbf0 0x55cb40cd2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 4 client.26609 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:13.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.238+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb40c04eb0 con 0x55cb40ccfbf0 2023-07-13T22:21:13.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.239+0000 7f1be7fff700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40ccfbf0 0x55cb40cd2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.241+0000 7f1bcbfff700 10 client.26609 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:13.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.241+0000 7f1be7fff700 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40ccfbf0 0x55cb40cd2000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=143e91eb5ec206fe server_cookie=f5b4c56087453d90 in_seq=0 out_seq=0 2023-07-13T22:21:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.242+0000 7f1bcbfff700 1 -- 172.21.15.26:0/1204167963 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb40c04eb0 con 0x55cb40ccfbf0 2023-07-13T22:21:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.242+0000 7f1bcbfff700 10 client.26609 handle_command_reply: tid=0 2023-07-13T22:21:13.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.242+0000 7f1c02a72b80 1 client.26609 shutdown 2023-07-13T22:21:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 msgr2=0x55cb40cc4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1be0003c30 tx=0x7f1be0003c60 comp rx=0 tx=0).stop 2023-07-13T22:21:13.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 shutdown_connections 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40ccfbf0 0x55cb40cd2000 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cbbc70 0x55cb40ccb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1bcbfff700 0 client.26609 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cba4d0 0x55cb40cc4140 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/1204167963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40cc4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 >> 172.21.15.26:0/1204167963 conn(0x55cb40cafda0 msgr2=0x55cb40cb48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 shutdown_connections 2023-07-13T22:21:13.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 1 -- 172.21.15.26:0/1204167963 wait complete. 2023-07-13T22:21:13.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.243+0000 7f1c02a72b80 20 client.26609 trim_cache size 0 max 16384 2023-07-13T22:21:13.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 Processor -- start 2023-07-13T22:21:13.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 -- start start 2023-07-13T22:21:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40ccb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbbc70 0x55cb40cbeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40cb1ec0 con 0x55cb40cba4d0 2023-07-13T22:21:13.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40cb64c0 con 0x55cb40cbbc70 2023-07-13T22:21:13.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.258+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40cc5170 con 0x55cb40cbae20 2023-07-13T22:21:13.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42200/0 (socket says 172.21.15.26:42200) 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1becf9b700 1 -- 172.21.15.26:0/3932355620 learned_addr learned my addr 172.21.15.26:0/3932355620 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1bed79c700 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbbc70 0x55cb40cbeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1becf9b700 1 -- 172.21.15.26:0/3932355620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 msgr2=0x55cb40ccb680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.259+0000 7f1becf9b700 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40ccb680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1becf9b700 1 -- 172.21.15.26:0/3932355620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbbc70 msgr2=0x55cb40cbeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1becf9b700 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbbc70 0x55cb40cbeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1becf9b700 1 -- 172.21.15.26:0/3932355620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb40a60250 con 0x55cb40cba4d0 2023-07-13T22:21:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1becf9b700 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1be000d010 tx=0x7f1be001dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96435a2066991492 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1bd77fe700 1 -- 172.21.15.26:0/3932355620 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000e070 con 0x55cb40cba4d0 2023-07-13T22:21:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1bd77fe700 1 -- 172.21.15.26:0/3932355620 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be0025440 con 0x55cb40cba4d0 2023-07-13T22:21:13.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.260+0000 7f1bd77fe700 1 -- 172.21.15.26:0/3932355620 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0005540 con 0x55cb40cba4d0 2023-07-13T22:21:13.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 -- 172.21.15.26:0/3932355620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 msgr2=0x55cb40cc37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1be000d010 tx=0x7f1be001dc50 comp rx=0 tx=0).stop 2023-07-13T22:21:13.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 -- 172.21.15.26:0/3932355620 shutdown_connections 2023-07-13T22:21:13.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc37f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbbc70 0x55cb40cbeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/3932355620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbae20 0x55cb40ccb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 -- 172.21.15.26:0/3932355620 >> 172.21.15.26:0/3932355620 conn(0x55cb40cafda0 msgr2=0x55cb40ccbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 -- 172.21.15.26:0/3932355620 shutdown_connections 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.261+0000 7f1c02a72b80 1 -- 172.21.15.26:0/3932355620 wait complete. 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.262+0000 7f1c02a72b80 1 -- start start 2023-07-13T22:21:13.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.262+0000 7f1c02a72b80 1 Processor -- start 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbae20 0x55cb40cc76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbbc70 0x55cb40cca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40ccac30 con 0x55cb40cba4d0 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40ccada0 con 0x55cb40cbae20 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb40ccaf60 con 0x55cb40cbbc70 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1c02a72b80 10 client.0 _ll_register_callbacks cb 0x7f1be0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:13.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1becf9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42216/0 (socket says 172.21.15.26:42216) 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1becf9b700 1 -- 172.21.15.26:0/2557023963 learned_addr learned my addr 172.21.15.26:0/2557023963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.263+0000 7f1bed79c700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbbc70 0x55cb40cca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1be7fff700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbae20 0x55cb40cc76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 -- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbbc70 msgr2=0x55cb40cca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbbc70 0x55cb40cca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 -- 172.21.15.26:0/2557023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbae20 msgr2=0x55cb40cc76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbae20 0x55cb40cc76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 -- 172.21.15.26:0/2557023963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb40a22c60 con 0x55cb40cba4d0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1becf9b700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1be00255b0 tx=0x7f1be0009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1fdbd258b3fa87 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1bcbfff700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.264+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be000e040 con 0x55cb40cba4d0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1be00093d0 con 0x55cb40cba4d0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1be00052e0 con 0x55cb40cba4d0 2023-07-13T22:21:13.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1be00054a0 con 0x55cb40cba4d0 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1c02a72b80 10 client.16200 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cb40a382b0 con 0x55cb40cba4d0 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.265+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1be00093d0 con 0x55cb40cba4d0 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1be000b0d0 con 0x55cb40cba4d0 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 10 client.16200 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 10 client.16200 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 20 client.16200 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40d24280 0x55cb40d26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 4 client.16200 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb40bc7050 con 0x55cb40d24280 2023-07-13T22:21:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.266+0000 7f1be7fff700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40d24280 0x55cb40d26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1be7fff700 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40d24280 0x55cb40d26690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb0ae850060bda5b server_cookie=106e03c10bb92b4b in_seq=0 out_seq=0 2023-07-13T22:21:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1bcbfff700 10 client.16200 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1bcbfff700 1 -- 172.21.15.26:0/2557023963 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb40bc7050 con 0x55cb40d24280 2023-07-13T22:21:13.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1bcbfff700 10 client.16200 handle_command_reply: tid=0 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 client.16200 shutdown 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 msgr2=0x55cb40cc7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1be00255b0 tx=0x7f1be0009050 comp rx=0 tx=0).stop 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 shutdown_connections 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55cb40d24280 0x55cb40d26690 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cb40cba4d0 0x55cb40cc7190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cb40cbbc70 0x55cb40cca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 --2- 172.21.15.26:0/2557023963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb40cbae20 0x55cb40cc76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 >> 172.21.15.26:0/2557023963 conn(0x55cb40cafda0 msgr2=0x55cb40cbeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1bcbfff700 0 client.16200 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.267+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 shutdown_connections 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1c02a72b80 1 -- 172.21.15.26:0/2557023963 wait complete. 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1c02a72b80 20 client.16200 trim_cache size 0 max 16384 2023-07-13T22:21:13.260 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:13.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 msgr2=0x7f1bd00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 0x7f1bd00a3800 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc006fd0 tx=0x7f1bdc012270 comp rx=0 tx=0).stop 2023-07-13T22:21:13.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 msgr2=0x7f1be81b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.268+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800d980 tx=0x7f1bd800dc90 comp rx=0 tx=0).stop 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 shutdown_connections 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1bd00a1350 0x7f1bd00a3800 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1be8070ee0 0x7f1be8113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1be810bd60 0x7f1be81b4930 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 --2- 172.21.15.26:0/1706475357 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be8071830 0x7f1be8113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:13.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 >> 172.21.15.26:0/1706475357 conn(0x7f1be806ca90 msgr2=0x7f1be806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 shutdown_connections 2023-07-13T22:21:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:13.269+0000 7f1bef1ff700 1 -- 172.21.15.26:0/1706475357 wait complete. 2023-07-13T22:21:13.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:21:13.569 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:13 smithi026 ceph-mon[95647]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 270 MiB/s wr, 86 op/s 2023-07-13T22:21:13.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:13 smithi117 ceph-mon[129074]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 270 MiB/s wr, 86 op/s 2023-07-13T22:21:13.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:13 smithi121 ceph-mon[129187]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 270 MiB/s wr, 86 op/s 2023-07-13T22:21:14.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.147+0000 7ff00d1ff700 1 -- 172.21.15.26:0/460700284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff00810bd60 msgr2=0x7ff00810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.147+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/460700284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff00810bd60 0x7ff00810e1f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b0a0 tx=0x7ff00000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 -- 172.21.15.26:0/460700284 shutdown_connections 2023-07-13T22:21:14.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/460700284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff008070ee0 0x7ff0080712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/460700284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff00810bd60 0x7ff00810e1f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/460700284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff008071830 0x7ff00810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 -- 172.21.15.26:0/460700284 >> 172.21.15.26:0/460700284 conn(0x7ff00806ca90 msgr2=0x7ff00806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 -- 172.21.15.26:0/460700284 shutdown_connections 2023-07-13T22:21:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.148+0000 7ff00d1ff700 1 -- 172.21.15.26:0/460700284 wait complete. 2023-07-13T22:21:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.150+0000 7ff00d1ff700 1 Processor -- start 2023-07-13T22:21:14.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.150+0000 7ff00d1ff700 1 -- start start 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff008071830 0x7ff008114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff00810bd60 0x7ff0081b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff008117160 con 0x7ff008071830 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081172d0 con 0x7ff00810bd60 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff00d1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0081b4d90 con 0x7ff008070ee0 2023-07-13T22:21:14.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41820/0 (socket says 172.21.15.26:41820) 2023-07-13T22:21:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.151+0000 7ff006d9d700 1 -- 172.21.15.26:0/3255901565 learned_addr learned my addr 172.21.15.26:0/3255901565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 -- 172.21.15.26:0/3255901565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff00810bd60 msgr2=0x7ff0081b4850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff00810bd60 0x7ff0081b4850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 -- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff008071830 msgr2=0x7ff008114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff008071830 0x7ff008114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 -- 172.21.15.26:0/3255901565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000009d00 con 0x7ff008070ee0 2023-07-13T22:21:14.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.152+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9d0 tx=0x7feffc00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.156+0000 7fefeffff700 1 -- 172.21.15.26:0/3255901565 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc009790 con 0x7ff008070ee0 2023-07-13T22:21:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.156+0000 7fefeffff700 1 -- 172.21.15.26:0/3255901565 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feffc010430 con 0x7ff008070ee0 2023-07-13T22:21:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.156+0000 7fefeffff700 1 -- 172.21.15.26:0/3255901565 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc018680 con 0x7ff008070ee0 2023-07-13T22:21:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.156+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0081b5070 con 0x7ff008070ee0 2023-07-13T22:21:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.156+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0081b5530 con 0x7ff008070ee0 2023-07-13T22:21:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.165+0000 7ff020abdb80 1 Processor -- start 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.167+0000 7ff020abdb80 1 -- start start 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.167+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb798e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.167+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.168+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb7b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.168+0000 7ff020abdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaca7ecb0 con 0x55ccacb7ac70 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.168+0000 7ff020abdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacad1a40 con 0x55ccacb794d0 2023-07-13T22:21:14.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.168+0000 7ff020abdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacacaed0 con 0x55ccacb79e20 2023-07-13T22:21:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.174+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.174+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41824/0 (socket says 172.21.15.26:41824) 2023-07-13T22:21:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.174+0000 7ff006d9d700 1 -- 172.21.15.26:0/3326638043 learned_addr learned my addr 172.21.15.26:0/3326638043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.177+0000 7ff006d9d700 1 -- 172.21.15.26:0/3326638043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 msgr2=0x55ccacb798e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.177+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb798e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.177+0000 7ff006d9d700 1 -- 172.21.15.26:0/3326638043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 msgr2=0x55ccacb7b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:14.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.177+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb7b0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.177+0000 7ff006d9d700 1 -- 172.21.15.26:0/3326638043 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccac8f8de0 con 0x55ccacb79e20 2023-07-13T22:21:14.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.185+0000 7fefeffff700 1 -- 172.21.15.26:0/3255901565 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7feffc0187e0 con 0x7ff008070ee0 2023-07-13T22:21:14.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.188+0000 7fefeffff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 0x7feff00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.189+0000 7fefeffff700 1 -- 172.21.15.26:0/3255901565 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feffc014070 con 0x7ff008070ee0 2023-07-13T22:21:14.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.189+0000 7ff006d9d700 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feffc010c70 tx=0x7feffc0109e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69626c79d3f73bf6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.196+0000 7fefed7fa700 1 -- 172.21.15.26:0/3326638043 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc04c070 con 0x55ccacb79e20 2023-07-13T22:21:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.196+0000 7fefed7fa700 1 -- 172.21.15.26:0/3326638043 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feffc097180 con 0x55ccacb79e20 2023-07-13T22:21:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.196+0000 7fefed7fa700 1 -- 172.21.15.26:0/3326638043 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0a3200 con 0x55ccacb79e20 2023-07-13T22:21:14.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.199+0000 7ff00759e700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 0x7feff00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.200+0000 7ff020abdb80 1 -- 172.21.15.26:0/3326638043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 msgr2=0x55ccacb7a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.200+0000 7ff020abdb80 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feffc010c70 tx=0x7feffc0109e0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.200+0000 7ff020abdb80 1 -- 172.21.15.26:0/3326638043 shutdown_connections 2023-07-13T22:21:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.200+0000 7ff020abdb80 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb7b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.201+0000 7ff020abdb80 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb798e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.201+0000 7ff020abdb80 1 --2- 172.21.15.26:0/3326638043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb7a6e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.201+0000 7ff020abdb80 1 -- 172.21.15.26:0/3326638043 >> 172.21.15.26:0/3326638043 conn(0x55ccacb6eda0 msgr2=0x55ccacb741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.201+0000 7ff020abdb80 1 -- 172.21.15.26:0/3326638043 shutdown_connections 2023-07-13T22:21:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.201+0000 7ff020abdb80 1 -- 172.21.15.26:0/3326638043 wait complete. 2023-07-13T22:21:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.202+0000 7ff020abdb80 1 -- start start 2023-07-13T22:21:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.202+0000 7ff00759e700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 0x7feff00a3750 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff000007a60 tx=0x7ff000006000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.202+0000 7ff020abdb80 1 Processor -- start 2023-07-13T22:21:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaca6aac0 con 0x55ccacb7ac70 2023-07-13T22:21:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccaca6a7c0 con 0x55ccacb794d0 2023-07-13T22:21:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccac94e920 con 0x55ccacb79e20 2023-07-13T22:21:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.203+0000 7ff020abdb80 10 client.0 _ll_register_callbacks cb 0x55ccaca68760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41832/0 (socket says 172.21.15.26:41832) 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 -- 172.21.15.26:0/86873957 learned_addr learned my addr 172.21.15.26:0/86873957 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00759e700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff006d9d700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 -- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 msgr2=0x55ccacb83120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 -- 172.21.15.26:0/86873957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 msgr2=0x55ccacb8a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.206+0000 7ff00659c700 1 -- 172.21.15.26:0/86873957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0081b5530 con 0x55ccacb79e20 2023-07-13T22:21:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.207+0000 7ff00659c700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feff800bd10 tx=0x7feff800d5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c079e73484900899 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.207+0000 7fefeaff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.207+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800be60 con 0x55ccacb79e20 2023-07-13T22:21:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.207+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feff8010460 con 0x55ccacb79e20 2023-07-13T22:21:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.207+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccaca244f0 con 0x55ccacb79e20 2023-07-13T22:21:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.208+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff8010620 con 0x55ccacb79e20 2023-07-13T22:21:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.208+0000 7ff020abdb80 10 client.26614 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.208+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ccac8e1c60 con 0x55ccacb79e20 2023-07-13T22:21:14.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.208+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ccaca244f0 con 0x55ccacb79e20 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feff801b9d0 con 0x55ccacb79e20 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 10 client.26614 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 10 client.26614 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 20 client.26614 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacb8f000 0x55ccacb91410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 4 client.26614 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccacac3eb0 con 0x55ccacb8f000 2023-07-13T22:21:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.209+0000 7ff006d9d700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacb8f000 0x55ccacb91410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.210+0000 7ff006d9d700 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacb8f000 0x55ccacb91410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b68f35900df3d45 server_cookie=b4e4af954ffa5df in_seq=0 out_seq=0 2023-07-13T22:21:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.210+0000 7fefeaff5700 10 client.26614 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7fefeaff5700 1 -- 172.21.15.26:0/86873957 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ccacac3eb0 con 0x55ccacb8f000 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7fefeaff5700 10 client.26614 handle_command_reply: tid=0 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 client.26614 shutdown 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 msgr2=0x55ccacb83660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feff800bd10 tx=0x7feff800d5c0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 shutdown_connections 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacb8f000 0x55ccacb91410 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 --2- 172.21.15.26:0/86873957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb79e20 0x55ccacb83660 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feff800bd10 tx=0x7feff800d5c0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.211+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 >> 172.21.15.26:0/86873957 conn(0x55ccacb6eda0 msgr2=0x55ccacb738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.212+0000 7fefeaff5700 0 client.26614 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.212+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 shutdown_connections 2023-07-13T22:21:14.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.213+0000 7ff020abdb80 1 -- 172.21.15.26:0/86873957 wait complete. 2023-07-13T22:21:14.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.213+0000 7ff020abdb80 20 client.26614 trim_cache size 0 max 16384 2023-07-13T22:21:14.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.242+0000 7ff020abdb80 1 Processor -- start 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.242+0000 7ff020abdb80 1 -- start start 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7b040 0x55ccacb8a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb83da0 con 0x55ccacb7b040 2023-07-13T22:21:14.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb842a0 con 0x55ccacb794d0 2023-07-13T22:21:14.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.243+0000 7ff020abdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb7a1f0 con 0x55ccacb7ac70 2023-07-13T22:21:14.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.246+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.246+0000 7ff006d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34076/0 (socket says 172.21.15.26:34076) 2023-07-13T22:21:14.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.246+0000 7ff006d9d700 1 -- 172.21.15.26:0/958794056 learned_addr learned my addr 172.21.15.26:0/958794056 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.246+0000 7ff00659c700 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.247+0000 7ff006d9d700 1 -- 172.21.15.26:0/958794056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 msgr2=0x55ccacb83660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.247+0000 7ff006d9d700 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb83660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.247+0000 7ff006d9d700 1 -- 172.21.15.26:0/958794056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7b040 msgr2=0x55ccacb8a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.247+0000 7ff006d9d700 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7b040 0x55ccacb8a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.247+0000 7ff006d9d700 1 -- 172.21.15.26:0/958794056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccac91f250 con 0x55ccacb794d0 2023-07-13T22:21:14.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.249+0000 7ff006d9d700 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feffc009350 tx=0x7feffc09ae40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ec51b2ac1ef575 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.250+0000 7fefedffb700 1 -- 172.21.15.26:0/958794056 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0a3200 con 0x55ccacb794d0 2023-07-13T22:21:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.250+0000 7fefedffb700 1 -- 172.21.15.26:0/958794056 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feffc097550 con 0x55ccacb794d0 2023-07-13T22:21:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.250+0000 7fefedffb700 1 -- 172.21.15.26:0/958794056 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0a2d70 con 0x55ccacb794d0 2023-07-13T22:21:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.251+0000 7ff020abdb80 1 -- 172.21.15.26:0/958794056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 msgr2=0x55ccacb83120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.251+0000 7ff020abdb80 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feffc009350 tx=0x7feffc09ae40 comp rx=0 tx=0).stop 2023-07-13T22:21:14.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.253+0000 7ff020abdb80 1 -- 172.21.15.26:0/958794056 shutdown_connections 2023-07-13T22:21:14.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.253+0000 7ff020abdb80 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb7b040 0x55ccacb8a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.253+0000 7ff020abdb80 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb794d0 0x55ccacb83120 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.253+0000 7ff020abdb80 1 --2- 172.21.15.26:0/958794056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb83660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.253+0000 7ff020abdb80 1 -- 172.21.15.26:0/958794056 >> 172.21.15.26:0/958794056 conn(0x55ccacb6eda0 msgr2=0x55ccacb91010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.258+0000 7ff020abdb80 1 -- 172.21.15.26:0/958794056 shutdown_connections 2023-07-13T22:21:14.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.258+0000 7ff020abdb80 1 -- 172.21.15.26:0/958794056 wait complete. 2023-07-13T22:21:14.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.259+0000 7ff020abdb80 1 -- start start 2023-07-13T22:21:14.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.259+0000 7ff020abdb80 1 Processor -- start 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 0x55ccacb89f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 0x55ccacb869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb86fa0 con 0x55ccacb794d0 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb870e0 con 0x55ccacb7b040 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccacb872a0 con 0x55ccacb7ac70 2023-07-13T22:21:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.260+0000 7ff020abdb80 10 client.0 _ll_register_callbacks cb 0x7feff800b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:14.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.261+0000 7ff00759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 0x55ccacb869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.261+0000 7ff00759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 0x55ccacb869d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34078/0 (socket says 172.21.15.26:34078) 2023-07-13T22:21:14.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.261+0000 7ff00759e700 1 -- 172.21.15.26:0/196177113 learned_addr learned my addr 172.21.15.26:0/196177113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.261+0000 7ff00659c700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.263+0000 7ff006d9d700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 0x55ccacb89f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.265+0000 7ff006d9d700 1 -- 172.21.15.26:0/196177113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 msgr2=0x55ccacb8a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.265+0000 7ff006d9d700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.265+0000 7ff006d9d700 1 -- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 msgr2=0x55ccacb869d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.265+0000 7ff006d9d700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 0x55ccacb869d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.265+0000 7ff006d9d700 1 -- 172.21.15.26:0/196177113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccac8f8de0 con 0x55ccacb794d0 2023-07-13T22:21:14.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.270+0000 7ff006d9d700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 0x55ccacb89f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feffc0972f0 tx=0x7feffc010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e519e88cfb811ce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.272+0000 7fefeaff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:14.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.272+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc0a3200 con 0x55ccacb794d0 2023-07-13T22:21:14.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.272+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7feffc09a430 con 0x55ccacb794d0 2023-07-13T22:21:14.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.273+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feffc097180 con 0x55ccacb794d0 2023-07-13T22:21:14.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.273+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc09a5f0 con 0x55ccacb794d0 2023-07-13T22:21:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.274+0000 7ff020abdb80 10 client.16209 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:14.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.274+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ccac8f72b0 con 0x55ccacb794d0 2023-07-13T22:21:14.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feffc097180 con 0x55ccacb794d0 2023-07-13T22:21:14.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feffc096c30 con 0x55ccacb794d0 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 10 client.16209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 20 client.16209 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacbe3e90 0x55ccacbe62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 4 client.16209 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.277+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccaca86050 con 0x55ccacbe3e90 2023-07-13T22:21:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.278+0000 7ff00659c700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacbe3e90 0x55ccacbe62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.280+0000 7ff00659c700 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacbe3e90 0x55ccacbe62a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0af285ff44e324c server_cookie=d0fe7752a0a11c36 in_seq=0 out_seq=0 2023-07-13T22:21:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.280+0000 7fefeaff5700 10 client.16209 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.284+0000 7fefeaff5700 1 -- 172.21.15.26:0/196177113 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccaca86050 con 0x55ccacbe3e90 2023-07-13T22:21:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.284+0000 7fefeaff5700 10 client.16209 handle_command_reply: tid=0 2023-07-13T22:21:14.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.284+0000 7ff020abdb80 1 client.16209 shutdown 2023-07-13T22:21:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.289+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 msgr2=0x55ccacb89f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.289+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 0x55ccacb89f30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feffc0972f0 tx=0x7feffc010990 comp rx=0 tx=0).stop 2023-07-13T22:21:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 shutdown_connections 2023-07-13T22:21:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ccacb794d0 0x55ccacb89f30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55ccacbe3e90 0x55ccacbe62a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ccacb7b040 0x55ccacb869d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 --2- 172.21.15.26:0/196177113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ccacb7ac70 0x55ccacb8a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 >> 172.21.15.26:0/196177113 conn(0x55ccacb6eda0 msgr2=0x55ccacb7da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.290+0000 7fefeaff5700 0 client.16209 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.291+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 shutdown_connections 2023-07-13T22:21:14.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.291+0000 7ff020abdb80 1 -- 172.21.15.26:0/196177113 wait complete. 2023-07-13T22:21:14.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.296+0000 7ff020abdb80 20 client.16209 trim_cache size 0 max 16384 2023-07-13T22:21:14.282 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:14.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.297+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 msgr2=0x7feff00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.297+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 0x7feff00a3750 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff000007a60 tx=0x7ff000006000 comp rx=0 tx=0).stop 2023-07-13T22:21:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.297+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 msgr2=0x7ff008113b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.297+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feffc00e9d0 tx=0x7feffc00ed90 comp rx=0 tx=0).stop 2023-07-13T22:21:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 shutdown_connections 2023-07-13T22:21:14.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff00810bd60 0x7ff0081b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff008070ee0 0x7ff008113b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7feff00a12a0 0x7feff00a3750 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 --2- 172.21.15.26:0/3255901565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff008071830 0x7ff008114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.298+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 >> 172.21.15.26:0/3255901565 conn(0x7ff00806ca90 msgr2=0x7ff00806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.302+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 shutdown_connections 2023-07-13T22:21:14.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.304+0000 7ff00d1ff700 1 -- 172.21.15.26:0/3255901565 wait complete. 2023-07-13T22:21:14.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:21:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 -- 172.21.15.26:0/908125725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88200716f0 msgr2=0x7f8820071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 --2- 172.21.15.26:0/908125725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88200716f0 0x7f8820071b00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f881800b0a0 tx=0x7f881800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 -- 172.21.15.26:0/908125725 shutdown_connections 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 --2- 172.21.15.26:0/908125725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88201080d0 0x7f882010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 --2- 172.21.15.26:0/908125725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88200716f0 0x7f8820071b00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 --2- 172.21.15.26:0/908125725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f8820107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 -- 172.21.15.26:0/908125725 >> 172.21.15.26:0/908125725 conn(0x7f882006cd80 msgr2=0x7f882006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 -- 172.21.15.26:0/908125725 shutdown_connections 2023-07-13T22:21:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.897+0000 7f88261ff700 1 -- 172.21.15.26:0/908125725 wait complete. 2023-07-13T22:21:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.899+0000 7f88261ff700 1 Processor -- start 2023-07-13T22:21:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.899+0000 7f88261ff700 1 -- start start 2023-07-13T22:21:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f882010c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88201080d0 0x7f8820108e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201095e0 con 0x7f8820107720 2023-07-13T22:21:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820109750 con 0x7f88201080d0 2023-07-13T22:21:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.900+0000 7f88261ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820109910 con 0x7f88200716f0 2023-07-13T22:21:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41868/0 (socket says 172.21.15.26:41868) 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 -- 172.21.15.26:0/4017370002 learned_addr learned my addr 172.21.15.26:0/4017370002 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881effd700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f882010c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881ffff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88201080d0 0x7f8820108e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 -- 172.21.15.26:0/4017370002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88201080d0 msgr2=0x7f8820108e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88201080d0 0x7f8820108e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 -- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 msgr2=0x7f882010c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f882010c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.901+0000 7f881f7fe700 1 -- 172.21.15.26:0/4017370002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8818009d00 con 0x7f88200716f0 2023-07-13T22:21:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.902+0000 7f881effd700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f882010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.902+0000 7f881f7fe700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f881800b830 tx=0x7f8818004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.903+0000 7f881cff9700 1 -- 172.21.15.26:0/4017370002 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800e040 con 0x7f88200716f0 2023-07-13T22:21:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.903+0000 7f881cff9700 1 -- 172.21.15.26:0/4017370002 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f881801db40 con 0x7f88200716f0 2023-07-13T22:21:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.903+0000 7f881cff9700 1 -- 172.21.15.26:0/4017370002 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881801bc00 con 0x7f88200716f0 2023-07-13T22:21:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.903+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88201a0080 con 0x7f88200716f0 2023-07-13T22:21:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.903+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201a05c0 con 0x7f88200716f0 2023-07-13T22:21:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.908+0000 7f881cff9700 1 -- 172.21.15.26:0/4017370002 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f881801d680 con 0x7f88200716f0 2023-07-13T22:21:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.911+0000 7f8839bccb80 1 Processor -- start 2023-07-13T22:21:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.911+0000 7f881cff9700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 0x7f88080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.911+0000 7f8839bccb80 1 -- start start 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f881cff9700 1 -- 172.21.15.26:0/4017370002 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f88180ce4f0 con 0x7f88200716f0 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe4659e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659fc70 0x55fe465a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe464a3cb0 con 0x55fe4659fc70 2023-07-13T22:21:14.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe464f6a40 con 0x55fe4659ee20 2023-07-13T22:21:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f8839bccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe464efed0 con 0x55fe4659e4d0 2023-07-13T22:21:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.912+0000 7f881effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34088/0 (socket says 172.21.15.26:34088) 2023-07-13T22:21:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 -- 172.21.15.26:0/931871217 learned_addr learned my addr 172.21.15.26:0/931871217 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe4659e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 -- 172.21.15.26:0/931871217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 msgr2=0x55fe4659e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe4659e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 -- 172.21.15.26:0/931871217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659fc70 msgr2=0x55fe465a00e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659fc70 0x55fe465a00e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 -- 172.21.15.26:0/931871217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4631dde0 con 0x55fe4659ee20 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881ffff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 0x7f88080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.913+0000 7f881effd700 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8810009ef0 tx=0x7f881000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa22e77baa8f308f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.914+0000 7f8805ffb700 1 -- 172.21.15.26:0/931871217 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000ed90 con 0x55fe4659ee20 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.914+0000 7f8805ffb700 1 -- 172.21.15.26:0/931871217 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8810013930 con 0x55fe4659ee20 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.914+0000 7f8805ffb700 1 -- 172.21.15.26:0/931871217 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810012890 con 0x55fe4659ee20 2023-07-13T22:21:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 -- 172.21.15.26:0/931871217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 msgr2=0x55fe4659f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8810009ef0 tx=0x7f881000e4b0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f881ffff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 0x7f88080a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f881400ab80 tx=0x7f881400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 -- 172.21.15.26:0/931871217 shutdown_connections 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659fc70 0x55fe465a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe4659e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 --2- 172.21.15.26:0/931871217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe4659f6e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 -- 172.21.15.26:0/931871217 >> 172.21.15.26:0/931871217 conn(0x55fe46593da0 msgr2=0x55fe465991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 -- 172.21.15.26:0/931871217 shutdown_connections 2023-07-13T22:21:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.915+0000 7f8839bccb80 1 -- 172.21.15.26:0/931871217 wait complete. 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.916+0000 7f8839bccb80 1 -- start start 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.916+0000 7f8839bccb80 1 Processor -- start 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659e4d0 0x55fe465a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4648fac0 con 0x55fe4659e4d0 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4648f7c0 con 0x55fe4659ee20 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe46373920 con 0x55fe4659fc70 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f8839bccb80 10 client.0 _ll_register_callbacks cb 0x55fe4648d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f881effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f881effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34090/0 (socket says 172.21.15.26:34090) 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.917+0000 7f881effd700 1 -- 172.21.15.26:0/3408246001 learned_addr learned my addr 172.21.15.26:0/3408246001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 -- 172.21.15.26:0/3408246001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 msgr2=0x55fe465af6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465af6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 -- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659e4d0 msgr2=0x55fe465a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659e4d0 0x55fe465a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 -- 172.21.15.26:0/3408246001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8810009b20 con 0x55fe4659ee20 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f881effd700 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8810006b30 tx=0x7f8810006ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6bc980eb45f11a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f87feffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810012810 con 0x55fe4659ee20 2023-07-13T22:21:14.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.918+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe464494f0 con 0x55fe4659ee20 2023-07-13T22:21:14.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8810012970 con 0x55fe4659ee20 2023-07-13T22:21:14.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8810012b30 con 0x55fe4659ee20 2023-07-13T22:21:14.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f8839bccb80 10 client.26639 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:14.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe46306c60 con 0x55fe4659ee20 2023-07-13T22:21:14.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe464494f0 con 0x55fe4659ee20 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8810024030 con 0x55fe4659ee20 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f8839bccb80 10 client.26639 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f8839bccb80 10 client.26639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.919+0000 7f8839bccb80 20 client.26639 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.920+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe465b3e50 0x55fe465b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.920+0000 7f8839bccb80 4 client.26639 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:14.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.920+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe464e8eb0 con 0x55fe465b3e50 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.920+0000 7f881f7fe700 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe465b3e50 0x55fe465b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.921+0000 7f881f7fe700 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe465b3e50 0x55fe465b6260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9f319fe8696812d server_cookie=16055b851b61dbaf in_seq=0 out_seq=0 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.921+0000 7f87feffd700 10 client.26639 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.921+0000 7f87feffd700 1 -- 172.21.15.26:0/3408246001 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe464e8eb0 con 0x55fe465b3e50 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.921+0000 7f87feffd700 10 client.26639 handle_command_reply: tid=0 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.921+0000 7f8839bccb80 1 client.26639 shutdown 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 msgr2=0x55fe465a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8810006b30 tx=0x7f8810006ee0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 shutdown_connections 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe465b3e50 0x55fe465b6260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659e4d0 0x55fe465a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 --2- 172.21.15.26:0/3408246001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659ee20 0x55fe465a86a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 >> 172.21.15.26:0/3408246001 conn(0x55fe46593da0 msgr2=0x55fe465988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f87feffd700 0 client.26639 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 shutdown_connections 2023-07-13T22:21:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.922+0000 7f8839bccb80 1 -- 172.21.15.26:0/3408246001 wait complete. 2023-07-13T22:21:14.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.923+0000 7f8839bccb80 20 client.26639 trim_cache size 0 max 16384 2023-07-13T22:21:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 Processor -- start 2023-07-13T22:21:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 -- start start 2023-07-13T22:21:14.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659fc70 0x55fe465a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465a5df0 con 0x55fe4659ee20 2023-07-13T22:21:14.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465a92b0 con 0x55fe4659fc70 2023-07-13T22:21:14.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.937+0000 7f8839bccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465b3b20 con 0x55fe4659e4d0 2023-07-13T22:21:14.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41896/0 (socket says 172.21.15.26:41896) 2023-07-13T22:21:14.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 -- 172.21.15.26:0/797701501 learned_addr learned my addr 172.21.15.26:0/797701501 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881effd700 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881ffff700 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659fc70 0x55fe465a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 -- 172.21.15.26:0/797701501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659fc70 msgr2=0x55fe465a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659fc70 0x55fe465a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 -- 172.21.15.26:0/797701501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 msgr2=0x55fe465af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.938+0000 7f881f7fe700 1 -- 172.21.15.26:0/797701501 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe46344250 con 0x55fe4659e4d0 2023-07-13T22:21:14.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.939+0000 7f881f7fe700 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f881809b930 tx=0x7f88180ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896570119fadb209 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.939+0000 7f88067fc700 1 -- 172.21.15.26:0/797701501 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881809f9d0 con 0x55fe4659e4d0 2023-07-13T22:21:14.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.939+0000 7f88067fc700 1 -- 172.21.15.26:0/797701501 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88180a3d60 con 0x55fe4659e4d0 2023-07-13T22:21:14.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.939+0000 7f88067fc700 1 -- 172.21.15.26:0/797701501 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881801bd60 con 0x55fe4659e4d0 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 -- 172.21.15.26:0/797701501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 msgr2=0x55fe465a00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f881809b930 tx=0x7f88180ce040 comp rx=0 tx=0).stop 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 -- 172.21.15.26:0/797701501 shutdown_connections 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659fc70 0x55fe465a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659e4d0 0x55fe465a00c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 --2- 172.21.15.26:0/797701501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 -- 172.21.15.26:0/797701501 >> 172.21.15.26:0/797701501 conn(0x55fe46593da0 msgr2=0x55fe465bab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 -- 172.21.15.26:0/797701501 shutdown_connections 2023-07-13T22:21:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.940+0000 7f8839bccb80 1 -- 172.21.15.26:0/797701501 wait complete. 2023-07-13T22:21:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.941+0000 7f8839bccb80 1 -- start start 2023-07-13T22:21:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.941+0000 7f8839bccb80 1 Processor -- start 2023-07-13T22:21:14.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465aefd0 con 0x55fe4659ee20 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465af140 con 0x55fe4659e4d0 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe465af300 con 0x55fe4659fc70 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f8839bccb80 10 client.0 _ll_register_callbacks cb 0x7f8810009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f881f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34112/0 (socket says 172.21.15.26:34112) 2023-07-13T22:21:14.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f881f7fe700 1 -- 172.21.15.26:0/1933894945 learned_addr learned my addr 172.21.15.26:0/1933894945 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.942+0000 7f881ffff700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 -- 172.21.15.26:0/1933894945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 msgr2=0x55fe465aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 -- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 msgr2=0x55fe465aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 -- 172.21.15.26:0/1933894945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201a05c0 con 0x55fe4659e4d0 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f881f7fe700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8818098410 tx=0x7f8818007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8344fe81402fe67b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:14.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f87feffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88180cc240 con 0x55fe4659e4d0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.943+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8818099d40 con 0x55fe4659e4d0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8810013930 con 0x55fe4659e4d0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f881801bd60 con 0x55fe4659e4d0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 10 client.26645 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55fe4631c2b0 con 0x55fe4659e4d0 2023-07-13T22:21:14.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8810013930 con 0x55fe4659e4d0 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f881809bc00 con 0x55fe4659e4d0 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 10 client.26645 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 10 client.26645 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 20 client.26645 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe466087e0 0x55fe4660abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 4 client.26645 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:14.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.944+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fe464ab050 con 0x55fe466087e0 2023-07-13T22:21:14.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.945+0000 7f881effd700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe466087e0 0x55fe4660abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.945+0000 7f881effd700 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe466087e0 0x55fe4660abf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=843ff252f37f9d3 server_cookie=511a643072e7214f in_seq=0 out_seq=0 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.945+0000 7f87feffd700 10 client.26645 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f87feffd700 1 -- 172.21.15.26:0/1933894945 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fe464ab050 con 0x55fe466087e0 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f87feffd700 10 client.26645 handle_command_reply: tid=0 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 client.26645 shutdown 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 msgr2=0x55fe465ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8818098410 tx=0x7f8818007500 comp rx=0 tx=0).stop 2023-07-13T22:21:14.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 shutdown_connections 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fe4659fc70 0x55fe465aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fe4659e4d0 0x55fe465ab530 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55fe466087e0 0x55fe4660abf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 --2- 172.21.15.26:0/1933894945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fe4659ee20 0x55fe465aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f87feffd700 0 client.26645 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.946+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 >> 172.21.15.26:0/1933894945 conn(0x55fe46593da0 msgr2=0x55fe465a6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.947+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 shutdown_connections 2023-07-13T22:21:14.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.947+0000 7f8839bccb80 1 -- 172.21.15.26:0/1933894945 wait complete. 2023-07-13T22:21:14.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.947+0000 7f8839bccb80 20 client.26645 trim_cache size 0 max 16384 2023-07-13T22:21:14.937 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 msgr2=0x7f88080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 0x7f88080a36d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f881400ab80 tx=0x7f881400a2d0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 msgr2=0x7f882010c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f881800b830 tx=0x7f8818004ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 shutdown_connections 2023-07-13T22:21:14.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88201080d0 0x7f8820108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88200716f0 0x7f882010c2b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f88080a1220 0x7f88080a36d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 --2- 172.21.15.26:0/4017370002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8820107720 0x7f882010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 >> 172.21.15.26:0/4017370002 conn(0x7f882006cd80 msgr2=0x7f882010cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 shutdown_connections 2023-07-13T22:21:14.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:14.948+0000 7f88261ff700 1 -- 172.21.15.26:0/4017370002 wait complete. 2023-07-13T22:21:14.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:21:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.411+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3460174267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648071830 msgr2=0x7fc64810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.411+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3460174267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648071830 0x7fc64810b7d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc638009990 tx=0x7fc638009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.412+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3460174267 shutdown_connections 2023-07-13T22:21:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.412+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3460174267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc64810bd60 0x7fc64810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.412+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3460174267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc6480712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.412+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3460174267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648071830 0x7fc64810b7d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.412+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3460174267 >> 172.21.15.26:0/3460174267 conn(0x7fc64806ca90 msgr2=0x7fc64806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.413+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3460174267 shutdown_connections 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.413+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3460174267 wait complete. 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.414+0000 7fc64e5ff700 1 Processor -- start 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 -- start start 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc648113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc648071830 0x7fc648113fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 0x7fc6481a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648117110 con 0x7fc648071830 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648117280 con 0x7fc64810bd60 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.415+0000 7fc64e5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6481a3ee0 con 0x7fc648070ee0 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.416+0000 7fc647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc648113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.416+0000 7fc647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc648113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41908/0 (socket says 172.21.15.26:41908) 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.416+0000 7fc647fff700 1 -- 172.21.15.26:0/3341139649 learned_addr learned my addr 172.21.15.26:0/3341139649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:15.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.416+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 0x7fc6481a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.417+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3341139649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 msgr2=0x7fc648113a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.417+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc648113a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.417+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc648071830 msgr2=0x7fc648113fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:15.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.417+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc648071830 0x7fc648113fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.417+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3341139649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380095f0 con 0x7fc64810bd60 2023-07-13T22:21:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.418+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 0x7fc6481a39a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc64000be20 tx=0x7fc640009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.418+0000 7fc6457fa700 1 -- 172.21.15.26:0/3341139649 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64000cc10 con 0x7fc64810bd60 2023-07-13T22:21:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.418+0000 7fc6457fa700 1 -- 172.21.15.26:0/3341139649 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc64000cd70 con 0x7fc64810bd60 2023-07-13T22:21:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.418+0000 7fc6457fa700 1 -- 172.21.15.26:0/3341139649 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400126c0 con 0x7fc64810bd60 2023-07-13T22:21:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.418+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6481a4160 con 0x7fc64810bd60 2023-07-13T22:21:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.419+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6481a46a0 con 0x7fc64810bd60 2023-07-13T22:21:15.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.427+0000 7fc661ffcb80 1 Processor -- start 2023-07-13T22:21:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 -- start start 2023-07-13T22:21:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da1348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da039cb0 con 0x5577da1344d0 2023-07-13T22:21:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da08ca40 con 0x5577da134e20 2023-07-13T22:21:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.428+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da085ed0 con 0x5577da135c70 2023-07-13T22:21:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.429+0000 7fc647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.429+0000 7fc647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34144/0 (socket says 172.21.15.26:34144) 2023-07-13T22:21:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.429+0000 7fc647fff700 1 -- 172.21.15.26:0/1438557586 learned_addr learned my addr 172.21.15.26:0/1438557586 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:15.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.430+0000 7fc647fff700 1 -- 172.21.15.26:0/1438557586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 msgr2=0x5577da1360e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.430+0000 7fc647fff700 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1360e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.430+0000 7fc647fff700 1 -- 172.21.15.26:0/1438557586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 msgr2=0x5577da1348e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.430+0000 7fc647fff700 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da1348e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.430+0000 7fc647fff700 1 -- 172.21.15.26:0/1438557586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d9eb3de0 con 0x5577da134e20 2023-07-13T22:21:15.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.433+0000 7fc647fff700 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009b80 tx=0x7fc63c009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3aaf1a1c91338d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.435+0000 7fc6457fa700 1 -- 172.21.15.26:0/3341139649 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc640012820 con 0x7fc64810bd60 2023-07-13T22:21:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.436+0000 7fc62e7fc700 1 -- 172.21.15.26:0/1438557586 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00e9f0 con 0x5577da134e20 2023-07-13T22:21:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.436+0000 7fc62e7fc700 1 -- 172.21.15.26:0/1438557586 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63c013470 con 0x5577da134e20 2023-07-13T22:21:15.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.436+0000 7fc62e7fc700 1 -- 172.21.15.26:0/1438557586 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c012630 con 0x5577da134e20 2023-07-13T22:21:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.438+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1438557586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 msgr2=0x5577da1356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.438+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009b80 tx=0x7fc63c009e90 comp rx=0 tx=0).stop 2023-07-13T22:21:15.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.438+0000 7fc6457fa700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 0x7fc6300a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.438+0000 7fc6457fa700 1 -- 172.21.15.26:0/3341139649 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc640006e40 con 0x7fc64810bd60 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.439+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1438557586 shutdown_connections 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.439+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da1348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.439+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.439+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1438557586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da1356e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.439+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1438557586 >> 172.21.15.26:0/1438557586 conn(0x5577da129da0 msgr2=0x5577da12f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.441+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1438557586 shutdown_connections 2023-07-13T22:21:15.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.441+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1438557586 wait complete. 2023-07-13T22:21:15.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.442+0000 7fc6477fe700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 0x7fc6300a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.442+0000 7fc661ffcb80 1 -- start start 2023-07-13T22:21:15.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 Processor -- start 2023-07-13T22:21:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da13e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da025ac0 con 0x5577da1344d0 2023-07-13T22:21:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da0257c0 con 0x5577da134e20 2023-07-13T22:21:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.443+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d9f09920 con 0x5577da135c70 2023-07-13T22:21:15.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.444+0000 7fc661ffcb80 10 client.0 _ll_register_callbacks cb 0x5577da023760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.444+0000 7fc64cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.444+0000 7fc64cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41910/0 (socket says 172.21.15.26:41910) 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.444+0000 7fc64cb9c700 1 -- 172.21.15.26:0/1047140119 learned_addr learned my addr 172.21.15.26:0/1047140119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc647fff700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc64cb9c700 1 -- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 msgr2=0x5577da13e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc64cb9c700 1 -- 172.21.15.26:0/1047140119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 msgr2=0x5577da13e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da13e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.445+0000 7fc64cb9c700 1 -- 172.21.15.26:0/1047140119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c0097b0 con 0x5577da135c70 2023-07-13T22:21:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.446+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc640098fc0 tx=0x7fc64009d0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3617878dad2b93d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.446+0000 7fc62bff7700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:15.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.446+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640098770 con 0x5577da135c70 2023-07-13T22:21:15.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.446+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc6400a1040 con 0x5577da135c70 2023-07-13T22:21:15.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.446+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577d9fdf4f0 con 0x5577da135c70 2023-07-13T22:21:15.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.448+0000 7fc6477fe700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 0x7fc6300a3800 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc638005b40 tx=0x7fc638005ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6400a1200 con 0x5577da135c70 2023-07-13T22:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc661ffcb80 10 client.26638 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5577d9e9cc60 con 0x5577da135c70 2023-07-13T22:21:15.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577d9fdf4f0 con 0x5577da135c70 2023-07-13T22:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc640012ad0 con 0x5577da135c70 2023-07-13T22:21:15.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc661ffcb80 10 client.26638 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc661ffcb80 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.457+0000 7fc661ffcb80 20 client.26638 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.458+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da149fc0 0x5577da14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.458+0000 7fc661ffcb80 4 client.26638 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.458+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577da07eeb0 con 0x5577da149fc0 2023-07-13T22:21:15.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.459+0000 7fc647fff700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da149fc0 0x5577da14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.459+0000 7fc647fff700 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da149fc0 0x5577da14c3d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0aaf19deb102205 server_cookie=cc29c407e3662439 in_seq=0 out_seq=0 2023-07-13T22:21:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.459+0000 7fc62bff7700 10 client.26638 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.460+0000 7fc62bff7700 1 -- 172.21.15.26:0/1047140119 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577da07eeb0 con 0x5577da149fc0 2023-07-13T22:21:15.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.460+0000 7fc62bff7700 10 client.26638 handle_command_reply: tid=0 2023-07-13T22:21:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.460+0000 7fc661ffcb80 1 client.26638 shutdown 2023-07-13T22:21:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 msgr2=0x5577da1456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc640098fc0 tx=0x7fc64009d0b0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 shutdown_connections 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da13e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da149fc0 0x5577da14c3d0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da1456e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/1047140119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da13e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 >> 172.21.15.26:0/1047140119 conn(0x5577da129da0 msgr2=0x5577da12e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc62bff7700 0 client.26638 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:15.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 shutdown_connections 2023-07-13T22:21:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.461+0000 7fc661ffcb80 1 -- 172.21.15.26:0/1047140119 wait complete. 2023-07-13T22:21:15.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.462+0000 7fc661ffcb80 20 client.26638 trim_cache size 0 max 16384 2023-07-13T22:21:15.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 Processor -- start 2023-07-13T22:21:15.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 -- start start 2023-07-13T22:21:15.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 0x5577da1348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da134e20 0x5577da145600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da12f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.490+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da1299e0 con 0x5577da134e20 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.491+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da12f710 con 0x5577da1344d0 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.491+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da13bd90 con 0x5577da135c70 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.491+0000 7fc647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 0x5577da1348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.491+0000 7fc647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 0x5577da1348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:34166/0 (socket says 172.21.15.26:34166) 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.491+0000 7fc647fff700 1 -- 172.21.15.26:0/3053433835 learned_addr learned my addr 172.21.15.26:0/3053433835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.492+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da12f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc6477fe700 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da134e20 0x5577da145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3053433835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 msgr2=0x5577da1348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 0x5577da1348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3053433835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da134e20 msgr2=0x5577da145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da134e20 0x5577da145600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.493+0000 7fc64cb9c700 1 -- 172.21.15.26:0/3053433835 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d9eda250 con 0x5577da135c70 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.494+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da12f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc640099010 tx=0x7fc64000cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6903b8eb42b568e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.494+0000 7fc62effd700 1 -- 172.21.15.26:0/3053433835 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400a1040 con 0x5577da135c70 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.494+0000 7fc62effd700 1 -- 172.21.15.26:0/3053433835 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6400a11a0 con 0x5577da135c70 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.494+0000 7fc62effd700 1 -- 172.21.15.26:0/3053433835 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640012d30 con 0x5577da135c70 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.495+0000 7fc661ffcb80 1 -- 172.21.15.26:0/3053433835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 msgr2=0x5577da12f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.495+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da12f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc640099010 tx=0x7fc64000cf00 comp rx=0 tx=0).stop 2023-07-13T22:21:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.495+0000 7fc661ffcb80 1 -- 172.21.15.26:0/3053433835 shutdown_connections 2023-07-13T22:21:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.495+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da12f1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.496+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da1344d0 0x5577da1348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.496+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/3053433835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da134e20 0x5577da145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.496+0000 7fc661ffcb80 1 -- 172.21.15.26:0/3053433835 >> 172.21.15.26:0/3053433835 conn(0x5577da129da0 msgr2=0x5577da12e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.509+0000 7fc661ffcb80 1 -- 172.21.15.26:0/3053433835 shutdown_connections 2023-07-13T22:21:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.509+0000 7fc661ffcb80 1 -- 172.21.15.26:0/3053433835 wait complete. 2023-07-13T22:21:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.510+0000 7fc661ffcb80 1 -- start start 2023-07-13T22:21:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.510+0000 7fc661ffcb80 1 Processor -- start 2023-07-13T22:21:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da141530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da141a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da144fd0 con 0x5577da1344d0 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da145140 con 0x5577da134e20 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577da145300 con 0x5577da135c70 2023-07-13T22:21:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.511+0000 7fc661ffcb80 10 client.0 _ll_register_callbacks cb 0x7fc63800b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:15.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.517+0000 7fc64cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.517+0000 7fc64cb9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41924/0 (socket says 172.21.15.26:41924) 2023-07-13T22:21:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.517+0000 7fc64cb9c700 1 -- 172.21.15.26:0/2925540799 learned_addr learned my addr 172.21.15.26:0/2925540799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.518+0000 7fc6477fe700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 -- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 msgr2=0x5577da141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 -- 172.21.15.26:0/2925540799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 msgr2=0x5577da141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 -- 172.21.15.26:0/2925540799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577d9e9cc60 con 0x5577da135c70 2023-07-13T22:21:15.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.519+0000 7fc64cb9c700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc64000be50 tx=0x7fc640099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcf228a448cf602d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:15.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc62bff7700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640098730 con 0x5577da135c70 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc640095080 con 0x5577da135c70 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc63c013470 con 0x5577da135c70 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6400c92d0 con 0x5577da135c70 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc661ffcb80 10 client.26644 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.522+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5577d9eb22b0 con 0x5577da135c70 2023-07-13T22:21:15.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc63c013470 con 0x5577da135c70 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc640099e60 con 0x5577da135c70 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 10 client.26644 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 10 client.26644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 20 client.26644 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da19e8a0 0x5577da1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 4 client.26644 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.523+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577da041050 con 0x5577da19e8a0 2023-07-13T22:21:15.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.525+0000 7fc647fff700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da19e8a0 0x5577da1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:15.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.526+0000 7fc647fff700 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da19e8a0 0x5577da1a0c80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e474ca89cd6b281 server_cookie=af9d566c46b74231 in_seq=0 out_seq=0 2023-07-13T22:21:15.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.526+0000 7fc62bff7700 10 client.26644 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:15.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.526+0000 7fc62bff7700 1 -- 172.21.15.26:0/2925540799 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577da041050 con 0x5577da19e8a0 2023-07-13T22:21:15.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.526+0000 7fc62bff7700 10 client.26644 handle_command_reply: tid=0 2023-07-13T22:21:15.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.526+0000 7fc661ffcb80 1 client.26644 shutdown 2023-07-13T22:21:15.535 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 msgr2=0x5577da144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc64000be50 tx=0x7fc640099690 comp rx=0 tx=0).stop 2023-07-13T22:21:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 shutdown_connections 2023-07-13T22:21:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577da1344d0 0x5577da141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5577da19e8a0 0x5577da1a0c80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577da135c70 0x5577da144a90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 --2- 172.21.15.26:0/2925540799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577da134e20 0x5577da141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.547+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 >> 172.21.15.26:0/2925540799 conn(0x5577da129da0 msgr2=0x5577da150cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.548+0000 7fc62bff7700 0 client.26644 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.548+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 shutdown_connections 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.548+0000 7fc661ffcb80 1 -- 172.21.15.26:0/2925540799 wait complete. 2023-07-13T22:21:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.549+0000 7fc661ffcb80 20 client.26644 trim_cache size 0 max 16384 2023-07-13T22:21:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.550+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 msgr2=0x7fc6300a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.550+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 0x7fc6300a3800 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc638005b40 tx=0x7fc638005ad0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.550+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 msgr2=0x7fc6481a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.550+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 0x7fc6481a39a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc64000be20 tx=0x7fc640009ed0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 shutdown_connections 2023-07-13T22:21:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc64810bd60 0x7fc6481a39a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc648070ee0 0x7fc648113a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc6300a1350 0x7fc6300a3800 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 --2- 172.21.15.26:0/3341139649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc648071830 0x7fc648113fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 >> 172.21.15.26:0/3341139649 conn(0x7fc64806ca90 msgr2=0x7fc64806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 shutdown_connections 2023-07-13T22:21:15.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:15.554+0000 7fc64e5ff700 1 -- 172.21.15.26:0/3341139649 wait complete. 2023-07-13T22:21:15.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:15 smithi117 ceph-mon[129074]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 97 op/s 2023-07-13T22:21:15.710 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:15 smithi026 ceph-mon[95647]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 97 op/s 2023-07-13T22:21:15.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:15 smithi121 ceph-mon[129187]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 97 op/s 2023-07-13T22:21:16.639 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:21:16.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:21:17.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 -- 172.21.15.26:0/2777623001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590070c70 msgr2=0x7fe5900710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 --2- 172.21.15.26:0/2777623001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590070c70 0x7fe5900710e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe584009990 tx=0x7fe584009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 -- 172.21.15.26:0/2777623001 shutdown_connections 2023-07-13T22:21:17.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 --2- 172.21.15.26:0/2777623001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590107a20 0x7fe590107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 --2- 172.21.15.26:0/2777623001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590071620 0x7fe590110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 --2- 172.21.15.26:0/2777623001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590070c70 0x7fe5900710e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.344+0000 7fe5953ff700 1 -- 172.21.15.26:0/2777623001 >> 172.21.15.26:0/2777623001 conn(0x7fe59006ca90 msgr2=0x7fe59006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.345+0000 7fe5953ff700 1 -- 172.21.15.26:0/2777623001 shutdown_connections 2023-07-13T22:21:17.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.345+0000 7fe5953ff700 1 -- 172.21.15.26:0/2777623001 wait complete. 2023-07-13T22:21:17.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.346+0000 7fe5953ff700 1 Processor -- start 2023-07-13T22:21:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 -- start start 2023-07-13T22:21:17.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590071620 0x7fe5901a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590107a20 0x7fe59019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901b4a50 con 0x7fe590070c70 2023-07-13T22:21:17.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901b4b90 con 0x7fe590107a20 2023-07-13T22:21:17.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.347+0000 7fe5953ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901b4cd0 con 0x7fe590071620 2023-07-13T22:21:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.349+0000 7fe58f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590107a20 0x7fe59019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.349+0000 7fe58effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.349+0000 7fe58effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42378/0 (socket says 172.21.15.26:42378) 2023-07-13T22:21:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.349+0000 7fe58effd700 1 -- 172.21.15.26:0/1769011966 learned_addr learned my addr 172.21.15.26:0/1769011966 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:17.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.349+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590071620 0x7fe5901a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.355+0000 7fe58effd700 1 -- 172.21.15.26:0/1769011966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590071620 msgr2=0x7fe5901a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.355+0000 7fe58effd700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590071620 0x7fe5901a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.355+0000 7fe58effd700 1 -- 172.21.15.26:0/1769011966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590107a20 msgr2=0x7fe59019c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.355+0000 7fe58effd700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590107a20 0x7fe59019c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.355+0000 7fe58effd700 1 -- 172.21.15.26:0/1769011966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009520 con 0x7fe590070c70 2023-07-13T22:21:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.357+0000 7fe58effd700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb30 tx=0x7fe58000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.362+0000 7fe577fff700 1 -- 172.21.15.26:0/1769011966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580009880 con 0x7fe590070c70 2023-07-13T22:21:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.362+0000 7fe577fff700 1 -- 172.21.15.26:0/1769011966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5800044e0 con 0x7fe590070c70 2023-07-13T22:21:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.362+0000 7fe577fff700 1 -- 172.21.15.26:0/1769011966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580003e30 con 0x7fe590070c70 2023-07-13T22:21:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.362+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5840095f0 con 0x7fe590070c70 2023-07-13T22:21:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.363+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5901b51f0 con 0x7fe590070c70 2023-07-13T22:21:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.372+0000 7fe577fff700 1 -- 172.21.15.26:0/1769011966 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe580004020 con 0x7fe590070c70 2023-07-13T22:21:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.377+0000 7fe577fff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 0x7fe5780a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.377+0000 7fe577fff700 1 -- 172.21.15.26:0/1769011966 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe580014070 con 0x7fe590070c70 2023-07-13T22:21:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.378+0000 7fe5a8d68b80 1 Processor -- start 2023-07-13T22:21:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 -- start start 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c093960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c094cf0 0x56307c095160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bf98d30 con 0x56307c094cf0 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bfebac0 con 0x56307c093550 2023-07-13T22:21:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.379+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bfe4f50 con 0x56307c093ea0 2023-07-13T22:21:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.384+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.384+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41944/0 (socket says 172.21.15.26:41944) 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.384+0000 7fe58e7fc700 1 -- 172.21.15.26:0/4030355225 learned_addr learned my addr 172.21.15.26:0/4030355225 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 -- 172.21.15.26:0/4030355225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 msgr2=0x56307c093960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c093960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 -- 172.21.15.26:0/4030355225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c094cf0 msgr2=0x56307c095160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c094cf0 0x56307c095160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 -- 172.21.15.26:0/4030355225 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307be12e60 con 0x56307c093ea0 2023-07-13T22:21:17.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.385+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe584015000 tx=0x7fe584005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38a27042fe6779c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.386+0000 7fe5757fa700 1 -- 172.21.15.26:0/4030355225 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400b990 con 0x56307c093ea0 2023-07-13T22:21:17.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.386+0000 7fe5757fa700 1 -- 172.21.15.26:0/4030355225 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58400f460 con 0x56307c093ea0 2023-07-13T22:21:17.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.386+0000 7fe5757fa700 1 -- 172.21.15.26:0/4030355225 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584024640 con 0x56307c093ea0 2023-07-13T22:21:17.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.388+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 0x7fe5780a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.392+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4030355225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 msgr2=0x56307c094760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.392+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 0x7fe5780a37d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe584005ac0 tx=0x7fe584005810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.392+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe584015000 tx=0x7fe584005ee0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.393+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4030355225 shutdown_connections 2023-07-13T22:21:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.393+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c094cf0 0x56307c095160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.393+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c093960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.393+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4030355225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c094760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.393+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4030355225 >> 172.21.15.26:0/4030355225 conn(0x56307c088e20 msgr2=0x56307c08e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.394+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4030355225 shutdown_connections 2023-07-13T22:21:17.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.394+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4030355225 wait complete. 2023-07-13T22:21:17.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.394+0000 7fe5a8d68b80 1 -- start start 2023-07-13T22:21:17.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.396+0000 7fe5a8d68b80 1 Processor -- start 2023-07-13T22:21:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c09d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c094cf0 0x56307c0a4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bf84b40 con 0x56307c093550 2023-07-13T22:21:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307bf84840 con 0x56307c094cf0 2023-07-13T22:21:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307be689a0 con 0x56307c093ea0 2023-07-13T22:21:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.398+0000 7fe5a8d68b80 10 client.0 _ll_register_callbacks cb 0x56307bf827e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.403+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c09d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.403+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c09d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41950/0 (socket says 172.21.15.26:41950) 2023-07-13T22:21:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.403+0000 7fe58e7fc700 1 -- 172.21.15.26:0/1471626146 learned_addr learned my addr 172.21.15.26:0/1471626146 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.404+0000 7fe58effd700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58f7fe700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c094cf0 0x56307c0a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58effd700 1 -- 172.21.15.26:0/1471626146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 msgr2=0x56307c09d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58effd700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c09d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58effd700 1 -- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c094cf0 msgr2=0x56307c0a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58effd700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c094cf0 0x56307c0a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.405+0000 7fe58effd700 1 -- 172.21.15.26:0/1471626146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe584020040 con 0x56307c093550 2023-07-13T22:21:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.406+0000 7fe58effd700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb00 tx=0x7fe580099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f4bcbd7555dc3a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.407+0000 7fe572ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.407+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58009a100 con 0x56307c093550 2023-07-13T22:21:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.407+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe58009e130 con 0x56307c093550 2023-07-13T22:21:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.407+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56307bf3e570 con 0x56307c093550 2023-07-13T22:21:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.408+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5800a2550 con 0x56307c093550 2023-07-13T22:21:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.408+0000 7fe5a8d68b80 10 client.16230 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:17.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.408+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56307bdfbce0 con 0x56307c093550 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.411+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56307bf3e570 con 0x56307c093550 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.412+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe5800995f0 con 0x56307c093550 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 20 client.16230 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0a8f70 0x56307c0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 4 client.16230 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:17.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.413+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56307bfddf30 con 0x56307c0a8f70 2023-07-13T22:21:17.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.414+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0a8f70 0x56307c0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.418+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0a8f70 0x56307c0ab380 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26441e81eb82d5c server_cookie=918f5c43efa9d882 in_seq=0 out_seq=0 2023-07-13T22:21:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.420+0000 7fe572ff5700 10 client.16230 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.420+0000 7fe572ff5700 1 -- 172.21.15.26:0/1471626146 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56307bfddf30 con 0x56307c0a8f70 2023-07-13T22:21:17.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.421+0000 7fe572ff5700 10 client.16230 handle_command_reply: tid=0 2023-07-13T22:21:17.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.422+0000 7fe5a8d68b80 1 client.16230 shutdown 2023-07-13T22:21:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.426+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 msgr2=0x56307c09d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.426+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb00 tx=0x7fe580099a60 comp rx=0 tx=0).stop 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.430+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 shutdown_connections 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.430+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09d190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.431+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0a8f70 0x56307c0ab380 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.431+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c094cf0 0x56307c0a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.431+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/1471626146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c093ea0 0x56307c09d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.431+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 >> 172.21.15.26:0/1471626146 conn(0x56307c088e20 msgr2=0x56307c08d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.433+0000 7fe572ff5700 0 client.16230 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:17.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.433+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 shutdown_connections 2023-07-13T22:21:17.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.434+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/1471626146 wait complete. 2023-07-13T22:21:17.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.436+0000 7fe5a8d68b80 20 client.16230 trim_cache size 0 max 16384 2023-07-13T22:21:17.446 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:17 smithi026 ceph-mon[95647]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 83 op/s 2023-07-13T22:21:17.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.550+0000 7fe5a8d68b80 1 Processor -- start 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.550+0000 7fe5a8d68b80 1 -- start start 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093ea0 0x56307c0a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c09cd60 con 0x56307c093550 2023-07-13T22:21:17.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c09e380 con 0x56307c093ea0 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.551+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c09e1e0 con 0x56307c094cf0 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.552+0000 7fe58effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.552+0000 7fe58effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42414/0 (socket says 172.21.15.26:42414) 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.552+0000 7fe58effd700 1 -- 172.21.15.26:0/4101109135 learned_addr learned my addr 172.21.15.26:0/4101109135 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 -- 172.21.15.26:0/4101109135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 msgr2=0x56307c0aaf80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0aaf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 -- 172.21.15.26:0/4101109135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093ea0 msgr2=0x56307c0a46d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093ea0 0x56307c0a46d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 -- 172.21.15.26:0/4101109135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307be392d0 con 0x56307c093550 2023-07-13T22:21:17.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.553+0000 7fe58effd700 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe580000c00 tx=0x7fe5800986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b319e7fa0c259 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.554+0000 7fe575ffb700 1 -- 172.21.15.26:0/4101109135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58009e130 con 0x56307c093550 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.554+0000 7fe575ffb700 1 -- 172.21.15.26:0/4101109135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5800a2550 con 0x56307c093550 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.554+0000 7fe575ffb700 1 -- 172.21.15.26:0/4101109135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800a2190 con 0x56307c093550 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.555+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4101109135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 msgr2=0x56307c09b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.555+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe580000c00 tx=0x7fe5800986b0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4101109135 shutdown_connections 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093550 0x56307c09b0c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/4101109135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093ea0 0x56307c0a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4101109135 >> 172.21.15.26:0/4101109135 conn(0x56307c088e20 msgr2=0x56307c08d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4101109135 shutdown_connections 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.556+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/4101109135 wait complete. 2023-07-13T22:21:17.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.557+0000 7fe5a8d68b80 1 -- start start 2023-07-13T22:21:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.557+0000 7fe5a8d68b80 1 Processor -- start 2023-07-13T22:21:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c0a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c0a18b0 con 0x56307c093ea0 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c0a1a20 con 0x56307c093550 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307c0fce20 con 0x56307c094cf0 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.558+0000 7fe5a8d68b80 10 client.0 _ll_register_callbacks cb 0x7fe580096510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42422/0 (socket says 172.21.15.26:42422) 2023-07-13T22:21:17.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 -- 172.21.15.26:0/2092914356 learned_addr learned my addr 172.21.15.26:0/2092914356 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 -- 172.21.15.26:0/2092914356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 msgr2=0x56307c0a1370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0a1370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 -- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 msgr2=0x56307c0a4390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c0a4390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.559+0000 7fe58e7fc700 1 -- 172.21.15.26:0/2092914356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56307bdfbce0 con 0x56307c093ea0 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.560+0000 7fe58e7fc700 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe58400fbc0 tx=0x7fe584024860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241685e17505c000 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.560+0000 7fe572ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.561+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584036040 con 0x56307c093ea0 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.561+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe584043320 con 0x56307c093ea0 2023-07-13T22:21:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.561+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe58400f460 con 0x56307c093ea0 2023-07-13T22:21:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.562+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe584024630 con 0x56307c093ea0 2023-07-13T22:21:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.562+0000 7fe5a8d68b80 10 client.16236 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.562+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56307be11330 con 0x56307c093ea0 2023-07-13T22:21:17.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.563+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe58400f460 con 0x56307c093ea0 2023-07-13T22:21:17.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.563+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe58400bb10 con 0x56307c093ea0 2023-07-13T22:21:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:17.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 20 client.16236 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0fe350 0x56307c100760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 4 client.16236 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:21:17.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.566+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56307bfa00d0 con 0x56307c0fe350 2023-07-13T22:21:17.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.574+0000 7fe58effd700 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0fe350 0x56307c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:17.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.575+0000 7fe58effd700 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0fe350 0x56307c100760 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2289df1589032371 server_cookie=e99e274ff5d90ee5 in_seq=0 out_seq=0 2023-07-13T22:21:17.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.575+0000 7fe572ff5700 10 client.16236 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:17.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.576+0000 7fe572ff5700 1 -- 172.21.15.26:0/2092914356 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56307bfa00d0 con 0x56307c0fe350 2023-07-13T22:21:17.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.576+0000 7fe572ff5700 10 client.16236 handle_command_reply: tid=0 2023-07-13T22:21:17.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.579+0000 7fe5a8d68b80 1 client.16236 shutdown 2023-07-13T22:21:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.580+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 msgr2=0x56307c0a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.580+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe58400fbc0 tx=0x7fe584024860 comp rx=0 tx=0).stop 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 shutdown_connections 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56307c0fe350 0x56307c100760 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56307c094cf0 0x56307c0a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56307c093550 0x56307c0a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 --2- 172.21.15.26:0/2092914356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56307c093ea0 0x56307c0a0e30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 >> 172.21.15.26:0/2092914356 conn(0x56307c088e20 msgr2=0x56307c0aaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe572ff5700 0 client.16236 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 shutdown_connections 2023-07-13T22:21:17.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.581+0000 7fe5a8d68b80 1 -- 172.21.15.26:0/2092914356 wait complete. 2023-07-13T22:21:17.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.582+0000 7fe5a8d68b80 20 client.16236 trim_cache size 0 max 16384 2023-07-13T22:21:17.578 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:17.578 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:21:17.579 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "89714725-5616-4a96-be7b-8ccbf43ff4d0", 2023-07-13T22:21:17.579 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:21:17.579 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 msgr2=0x7fe5780a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 0x7fe5780a37d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe584005ac0 tx=0x7fe584005810 comp rx=0 tx=0).stop 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 msgr2=0x7fe59019be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe58000eb30 tx=0x7fe58000ee40 comp rx=0 tx=0).stop 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 shutdown_connections 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe5780a1320 0x7fe5780a37d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe590070c70 0x7fe59019be50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe590107a20 0x7fe59019c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 --2- 172.21.15.26:0/1769011966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe590071620 0x7fe5901a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.594+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 >> 172.21.15.26:0/1769011966 conn(0x7fe59006ca90 msgr2=0x7fe59010ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.596+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 shutdown_connections 2023-07-13T22:21:17.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:17.596+0000 7fe5953ff700 1 -- 172.21.15.26:0/1769011966 wait complete. 2023-07-13T22:21:17.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:21:17.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:17 smithi121 ceph-mon[129187]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 83 op/s 2023-07-13T22:21:17.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:17 smithi117 ceph-mon[129074]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 83 op/s 2023-07-13T22:21:18.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/4220485604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 msgr2=0x7f1b080712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/4220485604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b080712f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0098f0 tx=0x7f1afc009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:18.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/4220485604 shutdown_connections 2023-07-13T22:21:18.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/4220485604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b0810bd60 0x7f1b0810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/4220485604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b080712f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/4220485604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b08071830 0x7f1b0810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.131+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/4220485604 >> 172.21.15.26:0/4220485604 conn(0x7f1b0806ca90 msgr2=0x7f1b0806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.132+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/4220485604 shutdown_connections 2023-07-13T22:21:18.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.132+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/4220485604 wait complete. 2023-07-13T22:21:18.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.133+0000 7f1b0c9ff700 1 Processor -- start 2023-07-13T22:21:18.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 -- start start 2023-07-13T22:21:18.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b081a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b0810bd60 0x7f1b081abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b081ac2b0 con 0x7f1b0810bd60 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0810ec40 con 0x7f1b08070ee0 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.134+0000 7f1b0c9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0810edd0 con 0x7f1b08071830 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.135+0000 7f1b05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.135+0000 7f1b05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42000/0 (socket says 172.21.15.26:42000) 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.135+0000 7f1b05d9b700 1 -- 172.21.15.26:0/731609382 learned_addr learned my addr 172.21.15.26:0/731609382 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.135+0000 7f1b0659c700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b081a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b0810bd60 0x7f1b081abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b05d9b700 1 -- 172.21.15.26:0/731609382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 msgr2=0x7f1b081a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b081a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b05d9b700 1 -- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b0810bd60 msgr2=0x7f1b081abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b0810bd60 0x7f1b081abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.136+0000 7f1b05d9b700 1 -- 172.21.15.26:0/731609382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1afc009520 con 0x7f1b08071830 2023-07-13T22:21:18.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.137+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b500 tx=0x7f1af800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.137+0000 7f1af77fe700 1 -- 172.21.15.26:0/731609382 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8011830 con 0x7f1b08071830 2023-07-13T22:21:18.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.137+0000 7f1af77fe700 1 -- 172.21.15.26:0/731609382 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af8011e50 con 0x7f1b08071830 2023-07-13T22:21:18.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.137+0000 7f1af77fe700 1 -- 172.21.15.26:0/731609382 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af800f2e0 con 0x7f1b08071830 2023-07-13T22:21:18.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.138+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0810f0b0 con 0x7f1b08071830 2023-07-13T22:21:18.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.139+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0810f5c0 con 0x7f1b08071830 2023-07-13T22:21:18.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.145+0000 7f1b2031eb80 1 Processor -- start 2023-07-13T22:21:18.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.146+0000 7f1b2031eb80 1 -- start start 2023-07-13T22:21:18.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399e0c70 0x55ec399e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec398e4cb0 con 0x55ec399df4d0 2023-07-13T22:21:18.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec39937a40 con 0x55ec399e0c70 2023-07-13T22:21:18.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec39930ed0 con 0x55ec399dfe20 2023-07-13T22:21:18.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b0659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b0659c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42006/0 (socket says 172.21.15.26:42006) 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.147+0000 7f1b0659c700 1 -- 172.21.15.26:0/1566794279 learned_addr learned my addr 172.21.15.26:0/1566794279 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 -- 172.21.15.26:0/1566794279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399e0c70 msgr2=0x55ec399e10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399e0c70 0x55ec399e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 -- 172.21.15.26:0/1566794279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 msgr2=0x55ec399df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 -- 172.21.15.26:0/1566794279 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec3975ede0 con 0x55ec399dfe20 2023-07-13T22:21:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.148+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00fbd0 tx=0x7f1afc00bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7220fd8a5f48e6d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.149+0000 7f1af4ff9700 1 -- 172.21.15.26:0/1566794279 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc01c050 con 0x55ec399dfe20 2023-07-13T22:21:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.149+0000 7f1af4ff9700 1 -- 172.21.15.26:0/1566794279 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1afc01d040 con 0x55ec399dfe20 2023-07-13T22:21:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.149+0000 7f1af4ff9700 1 -- 172.21.15.26:0/1566794279 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00ba30 con 0x55ec399dfe20 2023-07-13T22:21:18.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.153+0000 7f1af77fe700 1 -- 172.21.15.26:0/731609382 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1af8010580 con 0x7f1b08071830 2023-07-13T22:21:18.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.155+0000 7f1af77fe700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 0x7f1af00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.156+0000 7f1af77fe700 1 -- 172.21.15.26:0/731609382 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1af80cc250 con 0x7f1b08071830 2023-07-13T22:21:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.158+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1566794279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 msgr2=0x55ec399e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.158+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00fbd0 tx=0x7f1afc00bfa0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.158+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 0x7f1af00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1566794279 shutdown_connections 2023-07-13T22:21:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399e0c70 0x55ec399e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1566794279 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399e06e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1566794279 >> 172.21.15.26:0/1566794279 conn(0x55ec399d4da0 msgr2=0x55ec399da1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.159+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1566794279 shutdown_connections 2023-07-13T22:21:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.160+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1566794279 wait complete. 2023-07-13T22:21:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.160+0000 7f1b2031eb80 1 -- start start 2023-07-13T22:21:18.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.161+0000 7f1b2031eb80 1 Processor -- start 2023-07-13T22:21:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.161+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399e91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399dfe20 0x55ec399e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec398d0ac0 con 0x55ec399df4d0 2023-07-13T22:21:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec398d07c0 con 0x55ec399dfe20 2023-07-13T22:21:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec397b4920 con 0x55ec399e0c70 2023-07-13T22:21:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b2031eb80 10 client.0 _ll_register_callbacks cb 0x55ec398ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.162+0000 7f1b06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42008/0 (socket says 172.21.15.26:42008) 2023-07-13T22:21:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.163+0000 7f1b06d9d700 1 -- 172.21.15.26:0/1472038280 learned_addr learned my addr 172.21.15.26:0/1472038280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:18.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399e91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 -- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399dfe20 msgr2=0x55ec399e96e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399dfe20 0x55ec399e96e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 -- 172.21.15.26:0/1472038280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 msgr2=0x55ec399e91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399e91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 -- 172.21.15.26:0/1472038280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b08117640 con 0x55ec399e0c70 2023-07-13T22:21:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.164+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 0x7f1af00a3800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009b30 tx=0x7f1b00009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.165+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b000166d0 tx=0x7f1b00004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90af994bae68122 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.165+0000 7f1aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.165+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000147e0 con 0x55ec399e0c70 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.165+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1b00014940 con 0x55ec399e0c70 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.165+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec3988a4f0 con 0x55ec399e0c70 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.166+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b00009210 con 0x55ec399e0c70 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.166+0000 7f1b2031eb80 10 client.26668 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:18.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.166+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ec39747c60 con 0x55ec399e0c70 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.166+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec3988a4f0 con 0x55ec399e0c70 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.166+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b00029070 con 0x55ec399e0c70 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 10 client.26668 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 20 client.26668 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec399f5020 0x55ec399f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 4 client.26668 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:18.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec39929eb0 con 0x55ec399f5020 2023-07-13T22:21:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.167+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec399f5020 0x55ec399f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.169+0000 7f1b0659c700 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec399f5020 0x55ec399f7430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cb678764c95a52 server_cookie=a821228a8e43952f in_seq=0 out_seq=0 2023-07-13T22:21:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.169+0000 7f1aedffb700 10 client.26668 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.170+0000 7f1aedffb700 1 -- 172.21.15.26:0/1472038280 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec39929eb0 con 0x55ec399f5020 2023-07-13T22:21:18.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.170+0000 7f1aedffb700 10 client.26668 handle_command_reply: tid=0 2023-07-13T22:21:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.170+0000 7f1b2031eb80 1 client.26668 shutdown 2023-07-13T22:21:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 msgr2=0x55ec399f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1b000166d0 tx=0x7f1b00004b40 comp rx=0 tx=0).stop 2023-07-13T22:21:18.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 shutdown_connections 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399df4d0 0x55ec399e91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec399f5020 0x55ec399f7430 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f0720 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/1472038280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399dfe20 0x55ec399e96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 >> 172.21.15.26:0/1472038280 conn(0x55ec399d4da0 msgr2=0x55ec399d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.171+0000 7f1aedffb700 0 client.26668 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.172+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 shutdown_connections 2023-07-13T22:21:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.172+0000 7f1b2031eb80 1 -- 172.21.15.26:0/1472038280 wait complete. 2023-07-13T22:21:18.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.173+0000 7f1b2031eb80 20 client.26668 trim_cache size 0 max 16384 2023-07-13T22:21:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.215+0000 7f1b2031eb80 1 Processor -- start 2023-07-13T22:21:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.215+0000 7f1b2031eb80 1 -- start start 2023-07-13T22:21:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399dfe20 0x55ec399f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399e8d70 con 0x55ec399dfe20 2023-07-13T22:21:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399e01f0 con 0x55ec399df4d0 2023-07-13T22:21:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.216+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399dbd00 con 0x55ec399e0c70 2023-07-13T22:21:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.217+0000 7f1b06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.217+0000 7f1b06d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42012/0 (socket says 172.21.15.26:42012) 2023-07-13T22:21:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.217+0000 7f1b06d9d700 1 -- 172.21.15.26:0/139148203 learned_addr learned my addr 172.21.15.26:0/139148203 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.217+0000 7f1b0659c700 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399dfe20 0x55ec399f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b06d9d700 1 -- 172.21.15.26:0/139148203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 msgr2=0x55ec399e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b06d9d700 1 -- 172.21.15.26:0/139148203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399dfe20 msgr2=0x55ec399f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399dfe20 0x55ec399f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.218+0000 7f1b06d9d700 1 -- 172.21.15.26:0/139148203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec39785250 con 0x55ec399e0c70 2023-07-13T22:21:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.219+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b00004b10 tx=0x7f1b00007940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8bde1a3115d7366 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.223+0000 7f1af57fa700 1 -- 172.21.15.26:0/139148203 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0002a3a0 con 0x55ec399e0c70 2023-07-13T22:21:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.223+0000 7f1af57fa700 1 -- 172.21.15.26:0/139148203 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b0001a070 con 0x55ec399e0c70 2023-07-13T22:21:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.223+0000 7f1af57fa700 1 -- 172.21.15.26:0/139148203 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00015d10 con 0x55ec399e0c70 2023-07-13T22:21:18.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.224+0000 7f1b2031eb80 1 -- 172.21.15.26:0/139148203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 msgr2=0x55ec399f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.224+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1b00004b10 tx=0x7f1b00007940 comp rx=0 tx=0).stop 2023-07-13T22:21:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.245+0000 7f1b2031eb80 1 -- 172.21.15.26:0/139148203 shutdown_connections 2023-07-13T22:21:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.245+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399e0c70 0x55ec399f7030 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.245+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.245+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/139148203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399dfe20 0x55ec399f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.245+0000 7f1b2031eb80 1 -- 172.21.15.26:0/139148203 >> 172.21.15.26:0/139148203 conn(0x55ec399d4da0 msgr2=0x55ec399d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.252+0000 7f1b2031eb80 1 -- 172.21.15.26:0/139148203 shutdown_connections 2023-07-13T22:21:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.252+0000 7f1b2031eb80 1 -- 172.21.15.26:0/139148203 wait complete. 2023-07-13T22:21:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.253+0000 7f1b2031eb80 1 -- start start 2023-07-13T22:21:18.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 Processor -- start 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399e0c70 0x55ec399efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399f0060 con 0x55ec399e0c70 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399f01d0 con 0x55ec399df4d0 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.254+0000 7f1b2031eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec399f0390 con 0x55ec399dfe20 2023-07-13T22:21:18.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.255+0000 7f1b2031eb80 10 client.0 _ll_register_callbacks cb 0x7f1b00007ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.255+0000 7f1b0659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.256+0000 7f1b05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.256+0000 7f1b05d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:42016/0 (socket says 172.21.15.26:42016) 2023-07-13T22:21:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.256+0000 7f1b05d9b700 1 -- 172.21.15.26:0/2013922416 learned_addr learned my addr 172.21.15.26:0/2013922416 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:18.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b06d9d700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399e0c70 0x55ec399efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b05d9b700 1 -- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 msgr2=0x55ec399ec5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399ec5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b05d9b700 1 -- 172.21.15.26:0/2013922416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399e0c70 msgr2=0x55ec399efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399e0c70 0x55ec399efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.257+0000 7f1b05d9b700 1 -- 172.21.15.26:0/2013922416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec39747c60 con 0x55ec399dfe20 2023-07-13T22:21:18.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.258+0000 7f1b05d9b700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1af8099620 tx=0x7f1af8099070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece901d7374f524b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.260+0000 7f1aedffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.260+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8098d80 con 0x55ec399dfe20 2023-07-13T22:21:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.261+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1af80c3e40 con 0x55ec399dfe20 2023-07-13T22:21:18.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.261+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1afc01d040 con 0x55ec399dfe20 2023-07-13T22:21:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.261+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af8010380 con 0x55ec399dfe20 2023-07-13T22:21:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.261+0000 7f1b2031eb80 10 client.26674 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.261+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ec3975d2b0 con 0x55ec399dfe20 2023-07-13T22:21:18.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.262+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1af80b5030 con 0x55ec399dfe20 2023-07-13T22:21:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1af80a1650 con 0x55ec399dfe20 2023-07-13T22:21:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 10 client.26674 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:18.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 20 client.26674 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec39a49f90 0x55ec39a4c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 4 client.26674 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:21:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.263+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ec398ec050 con 0x55ec39a49f90 2023-07-13T22:21:18.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.265+0000 7f1b0659c700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec39a49f90 0x55ec39a4c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.266+0000 7f1b0659c700 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec39a49f90 0x55ec39a4c3a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aafcdbedd1a4d7bb server_cookie=ea35a5182700c350 in_seq=0 out_seq=0 2023-07-13T22:21:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.266+0000 7f1aedffb700 10 client.26674 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.266+0000 7f1aedffb700 1 -- 172.21.15.26:0/2013922416 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ec398ec050 con 0x55ec39a49f90 2023-07-13T22:21:18.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.266+0000 7f1aedffb700 10 client.26674 handle_command_reply: tid=0 2023-07-13T22:21:18.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.266+0000 7f1b2031eb80 1 client.26674 shutdown 2023-07-13T22:21:18.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 msgr2=0x55ec399ecb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1af8099620 tx=0x7f1af8099070 comp rx=0 tx=0).stop 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 shutdown_connections 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ec399e0c70 0x55ec399efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55ec39a49f90 0x55ec39a4c3a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ec399df4d0 0x55ec399ec5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 --2- 172.21.15.26:0/2013922416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec399dfe20 0x55ec399ecb00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 >> 172.21.15.26:0/2013922416 conn(0x55ec399d4da0 msgr2=0x55ec399f7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.267+0000 7f1aedffb700 0 client.26674 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.268+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 shutdown_connections 2023-07-13T22:21:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.268+0000 7f1b2031eb80 1 -- 172.21.15.26:0/2013922416 wait complete. 2023-07-13T22:21:18.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.268+0000 7f1b2031eb80 20 client.26674 trim_cache size 0 max 16384 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "89714725-5616-4a96-be7b-8ccbf43ff4d0": { 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "89714725-5616-4a96-be7b-8ccbf43ff4d0", 2023-07-13T22:21:18.255 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:18.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 msgr2=0x7f1af00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 0x7f1af00a3800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009b30 tx=0x7f1b00009040 comp rx=0 tx=0).stop 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 msgr2=0x7f1b081a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1af800b500 tx=0x7f1af800b810 comp rx=0 tx=0).stop 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 shutdown_connections 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1af00a1350 0x7f1af00a3800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b0810bd60 0x7f1b081abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b08070ee0 0x7f1b081a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 --2- 172.21.15.26:0/731609382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b08071830 0x7f1b081a4c70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:18.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.270+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 >> 172.21.15.26:0/731609382 conn(0x7f1b0806ca90 msgr2=0x7f1b0806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:18.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.280+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 shutdown_connections 2023-07-13T22:21:18.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:18.281+0000 7f1b0c9ff700 1 -- 172.21.15.26:0/731609382 wait complete. 2023-07-13T22:21:18.306 INFO:tasks.cephfs.filesystem:scrub status for tag:89714725-5616-4a96-be7b-8ccbf43ff4d0 - {'path': '/', 'tag': '89714725-5616-4a96-be7b-8ccbf43ff4d0', 'options': 'recursive,force'} 2023-07-13T22:21:18.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:21:18.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:21:18.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:21:18.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.699 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:18 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:21:18.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:21:18.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:21:18.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:18 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:21:18.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:21:18.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:21:18.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:18.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:18 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:19.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:19 smithi117 ceph-mon[129074]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 90 op/s 2023-07-13T22:21:19.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:19 smithi026 ceph-mon[95647]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 90 op/s 2023-07-13T22:21:19.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:19 smithi121 ceph-mon[129187]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 90 op/s 2023-07-13T22:21:21.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:21 smithi117 ceph-mon[129074]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2023-07-13T22:21:21.977 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:21 smithi026 ceph-mon[95647]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2023-07-13T22:21:21.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:21 smithi121 ceph-mon[129187]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 290 MiB/s wr, 89 op/s 2023-07-13T22:21:23.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:23 smithi117 ceph-mon[129074]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 85 op/s 2023-07-13T22:21:23.957 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:23 smithi026 ceph-mon[95647]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 85 op/s 2023-07-13T22:21:23.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:23 smithi121 ceph-mon[129187]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 265 MiB/s wr, 85 op/s 2023-07-13T22:21:25.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:25 smithi121 ceph-mon[129187]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 307 MiB/s wr, 95 op/s 2023-07-13T22:21:26.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:25 smithi117 ceph-mon[129074]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 307 MiB/s wr, 95 op/s 2023-07-13T22:21:26.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:25 smithi026 ceph-mon[95647]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 307 MiB/s wr, 95 op/s 2023-07-13T22:21:26.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:26 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:21:27.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:26 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:21:27.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:26 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:21:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:27 smithi117 ceph-mon[129074]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 261 MiB/s wr, 82 op/s 2023-07-13T22:21:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:27 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:21:28.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:27 smithi026 ceph-mon[95647]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 261 MiB/s wr, 82 op/s 2023-07-13T22:21:28.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:27 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:21:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:27 smithi121 ceph-mon[129187]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 261 MiB/s wr, 82 op/s 2023-07-13T22:21:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:27 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:21:30.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:29 smithi117 ceph-mon[129074]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 279 MiB/s wr, 91 op/s 2023-07-13T22:21:30.216 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:29 smithi026 ceph-mon[95647]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 279 MiB/s wr, 91 op/s 2023-07-13T22:21:30.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:29 smithi121 ceph-mon[129187]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 279 MiB/s wr, 91 op/s 2023-07-13T22:21:32.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:31 smithi117 ceph-mon[129074]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 299 MiB/s wr, 94 op/s 2023-07-13T22:21:32.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:31 smithi026 ceph-mon[95647]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 299 MiB/s wr, 94 op/s 2023-07-13T22:21:32.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:31 smithi121 ceph-mon[129187]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 299 MiB/s wr, 94 op/s 2023-07-13T22:21:34.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:33 smithi117 ceph-mon[129074]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 89 op/s 2023-07-13T22:21:34.202 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:33 smithi026 ceph-mon[95647]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 89 op/s 2023-07-13T22:21:34.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:33 smithi121 ceph-mon[129187]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 89 op/s 2023-07-13T22:21:36.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:35 smithi121 ceph-mon[129187]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 99 op/s 2023-07-13T22:21:36.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:35 smithi117 ceph-mon[129074]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 99 op/s 2023-07-13T22:21:36.451 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:35 smithi026 ceph-mon[95647]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 99 op/s 2023-07-13T22:21:38.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:38 smithi117 ceph-mon[129074]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 88 op/s 2023-07-13T22:21:38.463 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:38 smithi026 ceph-mon[95647]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 88 op/s 2023-07-13T22:21:38.489 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:38 smithi121 ceph-mon[129187]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 88 op/s 2023-07-13T22:21:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:40 smithi117 ceph-mon[129074]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 302 MiB/s wr, 98 op/s 2023-07-13T22:21:40.446 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:40 smithi026 ceph-mon[95647]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 302 MiB/s wr, 98 op/s 2023-07-13T22:21:40.488 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:40 smithi121 ceph-mon[129187]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 302 MiB/s wr, 98 op/s 2023-07-13T22:21:41.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:41 smithi117 ceph-mon[129074]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 324 MiB/s wr, 101 op/s 2023-07-13T22:21:41.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:41 smithi026 ceph-mon[95647]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 324 MiB/s wr, 101 op/s 2023-07-13T22:21:41.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:41 smithi121 ceph-mon[129187]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 324 MiB/s wr, 101 op/s 2023-07-13T22:21:42.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:42 smithi117 ceph-mon[129074]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:21:42.730 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:42 smithi026 ceph-mon[95647]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:21:42.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:42 smithi121 ceph-mon[129187]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 287 MiB/s wr, 92 op/s 2023-07-13T22:21:44.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:44 smithi117 ceph-mon[129074]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:21:44.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:44 smithi026 ceph-mon[95647]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:21:44.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:44 smithi121 ceph-mon[129187]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:21:46.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:46 smithi117 ceph-mon[129074]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 282 MiB/s wr, 89 op/s 2023-07-13T22:21:46.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:46 smithi026 ceph-mon[95647]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 282 MiB/s wr, 89 op/s 2023-07-13T22:21:46.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:46 smithi121 ceph-mon[129187]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 282 MiB/s wr, 89 op/s 2023-07-13T22:21:47.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:47 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:21:48.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:47 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:21:48.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:47 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:21:48.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:21:48.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.887+0000 7f44659ff700 1 -- 172.21.15.26:0/2457241917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460100560 msgr2=0x7f4460102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.887+0000 7f44659ff700 1 --2- 172.21.15.26:0/2457241917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460100560 0x7f4460102940 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4450009990 tx=0x7f4450009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.888+0000 7f44659ff700 1 -- 172.21.15.26:0/2457241917 shutdown_connections 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.888+0000 7f44659ff700 1 --2- 172.21.15.26:0/2457241917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460100560 0x7f4460102940 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.888+0000 7f44659ff700 1 --2- 172.21.15.26:0/2457241917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44601057a0 0x7f4460107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.888+0000 7f44659ff700 1 --2- 172.21.15.26:0/2457241917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4460102e80 0x7f4460105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.888+0000 7f44659ff700 1 -- 172.21.15.26:0/2457241917 >> 172.21.15.26:0/2457241917 conn(0x7f44600fa000 msgr2=0x7f44600fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.889+0000 7f44659ff700 1 -- 172.21.15.26:0/2457241917 shutdown_connections 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.889+0000 7f44659ff700 1 -- 172.21.15.26:0/2457241917 wait complete. 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.890+0000 7f44659ff700 1 Processor -- start 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 -- start start 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4460100560 0x7f4460193850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460102e80 0x7f4460193d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446019b3d0 con 0x7f4460102e80 2023-07-13T22:21:48.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4460069f30 con 0x7f4460100560 2023-07-13T22:21:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.891+0000 7f44659ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446006a0c0 con 0x7f44601057a0 2023-07-13T22:21:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.892+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.892+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60338/0 (socket says 172.21.15.26:60338) 2023-07-13T22:21:48.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.892+0000 7f445f7fe700 1 -- 172.21.15.26:0/2398071025 learned_addr learned my addr 172.21.15.26:0/2398071025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445effd700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4460100560 0x7f4460193850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445f7fe700 1 -- 172.21.15.26:0/2398071025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4460100560 msgr2=0x7f4460193850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445f7fe700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4460100560 0x7f4460193850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445f7fe700 1 -- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460102e80 msgr2=0x7f4460193d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445f7fe700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460102e80 0x7f4460193d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.893+0000 7f445f7fe700 1 -- 172.21.15.26:0/2398071025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44500095f0 con 0x7f44601057a0 2023-07-13T22:21:48.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.894+0000 7f445f7fe700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f445800b780 tx=0x7f445800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.909+0000 7f44649fd700 1 -- 172.21.15.26:0/2398071025 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800f840 con 0x7f44601057a0 2023-07-13T22:21:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.909+0000 7f44649fd700 1 -- 172.21.15.26:0/2398071025 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445800fe60 con 0x7f44601057a0 2023-07-13T22:21:48.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.909+0000 7f44649fd700 1 -- 172.21.15.26:0/2398071025 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800d3d0 con 0x7f44601057a0 2023-07-13T22:21:48.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.913+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446006a3a0 con 0x7f44601057a0 2023-07-13T22:21:48.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.913+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446006a8b0 con 0x7f44601057a0 2023-07-13T22:21:48.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.922+0000 7f4479274b80 1 Processor -- start 2023-07-13T22:21:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 -- start start 2023-07-13T22:21:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6a46cb0 con 0x55cfe6b42c70 2023-07-13T22:21:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6a99a40 con 0x55cfe6b414d0 2023-07-13T22:21:48.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.923+0000 7f4479274b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6a92ed0 con 0x55cfe6b41e20 2023-07-13T22:21:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.933+0000 7f445effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.936+0000 7f445effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60344/0 (socket says 172.21.15.26:60344) 2023-07-13T22:21:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.936+0000 7f445effd700 1 -- 172.21.15.26:0/378915083 learned_addr learned my addr 172.21.15.26:0/378915083 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.936+0000 7f445e7fc700 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.936+0000 7f445e7fc700 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.938+0000 7f44649fd700 1 -- 172.21.15.26:0/2398071025 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4458017670 con 0x7f44601057a0 2023-07-13T22:21:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.940+0000 7f445effd700 1 -- 172.21.15.26:0/378915083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 msgr2=0x55cfe6b418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.940+0000 7f445effd700 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.940+0000 7f445effd700 1 -- 172.21.15.26:0/378915083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 msgr2=0x55cfe6b430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.940+0000 7f445effd700 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b430e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.940+0000 7f445effd700 1 -- 172.21.15.26:0/378915083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfe68c0de0 con 0x55cfe6b41e20 2023-07-13T22:21:48.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.941+0000 7f445effd700 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4450006010 tx=0x7f4450009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59bdab1dff65336 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.954+0000 7f44649fd700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 0x7f44480a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.955+0000 7f44649fd700 1 -- 172.21.15.26:0/2398071025 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f44580ccba0 con 0x7f44601057a0 2023-07-13T22:21:48.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.957+0000 7f445e7fc700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 0x7f44480a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.957+0000 7f4445ffb700 1 -- 172.21.15.26:0/378915083 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445001d070 con 0x55cfe6b41e20 2023-07-13T22:21:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.957+0000 7f4445ffb700 1 -- 172.21.15.26:0/378915083 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445000be80 con 0x55cfe6b41e20 2023-07-13T22:21:48.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.957+0000 7f4445ffb700 1 -- 172.21.15.26:0/378915083 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450005620 con 0x55cfe6b41e20 2023-07-13T22:21:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.958+0000 7f4479274b80 1 -- 172.21.15.26:0/378915083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 msgr2=0x55cfe6b426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.958+0000 7f4479274b80 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4450006010 tx=0x7f4450009ee0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.959+0000 7f445e7fc700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 0x7f44480a3800 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4454001d60 tx=0x7f445400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 -- 172.21.15.26:0/378915083 shutdown_connections 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 --2- 172.21.15.26:0/378915083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b41e20 0x55cfe6b426e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 -- 172.21.15.26:0/378915083 >> 172.21.15.26:0/378915083 conn(0x55cfe6b36da0 msgr2=0x55cfe6b3c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.960+0000 7f4479274b80 1 -- 172.21.15.26:0/378915083 shutdown_connections 2023-07-13T22:21:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.961+0000 7f4479274b80 1 -- 172.21.15.26:0/378915083 wait complete. 2023-07-13T22:21:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.961+0000 7f4479274b80 1 -- start start 2023-07-13T22:21:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.962+0000 7f4479274b80 1 Processor -- start 2023-07-13T22:21:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6a32ac0 con 0x55cfe6b41e20 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6a327c0 con 0x55cfe6b414d0 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6916920 con 0x55cfe6b42c70 2023-07-13T22:21:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.963+0000 7f4479274b80 10 client.0 _ll_register_callbacks cb 0x55cfe6a30760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.964+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.964+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60358/0 (socket says 172.21.15.26:60358) 2023-07-13T22:21:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.964+0000 7f445f7fe700 1 -- 172.21.15.26:0/1806336167 learned_addr learned my addr 172.21.15.26:0/1806336167 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:48.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.964+0000 7f445effd700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 95 op/s 2023-07-13T22:21:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.953 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:48 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.966+0000 7f445f7fe700 1 -- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 msgr2=0x55cfe6b4b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.966+0000 7f445f7fe700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b4b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.966+0000 7f445f7fe700 1 -- 172.21.15.26:0/1806336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 msgr2=0x55cfe6b4b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.966+0000 7f445f7fe700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.966+0000 7f445f7fe700 1 -- 172.21.15.26:0/1806336167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44600fe840 con 0x55cfe6b42c70 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.968+0000 7f445f7fe700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44580969d0 tx=0x7f445809e010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16ca025df5c802b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.969+0000 7f443effd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.969+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580983c0 con 0x55cfe6b42c70 2023-07-13T22:21:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.969+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4458099450 con 0x55cfe6b42c70 2023-07-13T22:21:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.971+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cfe69ec4f0 con 0x55cfe6b42c70 2023-07-13T22:21:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.971+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4458099630 con 0x55cfe6b42c70 2023-07-13T22:21:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 10 client.26683 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cfe68a9c60 con 0x55cfe6b42c70 2023-07-13T22:21:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cfe69ec4f0 con 0x55cfe6b42c70 2023-07-13T22:21:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4458094b70 con 0x55cfe6b42c70 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 10 client.26683 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 10 client.26683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 20 client.26683 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6b56d50 0x55cfe6b59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.972+0000 7f4479274b80 4 client.26683 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.973+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfe6a8beb0 con 0x55cfe6b56d50 2023-07-13T22:21:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.973+0000 7f445effd700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6b56d50 0x55cfe6b59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.975+0000 7f445effd700 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6b56d50 0x55cfe6b59160 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f364d3c3f3864e74 server_cookie=9b7798cb23c4b857 in_seq=0 out_seq=0 2023-07-13T22:21:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.975+0000 7f443effd700 10 client.26683 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.976+0000 7f443effd700 1 -- 172.21.15.26:0/1806336167 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cfe6a8beb0 con 0x55cfe6b56d50 2023-07-13T22:21:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.976+0000 7f443effd700 10 client.26683 handle_command_reply: tid=0 2023-07-13T22:21:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.977+0000 7f4479274b80 1 client.26683 shutdown 2023-07-13T22:21:48.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.977+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 msgr2=0x55cfe6b52680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.977+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f44580969d0 tx=0x7f445809e010 comp rx=0 tx=0).stop 2023-07-13T22:21:48.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 shutdown_connections 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6b56d50 0x55cfe6b59160 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b52680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b4b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 --2- 172.21.15.26:0/1806336167 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 >> 172.21.15.26:0/1806336167 conn(0x55cfe6b36da0 msgr2=0x55cfe6b3b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f443effd700 0 client.26683 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:48.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 shutdown_connections 2023-07-13T22:21:48.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.978+0000 7f4479274b80 1 -- 172.21.15.26:0/1806336167 wait complete. 2023-07-13T22:21:48.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.979+0000 7f4479274b80 20 client.26683 trim_cache size 0 max 16384 2023-07-13T22:21:48.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.999+0000 7f4479274b80 1 Processor -- start 2023-07-13T22:21:48.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.999+0000 7f4479274b80 1 -- start start 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:48.999+0000 7f4479274b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f4479274b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f4479274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f4479274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b56b30 con 0x55cfe6b42c70 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f4479274b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b4bd80 con 0x55cfe6b41e20 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f4479274b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b48dd0 con 0x55cfe6b414d0 2023-07-13T22:21:48.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f445effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f445effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60364/0 (socket says 172.21.15.26:60364) 2023-07-13T22:21:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.000+0000 7f445effd700 1 -- 172.21.15.26:0/3145510092 learned_addr learned my addr 172.21.15.26:0/3145510092 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:48.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445e7fc700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445f7fe700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 -- 172.21.15.26:0/3145510092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b41e20 msgr2=0x55cfe6b52640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 -- 172.21.15.26:0/3145510092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 msgr2=0x55cfe6b45b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b45b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 -- 172.21.15.26:0/3145510092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfe68e7250 con 0x55cfe6b414d0 2023-07-13T22:21:48.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445f7fe700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:48.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f445effd700 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4450009f50 tx=0x7f4450003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3923c580d268d833 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.001+0000 7f44467fc700 1 -- 172.21.15.26:0/3145510092 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445001d040 con 0x55cfe6b414d0 2023-07-13T22:21:48.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.002+0000 7f44467fc700 1 -- 172.21.15.26:0/3145510092 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445001e040 con 0x55cfe6b414d0 2023-07-13T22:21:48.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.002+0000 7f44467fc700 1 -- 172.21.15.26:0/3145510092 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500054c0 con 0x55cfe6b414d0 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 95 op/s 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.990 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:48 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- 172.21.15.26:0/3145510092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 msgr2=0x55cfe6b49020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4450009f50 tx=0x7f4450003680 comp rx=0 tx=0).stop 2023-07-13T22:21:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- 172.21.15.26:0/3145510092 shutdown_connections 2023-07-13T22:21:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b42c70 0x55cfe6b45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b414d0 0x55cfe6b49020 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 --2- 172.21.15.26:0/3145510092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- 172.21.15.26:0/3145510092 >> 172.21.15.26:0/3145510092 conn(0x55cfe6b36da0 msgr2=0x55cfe6b49e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- 172.21.15.26:0/3145510092 shutdown_connections 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- 172.21.15.26:0/3145510092 wait complete. 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.003+0000 7f4479274b80 1 -- start start 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 Processor -- start 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b51e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b4e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b4ede0 con 0x55cfe6b41e20 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b4ef50 con 0x55cfe6b414d0 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.004+0000 7f4479274b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfe6b4f110 con 0x55cfe6b42c70 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.005+0000 7f4479274b80 10 client.0 _ll_register_callbacks cb 0x7f4458099950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.005+0000 7f445e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b51e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445e7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50366/0 (socket says 172.21.15.26:50366) 2023-07-13T22:21:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445e7fc700 1 -- 172.21.15.26:0/2623645336 learned_addr learned my addr 172.21.15.26:0/2623645336 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 -- 172.21.15.26:0/2623645336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 msgr2=0x55cfe6b4e8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b4e8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 -- 172.21.15.26:0/2623645336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 msgr2=0x55cfe6b52340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445effd700 1 -- 172.21.15.26:0/2623645336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450026040 con 0x55cfe6b414d0 2023-07-13T22:21:48.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.006+0000 7f445e7fc700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:21:48.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.007+0000 7f445effd700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b51e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f44500036f0 tx=0x7f445002c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb1e41a712643cb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.007+0000 7f443effd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.007+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500055f0 con 0x55cfe6b414d0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.007+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f445000be80 con 0x55cfe6b414d0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.007+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f445002c9f0 con 0x55cfe6b414d0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f445002cbd0 con 0x55cfe6b414d0 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 10 client.26687 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:48.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55cfe68bf2b0 con 0x55cfe6b414d0 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f445000be80 con 0x55cfe6b414d0 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f445000f6d0 con 0x55cfe6b414d0 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 10 client.26687 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 20 client.26687 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6babc30 0x55cfe6bae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 4 client.26687 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:21:48.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.008+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfe6a4e050 con 0x55cfe6babc30 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.009+0000 7f445e7fc700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6babc30 0x55cfe6bae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.010+0000 7f445e7fc700 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6babc30 0x55cfe6bae040 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15e573273e40d808 server_cookie=7f6a7df4533e511e in_seq=0 out_seq=0 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.010+0000 7f443effd700 10 client.26687 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.010+0000 7f443effd700 1 -- 172.21.15.26:0/2623645336 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cfe6a4e050 con 0x55cfe6babc30 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.010+0000 7f443effd700 10 client.26687 handle_command_reply: tid=0 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.010+0000 7f4479274b80 1 client.26687 shutdown 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 msgr2=0x55cfe6b51e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b51e00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f44500036f0 tx=0x7f445002c520 comp rx=0 tx=0).stop 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 shutdown_connections 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55cfe6babc30 0x55cfe6bae040 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cfe6b42c70 0x55cfe6b4e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfe6b414d0 0x55cfe6b51e00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 --2- 172.21.15.26:0/2623645336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55cfe6b41e20 0x55cfe6b52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 >> 172.21.15.26:0/2623645336 conn(0x55cfe6b36da0 msgr2=0x55cfe6b49020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f443effd700 0 client.26687 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 shutdown_connections 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.011+0000 7f4479274b80 1 -- 172.21.15.26:0/2623645336 wait complete. 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.012+0000 7f4479274b80 20 client.26687 trim_cache size 0 max 16384 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:21:49.002 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:21:49.003 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:49.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.012+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 msgr2=0x7f44480a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.012+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 0x7f44480a3800 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4454001d60 tx=0x7f445400b040 comp rx=0 tx=0).stop 2023-07-13T22:21:49.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.012+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 msgr2=0x7f446019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.012+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f445800b780 tx=0x7f445800bb40 comp rx=0 tx=0).stop 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 shutdown_connections 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44601057a0 0x7f446019ae00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4460100560 0x7f4460193850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f44480a1350 0x7f44480a3800 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 --2- 172.21.15.26:0/2398071025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4460102e80 0x7f4460193d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:49.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 >> 172.21.15.26:0/2398071025 conn(0x7f44600fa000 msgr2=0x7f44601011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:49.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 shutdown_connections 2023-07-13T22:21:49.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:49.013+0000 7f44659ff700 1 -- 172.21.15.26:0/2398071025 wait complete. 2023-07-13T22:21:49.035 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:21:49.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:21:49.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 95 op/s 2023-07-13T22:21:49.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:48 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:49.965 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:49 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi121 to 196.1M 2023-07-13T22:21:49.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:49 smithi026 ceph-mon[95647]: Unable to set osd_memory_target on smithi121 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-13T22:21:49.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:49 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi117 to 196.2M 2023-07-13T22:21:49.966 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:49 smithi026 ceph-mon[95647]: Unable to set osd_memory_target on smithi117 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-13T22:21:50.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:49 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi121 to 196.1M 2023-07-13T22:21:50.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:49 smithi117 ceph-mon[129074]: Unable to set osd_memory_target on smithi121 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-13T22:21:50.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:49 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi117 to 196.2M 2023-07-13T22:21:50.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:49 smithi117 ceph-mon[129074]: Unable to set osd_memory_target on smithi117 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-13T22:21:50.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 -- 172.21.15.26:0/3006550176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484071820 msgr2=0x7fd48410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 --2- 172.21.15.26:0/3006550176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484071820 0x7fd48410b7d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd478009990 tx=0x7fd478009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 -- 172.21.15.26:0/3006550176 shutdown_connections 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 --2- 172.21.15.26:0/3006550176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd484070ed0 0x7fd4840712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 --2- 172.21.15.26:0/3006550176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd48410bd60 0x7fd48410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 --2- 172.21.15.26:0/3006550176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484071820 0x7fd48410b7d0 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd478009990 tx=0x7fd478009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.215+0000 7fd48afff700 1 -- 172.21.15.26:0/3006550176 >> 172.21.15.26:0/3006550176 conn(0x7fd48406ca40 msgr2=0x7fd48406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.216+0000 7fd48afff700 1 -- 172.21.15.26:0/3006550176 shutdown_connections 2023-07-13T22:21:50.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.216+0000 7fd48afff700 1 -- 172.21.15.26:0/3006550176 wait complete. 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.217+0000 7fd48afff700 1 Processor -- start 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.218+0000 7fd48afff700 1 -- start start 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.218+0000 7fd48afff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.218+0000 7fd48afff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd48410bd60 0x7fd4841a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.218+0000 7fd48afff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4841a9ea0 0x7fd4841ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.218+0000 7fd48afff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4841ac850 con 0x7fd4841a9ea0 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.219+0000 7fd48afff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48410ec40 con 0x7fd48410bd60 2023-07-13T22:21:50.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.219+0000 7fd48afff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48410edd0 con 0x7fd484070ed0 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.219+0000 7fd488d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.219+0000 7fd488d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60388/0 (socket says 172.21.15.26:60388) 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.219+0000 7fd488d9b700 1 -- 172.21.15.26:0/1651756216 learned_addr learned my addr 172.21.15.26:0/1651756216 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.220+0000 7fd488d9b700 1 -- 172.21.15.26:0/1651756216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd48410bd60 msgr2=0x7fd4841a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.220+0000 7fd488d9b700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd48410bd60 0x7fd4841a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.220+0000 7fd488d9b700 1 -- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4841a9ea0 msgr2=0x7fd4841ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.220+0000 7fd488d9b700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4841a9ea0 0x7fd4841ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.220+0000 7fd488d9b700 1 -- 172.21.15.26:0/1651756216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780095f0 con 0x7fd484070ed0 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.221+0000 7fd488d9b700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd47400b530 tx=0x7fd47400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.229+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474011830 con 0x7fd484070ed0 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.229+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd474011e50 con 0x7fd484070ed0 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.229+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474010e50 con 0x7fd484070ed0 2023-07-13T22:21:50.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.230+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48410f0b0 con 0x7fd484070ed0 2023-07-13T22:21:50.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.230+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48410f5c0 con 0x7fd484070ed0 2023-07-13T22:21:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.237+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd4740104f0 con 0x7fd484070ed0 2023-07-13T22:21:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.240+0000 7fd481ffb700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 0x7fd46c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:50.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.241+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd4740cc360 con 0x7fd484070ed0 2023-07-13T22:21:50.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.242+0000 7fd483fff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 0x7fd46c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:50.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:49 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi121 to 196.1M 2023-07-13T22:21:50.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:49 smithi121 ceph-mon[129187]: Unable to set osd_memory_target on smithi121 to 205709414: error parsing value: Value '205709414' is below minimum 939524096 2023-07-13T22:21:50.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:49 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi117 to 196.2M 2023-07-13T22:21:50.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:49 smithi121 ceph-mon[129187]: Unable to set osd_memory_target on smithi117 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-13T22:21:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.249+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484067370 con 0x7fd484070ed0 2023-07-13T22:21:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.264+0000 7fd483fff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 0x7fd46c0a3870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478009550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:50.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.269+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd484067370 con 0x7fd484070ed0 2023-07-13T22:21:50.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.872+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd484067300 con 0x7fd484070ed0 2023-07-13T22:21:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.905+0000 7fd481ffb700 1 -- 172.21.15.26:0/1651756216 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fd474015030 con 0x7fd484070ed0 2023-07-13T22:21:50.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:21:50.893 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:21:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.925+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 msgr2=0x7fd46c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.925+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 0x7fd46c0a3870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478009550 comp rx=0 tx=0).stop 2023-07-13T22:21:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.925+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 msgr2=0x7fd4841a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.925+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd47400b530 tx=0x7fd47400b8f0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 shutdown_connections 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd46c0a13c0 0x7fd46c0a3870 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4841a9ea0 0x7fd4841ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd484070ed0 0x7fd4841a4900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 --2- 172.21.15.26:0/1651756216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd48410bd60 0x7fd4841a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.936+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 >> 172.21.15.26:0/1651756216 conn(0x7fd48406ca40 msgr2=0x7fd48406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.937+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 shutdown_connections 2023-07-13T22:21:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:50.937+0000 7fd48afff700 1 -- 172.21.15.26:0/1651756216 wait complete. 2023-07-13T22:21:50.940 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:21:51.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:21:51.183 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.184 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.184 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.184 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-07-13T22:21:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: Unable to set osd_memory_target on smithi026 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-13T22:21:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:21:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:21:51.185 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:21:51.186 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:50 smithi026 ceph-mon[95647]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:21:51.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-07-13T22:21:51.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: Unable to set osd_memory_target on smithi026 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-13T22:21:51.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:21:51.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:21:51.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:21:51.190 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:50 smithi117 ceph-mon[129074]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:21:51.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: Adjusting osd_memory_target on smithi026 to 222.0M 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: Unable to set osd_memory_target on smithi026 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:21:51.240 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:50 smithi121 ceph-mon[129187]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:21:51.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.571+0000 7f0fa5dff700 1 -- 172.21.15.26:0/868965604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa010bd60 msgr2=0x7f0fa010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.571+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/868965604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa010bd60 0x7f0fa010e1f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800b0a0 tx=0x7f0f9800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.572+0000 7f0fa5dff700 1 -- 172.21.15.26:0/868965604 shutdown_connections 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.572+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/868965604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa010bd60 0x7f0fa010e1f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.572+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/868965604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0070ee0 0x7f0fa00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.572+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/868965604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa0071830 0x7f0fa010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.573+0000 7f0fa5dff700 1 -- 172.21.15.26:0/868965604 >> 172.21.15.26:0/868965604 conn(0x7f0fa006ca90 msgr2=0x7f0fa006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.573+0000 7f0fa5dff700 1 -- 172.21.15.26:0/868965604 shutdown_connections 2023-07-13T22:21:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.573+0000 7f0fa5dff700 1 -- 172.21.15.26:0/868965604 wait complete. 2023-07-13T22:21:51.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 Processor -- start 2023-07-13T22:21:51.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 -- start start 2023-07-13T22:21:51.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0071830 0x7f0fa0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa010bd60 0x7f0fa01b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa0117060 con 0x7f0fa010bd60 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01171d0 con 0x7f0fa0070ee0 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.575+0000 7f0fa5dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa01b4e20 con 0x7f0fa0071830 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.576+0000 7f0f9f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.576+0000 7f0f9f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49116/0 (socket says 172.21.15.26:49116) 2023-07-13T22:21:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.576+0000 7f0f9f7fe700 1 -- 172.21.15.26:0/1354564902 learned_addr learned my addr 172.21.15.26:0/1354564902 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.576+0000 7f0f9effd700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0071830 0x7f0fa0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa010bd60 0x7f0fa01b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9f7fe700 1 -- 172.21.15.26:0/1354564902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0071830 msgr2=0x7f0fa0113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9f7fe700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0071830 0x7f0fa0113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9f7fe700 1 -- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa010bd60 msgr2=0x7f0fa01b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9f7fe700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa010bd60 0x7f0fa01b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.578+0000 7f0f9f7fe700 1 -- 172.21.15.26:0/1354564902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98009d00 con 0x7f0fa0070ee0 2023-07-13T22:21:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.579+0000 7f0f9f7fe700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9a0 tx=0x7f0f9000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.581+0000 7f0f9cff9700 1 -- 172.21.15.26:0/1354564902 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90009790 con 0x7f0fa0070ee0 2023-07-13T22:21:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.581+0000 7f0f9cff9700 1 -- 172.21.15.26:0/1354564902 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9000cb70 con 0x7f0fa0070ee0 2023-07-13T22:21:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.581+0000 7f0f9cff9700 1 -- 172.21.15.26:0/1354564902 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f90010ea0 con 0x7f0fa0070ee0 2023-07-13T22:21:51.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.581+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa01b4fc0 con 0x7f0fa0070ee0 2023-07-13T22:21:51.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.581+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa01b53f0 con 0x7f0fa0070ee0 2023-07-13T22:21:51.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.589+0000 7f0fb96f3b80 1 Processor -- start 2023-07-13T22:21:51.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 -- start start 2023-07-13T22:21:51.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c76c70 0x55b636c770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636b7acb0 con 0x55b636c76c70 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636bcda40 con 0x55b636c754d0 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.593+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636bc6ed0 con 0x55b636c75e20 2023-07-13T22:21:51.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.594+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.594+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49124/0 (socket says 172.21.15.26:49124) 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.594+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3465861199 learned_addr learned my addr 172.21.15.26:0/3465861199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3465861199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 msgr2=0x55b636c766e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c766e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3465861199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c76c70 msgr2=0x55b636c770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c76c70 0x55b636c770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3465861199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6369f4de0 con 0x55b636c754d0 2023-07-13T22:21:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.599+0000 7f0f9cff9700 1 -- 172.21.15.26:0/1354564902 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0f90010430 con 0x7f0fa0070ee0 2023-07-13T22:21:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.604+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0f98015000 tx=0x7f0f98007540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fefabe3c03d6c42 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.605+0000 7f0f9cff9700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 0x7f0f880a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.605+0000 7f0f9cff9700 1 -- 172.21.15.26:0/1354564902 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f90014070 con 0x7f0fa0070ee0 2023-07-13T22:21:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.606+0000 7f0f8dffb700 1 -- 172.21.15.26:0/3465861199 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800e050 con 0x55b636c754d0 2023-07-13T22:21:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.606+0000 7f0f8dffb700 1 -- 172.21.15.26:0/3465861199 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f98003a00 con 0x55b636c754d0 2023-07-13T22:21:51.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.606+0000 7f0f8dffb700 1 -- 172.21.15.26:0/3465861199 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98005680 con 0x55b636c754d0 2023-07-13T22:21:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.607+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3465861199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 msgr2=0x55b636c758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.607+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0f98015000 tx=0x7f0f98007540 comp rx=0 tx=0).stop 2023-07-13T22:21:51.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3465861199 shutdown_connections 2023-07-13T22:21:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c76c70 0x55b636c770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c758e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3465861199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3465861199 >> 172.21.15.26:0/3465861199 conn(0x55b636c6ada0 msgr2=0x55b636c701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3465861199 shutdown_connections 2023-07-13T22:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.609+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3465861199 wait complete. 2023-07-13T22:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.610+0000 7f0fb96f3b80 1 -- start start 2023-07-13T22:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.610+0000 7f0fb96f3b80 1 Processor -- start 2023-07-13T22:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636b66ac0 con 0x55b636c75e20 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636b667c0 con 0x55b636c754d0 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636a4a920 con 0x55b636c76c70 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.611+0000 7f0fb96f3b80 10 client.0 _ll_register_callbacks cb 0x55b636b64760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.612+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.612+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60416/0 (socket says 172.21.15.26:60416) 2023-07-13T22:21:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.612+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3526169512 learned_addr learned my addr 172.21.15.26:0/3526169512 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.612+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.613+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 msgr2=0x55b636c7f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.613+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.613+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3526169512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 msgr2=0x55b636c7f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.613+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.613+0000 7f0f9ffff700 1 -- 172.21.15.26:0/3526169512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa0117640 con 0x55b636c76c70 2023-07-13T22:21:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.614+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005190 tx=0x7f0f980099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8882706835665da server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.616+0000 7f0f82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.616+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800e030 con 0x55b636c76c70 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.616+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f98005610 con 0x55b636c76c70 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.617+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b636b204f0 con 0x55b636c76c70 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.617+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f980057d0 con 0x55b636c76c70 2023-07-13T22:21:51.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.618+0000 7f0fb96f3b80 10 client.26701 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:51.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.618+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b6369ddc60 con 0x55b636c76c70 2023-07-13T22:21:51.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b636b204f0 con 0x55b636c76c70 2023-07-13T22:21:51.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f9802f070 con 0x55b636c76c70 2023-07-13T22:21:51.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 10 client.26701 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 10 client.26701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 20 client.26701 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636c8a9d0 0x55b636c8cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 4 client.26701 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:51.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.623+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b636bbfeb0 con 0x55b636c8a9d0 2023-07-13T22:21:51.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.626+0000 7f0f9f7fe700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 0x7f0f880a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.627+0000 7f0f9f7fe700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 0x7f0f880a3800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f90096820 tx=0x7f0f90010ba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.627+0000 7f0f9effd700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636c8a9d0 0x55b636c8cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.630+0000 7f0f82ffd700 10 client.26701 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:51.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.630+0000 7f0f9effd700 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636c8a9d0 0x55b636c8cde0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db671805ad1b7547 server_cookie=6cf2f60d5225c067 in_seq=0 out_seq=0 2023-07-13T22:21:51.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.632+0000 7f0f82ffd700 1 -- 172.21.15.26:0/3526169512 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b636bbfeb0 con 0x55b636c8a9d0 2023-07-13T22:21:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.632+0000 7f0f82ffd700 10 client.26701 handle_command_reply: tid=0 2023-07-13T22:21:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.632+0000 7f0fb96f3b80 1 client.26701 shutdown 2023-07-13T22:21:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.632+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 msgr2=0x55b636c86750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.632+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005190 tx=0x7f0f980099e0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 shutdown_connections 2023-07-13T22:21:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636c8a9d0 0x55b636c8cde0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c76c70 0x55b636c86750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/3526169512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c7f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 >> 172.21.15.26:0/3526169512 conn(0x55b636c6ada0 msgr2=0x55b636c6f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0f82ffd700 0 client.26701 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:51.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 shutdown_connections 2023-07-13T22:21:51.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.633+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/3526169512 wait complete. 2023-07-13T22:21:51.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.637+0000 7f0fb96f3b80 20 client.26701 trim_cache size 0 max 16384 2023-07-13T22:21:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.671+0000 7f0fb96f3b80 1 Processor -- start 2023-07-13T22:21:51.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 -- start start 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c6a970 con 0x55b636c754d0 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c876c0 con 0x55b636c76c70 2023-07-13T22:21:51.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.672+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c7fe50 con 0x55b636c75e20 2023-07-13T22:21:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.673+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.673+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49148/0 (socket says 172.21.15.26:49148) 2023-07-13T22:21:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.673+0000 7f0f9ffff700 1 -- 172.21.15.26:0/1397128336 learned_addr learned my addr 172.21.15.26:0/1397128336 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.674+0000 7f0f9effd700 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.674+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.674+0000 7f0f9ffff700 1 -- 172.21.15.26:0/1397128336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 msgr2=0x55b636c866e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.674+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c866e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.675+0000 7f0f9ffff700 1 -- 172.21.15.26:0/1397128336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c754d0 msgr2=0x55b636c7f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.675+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.675+0000 7f0f9ffff700 1 -- 172.21.15.26:0/1397128336 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b636c77560 con 0x55b636c76c70 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.675+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802fb70 tx=0x7f0f98009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b244f1154f4e74c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.676+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/1397128336 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800e070 con 0x55b636c76c70 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.676+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/1397128336 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f980233f0 con 0x55b636c76c70 2023-07-13T22:21:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.676+0000 7f0f8e7fc700 1 -- 172.21.15.26:0/1397128336 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9802fba0 con 0x55b636c76c70 2023-07-13T22:21:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/1397128336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 msgr2=0x55b636c7f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0f9802fb70 tx=0x7f0f98009450 comp rx=0 tx=0).stop 2023-07-13T22:21:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/1397128336 shutdown_connections 2023-07-13T22:21:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c754d0 0x55b636c7f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c7f710 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/1397128336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c75e20 0x55b636c866e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/1397128336 >> 172.21.15.26:0/1397128336 conn(0x55b636c6ada0 msgr2=0x55b636c79bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/1397128336 shutdown_connections 2023-07-13T22:21:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.677+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/1397128336 wait complete. 2023-07-13T22:21:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.678+0000 7f0fb96f3b80 1 -- start start 2023-07-13T22:21:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.678+0000 7f0fb96f3b80 1 Processor -- start 2023-07-13T22:21:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c86570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c82ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c83050 con 0x55b636c75e20 2023-07-13T22:21:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c831c0 con 0x55b636c76c70 2023-07-13T22:21:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b636c83380 con 0x55b636c754d0 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0fb96f3b80 10 client.0 _ll_register_callbacks cb 0x7f0f9800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0f9ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60422/0 (socket says 172.21.15.26:60422) 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0f9ffff700 1 -- 172.21.15.26:0/4057611016 learned_addr learned my addr 172.21.15.26:0/4057611016 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.679+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c82ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9effd700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c86570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 -- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 msgr2=0x55b636c82ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c82ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 -- 172.21.15.26:0/4057611016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 msgr2=0x55b636c86570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c86570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 -- 172.21.15.26:0/4057611016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6369ddc60 con 0x55b636c754d0 2023-07-13T22:21:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f9ffff700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007580 tx=0x7f0f98033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359c4add797db6d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.680+0000 7f0f82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800e070 con 0x55b636c754d0 2023-07-13T22:21:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0f98005610 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f98003a00 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f980057d0 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0fb96f3b80 10 client.26707 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b636a1b250 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.681+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f98003a00 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0f98047070 con 0x55b636c754d0 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 10 client.26707 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:51.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 10 client.26707 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 20 client.26707 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636cdf780 0x55b636ce1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 4 client.26707 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.682+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b636b82050 con 0x55b636cdf780 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.683+0000 7f0f9effd700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636cdf780 0x55b636ce1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0f82ffd700 10 client.26707 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0f9effd700 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636cdf780 0x55b636ce1b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=675073af0875afce server_cookie=8f07e292e13b15c0 in_seq=0 out_seq=0 2023-07-13T22:21:51.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0f82ffd700 1 -- 172.21.15.26:0/4057611016 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b636b82050 con 0x55b636cdf780 2023-07-13T22:21:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0f82ffd700 10 client.26707 handle_command_reply: tid=0 2023-07-13T22:21:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0fb96f3b80 1 client.26707 shutdown 2023-07-13T22:21:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 msgr2=0x55b636c86030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.684+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007580 tx=0x7f0f98033040 comp rx=0 tx=0).stop 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 shutdown_connections 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55b636cdf780 0x55b636ce1b60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b636c76c70 0x55b636c82ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b636c754d0 0x55b636c86030 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 --2- 172.21.15.26:0/4057611016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b636c75e20 0x55b636c86570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0f82ffd700 0 client.26707 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:21:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 >> 172.21.15.26:0/4057611016 conn(0x55b636c6ada0 msgr2=0x55b636c7ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 shutdown_connections 2023-07-13T22:21:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.685+0000 7f0fb96f3b80 1 -- 172.21.15.26:0/4057611016 wait complete. 2023-07-13T22:21:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.686+0000 7f0fb96f3b80 20 client.26707 trim_cache size 0 max 16384 2023-07-13T22:21:51.674 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.686+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 msgr2=0x7f0f880a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.686+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 0x7f0f880a3800 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0f90096820 tx=0x7f0f90010ba0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.686+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 msgr2=0x7f0fa0113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.686+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000e9a0 tx=0x7f0f9000ecb0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 shutdown_connections 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0f880a1350 0x7f0f880a3800 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0fa010bd60 0x7f0fa01b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fa0070ee0 0x7f0fa0113a40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 --2- 172.21.15.26:0/1354564902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fa0071830 0x7f0fa0113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 >> 172.21.15.26:0/1354564902 conn(0x7f0fa006ca90 msgr2=0x7f0fa006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 shutdown_connections 2023-07-13T22:21:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:51.687+0000 7f0fa5dff700 1 -- 172.21.15.26:0/1354564902 wait complete. 2023-07-13T22:21:51.708 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:21:52.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:51 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1651756216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:52.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:51 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1651756216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:52.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:51 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1651756216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:21:52.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.281+0000 7f80461ff700 1 -- 172.21.15.26:0/2269478847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 msgr2=0x7f804010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.281+0000 7f80461ff700 1 --2- 172.21.15.26:0/2269478847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f804010e1f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f80300098f0 tx=0x7f8030009c00 comp rx=0 tx=0).stop 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 -- 172.21.15.26:0/2269478847 shutdown_connections 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 --2- 172.21.15.26:0/2269478847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f804010e1f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 --2- 172.21.15.26:0/2269478847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040070ee0 0x7f80400712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 --2- 172.21.15.26:0/2269478847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040071830 0x7f804010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 -- 172.21.15.26:0/2269478847 >> 172.21.15.26:0/2269478847 conn(0x7f804006ca90 msgr2=0x7f804006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 -- 172.21.15.26:0/2269478847 shutdown_connections 2023-07-13T22:21:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.288+0000 7f80461ff700 1 -- 172.21.15.26:0/2269478847 wait complete. 2023-07-13T22:21:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.290+0000 7f80461ff700 1 Processor -- start 2023-07-13T22:21:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.290+0000 7f80461ff700 1 -- start start 2023-07-13T22:21:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.290+0000 7f80461ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 0x7f8040113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.290+0000 7f80461ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040071830 0x7f8040113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f80461ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f80461ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401170c0 con 0x7f8040070ee0 2023-07-13T22:21:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f80461ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040117230 con 0x7f804010bd60 2023-07-13T22:21:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f80461ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401b4d90 con 0x7f8040071830 2023-07-13T22:21:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f803ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f803ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:49162/0 (socket says 172.21.15.26:49162) 2023-07-13T22:21:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f803ffff700 1 -- 172.21.15.26:0/4060970768 learned_addr learned my addr 172.21.15.26:0/4060970768 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f803f7fe700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 0x7f8040113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.291+0000 7f803effd700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040071830 0x7f8040113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 -- 172.21.15.26:0/4060970768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040071830 msgr2=0x7f8040113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040071830 0x7f8040113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 -- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 msgr2=0x7f8040113aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 0x7f8040113aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 -- 172.21.15.26:0/4060970768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030009520 con 0x7f804010bd60 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803f7fe700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 0x7f8040113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.292+0000 7f803ffff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f803000b2e0 tx=0x7f803000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.293+0000 7f803cff9700 1 -- 172.21.15.26:0/4060970768 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803001c070 con 0x7f804010bd60 2023-07-13T22:21:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.293+0000 7f803cff9700 1 -- 172.21.15.26:0/4060970768 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803000bba0 con 0x7f804010bd60 2023-07-13T22:21:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.293+0000 7f803cff9700 1 -- 172.21.15.26:0/4060970768 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030004e40 con 0x7f804010bd60 2023-07-13T22:21:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.293+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80401b4f90 con 0x7f804010bd60 2023-07-13T22:21:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.293+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80401b5300 con 0x7f804010bd60 2023-07-13T22:21:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.298+0000 7f8059af8b80 1 Processor -- start 2023-07-13T22:21:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.298+0000 7f803cff9700 1 -- 172.21.15.26:0/4060970768 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8030005030 con 0x7f804010bd60 2023-07-13T22:21:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 -- start start 2023-07-13T22:21:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c6c70 0x561af31c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af30cacb0 con 0x561af31c6c70 2023-07-13T22:21:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af311da40 con 0x561af31c5e20 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.299+0000 7f8059af8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af3116ed0 con 0x561af31c54d0 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.300+0000 7f803effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.300+0000 7f803effd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60464/0 (socket says 172.21.15.26:60464) 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.300+0000 7f803effd700 1 -- 172.21.15.26:0/565272747 learned_addr learned my addr 172.21.15.26:0/565272747 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.300+0000 7f803effd700 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c6c70 0x561af31c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.300+0000 7f803f7fe700 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 -- 172.21.15.26:0/565272747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 msgr2=0x561af31c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 -- 172.21.15.26:0/565272747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c6c70 msgr2=0x561af31c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c6c70 0x561af31c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 -- 172.21.15.26:0/565272747 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af2f44de0 con 0x561af31c54d0 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f803effd700 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8038007ac0 tx=0x7f803800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad337b1569223c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f802dffb700 1 -- 172.21.15.26:0/565272747 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038004bb0 con 0x561af31c54d0 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f802dffb700 1 -- 172.21.15.26:0/565272747 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803800f070 con 0x561af31c54d0 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.301+0000 7f802dffb700 1 -- 172.21.15.26:0/565272747 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803801b6e0 con 0x561af31c54d0 2023-07-13T22:21:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.302+0000 7f803cff9700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 0x7f80280a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.302+0000 7f803cff9700 1 -- 172.21.15.26:0/4060970768 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80300cda40 con 0x7f804010bd60 2023-07-13T22:21:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 -- 172.21.15.26:0/565272747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 msgr2=0x561af31c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8038007ac0 tx=0x7f803800b040 comp rx=0 tx=0).stop 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 -- 172.21.15.26:0/565272747 shutdown_connections 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c6c70 0x561af31c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31c58e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 --2- 172.21.15.26:0/565272747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 -- 172.21.15.26:0/565272747 >> 172.21.15.26:0/565272747 conn(0x561af31bada0 msgr2=0x561af31c01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f803f7fe700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 0x7f80280a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 -- 172.21.15.26:0/565272747 shutdown_connections 2023-07-13T22:21:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.303+0000 7f8059af8b80 1 -- 172.21.15.26:0/565272747 wait complete. 2023-07-13T22:21:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.304+0000 7f8059af8b80 1 -- start start 2023-07-13T22:21:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.304+0000 7f8059af8b80 1 Processor -- start 2023-07-13T22:21:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f803f7fe700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 0x7f80280a3750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f803400be30 tx=0x7f803400b5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31d66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af30b6ac0 con 0x561af31c54d0 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af30b67c0 con 0x561af31c5e20 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.305+0000 7f8059af8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af2f9a920 con 0x561af31c6c70 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.306+0000 7f8059af8b80 10 client.0 _ll_register_callbacks cb 0x561af30b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50486/0 (socket says 172.21.15.26:50486) 2023-07-13T22:21:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803f7fe700 1 -- 172.21.15.26:0/648834954 learned_addr learned my addr 172.21.15.26:0/648834954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803effd700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803f7fe700 1 -- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 msgr2=0x561af31d66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.307+0000 7f803ffff700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31d66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803f7fe700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31d66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803f7fe700 1 -- 172.21.15.26:0/648834954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 msgr2=0x561af31cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803f7fe700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803f7fe700 1 -- 172.21.15.26:0/648834954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8038011040 con 0x561af31c54d0 2023-07-13T22:21:52.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803ffff700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f803f7fe700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f803400bfd0 tx=0x7f8034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7239e2cca3fdb1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.308+0000 7f8022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803401de10 con 0x561af31c54d0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f803401d240 con 0x561af31c54d0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561af30704f0 con 0x561af31c54d0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80340108c0 con 0x561af31c54d0 2023-07-13T22:21:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8059af8b80 10 client.16281 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.309+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561af2f2dc60 con 0x561af31c54d0 2023-07-13T22:21:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561af30704f0 con 0x561af31c54d0 2023-07-13T22:21:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f803402f070 con 0x561af31c54d0 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 10 client.16281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 20 client.16281 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af31daf70 0x561af31dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 4 client.16281 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.310+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561af310feb0 con 0x561af31daf70 2023-07-13T22:21:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.311+0000 7f803effd700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af31daf70 0x561af31dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.312+0000 7f803effd700 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af31daf70 0x561af31dd380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e05a45e90c08e19 server_cookie=a383c74ab3156ff3 in_seq=0 out_seq=0 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.312+0000 7f8022ffd700 10 client.16281 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.312+0000 7f8022ffd700 1 -- 172.21.15.26:0/648834954 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561af310feb0 con 0x561af31daf70 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.312+0000 7f8022ffd700 10 client.16281 handle_command_reply: tid=0 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.313+0000 7f8059af8b80 1 client.16281 shutdown 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.313+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 msgr2=0x561af31cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.313+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f803400bfd0 tx=0x7f8034004750 comp rx=0 tx=0).stop 2023-07-13T22:21:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 shutdown_connections 2023-07-13T22:21:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af31daf70 0x561af31dd380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31cf160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31d66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 --2- 172.21.15.26:0/648834954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 >> 172.21.15.26:0/648834954 conn(0x561af31bada0 msgr2=0x561af31bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8022ffd700 0 client.16281 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 shutdown_connections 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.314+0000 7f8059af8b80 1 -- 172.21.15.26:0/648834954 wait complete. 2023-07-13T22:21:52.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.315+0000 7f8059af8b80 20 client.16281 trim_cache size 0 max 16384 2023-07-13T22:21:52.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.333+0000 7f8059af8b80 1 Processor -- start 2023-07-13T22:21:52.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.333+0000 7f8059af8b80 1 -- start start 2023-07-13T22:21:52.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.333+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.333+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.334+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.334+0000 7f8059af8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31bcec0 con 0x561af31c54d0 2023-07-13T22:21:52.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.334+0000 7f8059af8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31c1d00 con 0x561af31c5e20 2023-07-13T22:21:52.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.334+0000 7f8059af8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31c6340 con 0x561af31c6c70 2023-07-13T22:21:52.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60472/0 (socket says 172.21.15.26:60472) 2023-07-13T22:21:52.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803ffff700 1 -- 172.21.15.26:0/1911464583 learned_addr learned my addr 172.21.15.26:0/1911464583 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:52.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803f7fe700 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803f7fe700 1 -- 172.21.15.26:0/1911464583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 msgr2=0x561af31cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803f7fe700 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803f7fe700 1 -- 172.21.15.26:0/1911464583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 msgr2=0x561af31d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.335+0000 7f803effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.336+0000 7f803f7fe700 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.336+0000 7f803f7fe700 1 -- 172.21.15.26:0/1911464583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561af2f6b250 con 0x561af31c54d0 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.336+0000 7f803f7fe700 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31ce810 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8034012ac0 tx=0x7f803400b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405e7629f1c6ad1e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.337+0000 7f803effd700 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.337+0000 7f802e7fc700 1 -- 172.21.15.26:0/1911464583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034004020 con 0x561af31c54d0 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.337+0000 7f802e7fc700 1 -- 172.21.15.26:0/1911464583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803402c070 con 0x561af31c54d0 2023-07-13T22:21:52.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.337+0000 7f802e7fc700 1 -- 172.21.15.26:0/1911464583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034005330 con 0x561af31c54d0 2023-07-13T22:21:52.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 -- 172.21.15.26:0/1911464583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 msgr2=0x561af31ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31ce810 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8034012ac0 tx=0x7f803400b6b0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 -- 172.21.15.26:0/1911464583 shutdown_connections 2023-07-13T22:21:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c54d0 0x561af31ce810 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c6c70 0x561af31cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 --2- 172.21.15.26:0/1911464583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c5e20 0x561af31d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.338+0000 7f8059af8b80 1 -- 172.21.15.26:0/1911464583 >> 172.21.15.26:0/1911464583 conn(0x561af31bada0 msgr2=0x561af31e1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.339+0000 7f8059af8b80 1 -- 172.21.15.26:0/1911464583 shutdown_connections 2023-07-13T22:21:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.339+0000 7f8059af8b80 1 -- 172.21.15.26:0/1911464583 wait complete. 2023-07-13T22:21:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.339+0000 7f8059af8b80 1 -- start start 2023-07-13T22:21:52.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.340+0000 7f8059af8b80 1 Processor -- start 2023-07-13T22:21:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 0x561af31d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c6c70 0x561af31d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31d62f0 con 0x561af31c5e20 2023-07-13T22:21:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31d6460 con 0x561af31c6c70 2023-07-13T22:21:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561af31d6620 con 0x561af31c54d0 2023-07-13T22:21:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.341+0000 7f8059af8b80 10 client.0 _ll_register_callbacks cb 0x561af30b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60474/0 (socket says 172.21.15.26:60474) 2023-07-13T22:21:52.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 -- 172.21.15.26:0/3271981948 learned_addr learned my addr 172.21.15.26:0/3271981948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:52.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803effd700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 0x561af31d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 -- 172.21.15.26:0/3271981948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c6c70 msgr2=0x561af31d5db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:52.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c6c70 0x561af31d5db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 -- 172.21.15.26:0/3271981948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 msgr2=0x561af31d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 0x561af31d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.342+0000 7f803f7fe700 1 -- 172.21.15.26:0/3271981948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561af2f2dc60 con 0x561af31c54d0 2023-07-13T22:21:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.343+0000 7f803effd700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 0x561af31d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.343+0000 7f803f7fe700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8034017040 tx=0x7f803401d2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8a7947da173868 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:52.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.345+0000 7f8022ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.345+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034004430 con 0x561af31c54d0 2023-07-13T22:21:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.345+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f80340050b0 con 0x561af31c54d0 2023-07-13T22:21:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.346+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f803800f070 con 0x561af31c54d0 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.346+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8034005270 con 0x561af31c54d0 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.346+0000 7f8059af8b80 10 client.26722 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.346+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561af2f432b0 con 0x561af31c54d0 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f803800f070 con 0x561af31c54d0 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f803401da80 con 0x561af31c54d0 2023-07-13T22:21:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 10 client.26722 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 10 client.26722 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 20 client.26722 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af322fb80 0x561af3231f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 4 client.26722 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.347+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561af30d2050 con 0x561af322fb80 2023-07-13T22:21:52.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.349+0000 7f803effd700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af322fb80 0x561af3231f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:52.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.350+0000 7f8022ffd700 10 client.26722 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:52.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.350+0000 7f803effd700 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af322fb80 0x561af3231f90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e70b02366b544662 server_cookie=8947a83759cd1d41 in_seq=0 out_seq=0 2023-07-13T22:21:52.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.351+0000 7f8022ffd700 1 -- 172.21.15.26:0/3271981948 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561af30d2050 con 0x561af322fb80 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.351+0000 7f8022ffd700 10 client.26722 handle_command_reply: tid=0 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.351+0000 7f8059af8b80 1 client.26722 shutdown 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.351+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 msgr2=0x561af31d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.351+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8034017040 tx=0x7f803401d2f0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 shutdown_connections 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x561af322fb80 0x561af3231f90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561af31c6c70 0x561af31d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561af31c54d0 0x561af31d2850 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 --2- 172.21.15.26:0/3271981948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561af31c5e20 0x561af31d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 >> 172.21.15.26:0/3271981948 conn(0x561af31bada0 msgr2=0x561af31c9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.352+0000 7f8022ffd700 0 client.26722 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:21:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.353+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 shutdown_connections 2023-07-13T22:21:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.353+0000 7f8059af8b80 1 -- 172.21.15.26:0/3271981948 wait complete. 2023-07-13T22:21:52.341 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.353+0000 7f8059af8b80 20 client.26722 trim_cache size 0 max 16384 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.354+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 msgr2=0x7f80280a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.354+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 0x7f80280a3750 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f803400be30 tx=0x7f803400b5c0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.354+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 msgr2=0x7f80401b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.354+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f803000b2e0 tx=0x7f803000b840 comp rx=0 tx=0).stop 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 shutdown_connections 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f80280a12a0 0x7f80280a3750 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8040070ee0 0x7f8040113aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804010bd60 0x7f80401b4850 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 --2- 172.21.15.26:0/4060970768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8040071830 0x7f8040113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 >> 172.21.15.26:0/4060970768 conn(0x7f804006ca90 msgr2=0x7f804006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 shutdown_connections 2023-07-13T22:21:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:52.355+0000 7f80461ff700 1 -- 172.21.15.26:0/4060970768 wait complete. 2023-07-13T22:21:52.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:21:52.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:52 smithi026 ceph-mon[95647]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 84 op/s 2023-07-13T22:21:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 -- 172.21.15.26:0/2801233264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c10bd60 msgr2=0x7f082c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 --2- 172.21.15.26:0/2801233264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c10bd60 0x7f082c10e1f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f082400b0a0 tx=0x7f082400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 -- 172.21.15.26:0/2801233264 shutdown_connections 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 --2- 172.21.15.26:0/2801233264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c070ee0 0x7f082c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 --2- 172.21.15.26:0/2801233264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c10bd60 0x7f082c10e1f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 --2- 172.21.15.26:0/2801233264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c071830 0x7f082c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 -- 172.21.15.26:0/2801233264 >> 172.21.15.26:0/2801233264 conn(0x7f082c06ca90 msgr2=0x7f082c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.038+0000 7f08323ff700 1 -- 172.21.15.26:0/2801233264 shutdown_connections 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.039+0000 7f08323ff700 1 -- 172.21.15.26:0/2801233264 wait complete. 2023-07-13T22:21:53.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.040+0000 7f08323ff700 1 Processor -- start 2023-07-13T22:21:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.040+0000 7f08323ff700 1 -- start start 2023-07-13T22:21:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 0x7f082c113c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c071830 0x7f082c114160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 0x7f082c1b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1147c0 con 0x7f082c071830 2023-07-13T22:21:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1b4ea0 con 0x7f082c070ee0 2023-07-13T22:21:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.041+0000 7f08323ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1b4fe0 con 0x7f082c10bd60 2023-07-13T22:21:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.042+0000 7f082bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 0x7f082c113c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.042+0000 7f082bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 0x7f082c113c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60454/0 (socket says 172.21.15.26:60454) 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.042+0000 7f082bfff700 1 -- 172.21.15.26:0/3171734409 learned_addr learned my addr 172.21.15.26:0/3171734409 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.043+0000 7f082b7fe700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c071830 0x7f082c114160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.043+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 0x7f082c1b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 -- 172.21.15.26:0/3171734409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 msgr2=0x7f082c113c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 0x7f082c113c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 -- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c071830 msgr2=0x7f082c114160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c071830 0x7f082c114160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 -- 172.21.15.26:0/3171734409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824009d00 con 0x7f082c10bd60 2023-07-13T22:21:53.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.044+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 0x7f082c1b4960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f082400b070 tx=0x7f0824004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.047+0000 7f08297fa700 1 -- 172.21.15.26:0/3171734409 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082400e040 con 0x7f082c10bd60 2023-07-13T22:21:53.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.047+0000 7f08297fa700 1 -- 172.21.15.26:0/3171734409 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082400ce90 con 0x7f082c10bd60 2023-07-13T22:21:53.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.047+0000 7f08297fa700 1 -- 172.21.15.26:0/3171734409 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08240057c0 con 0x7f082c10bd60 2023-07-13T22:21:53.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.047+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c1b5210 con 0x7f082c10bd60 2023-07-13T22:21:53.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.048+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c1b5660 con 0x7f082c10bd60 2023-07-13T22:21:53.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.055+0000 7f0845d1bb80 1 Processor -- start 2023-07-13T22:21:53.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 -- start start 2023-07-13T22:21:53.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750d8d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750d8e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750d8f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750c92cb0 con 0x561750d8de20 2023-07-13T22:21:53.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750ce5a40 con 0x561750d8ec70 2023-07-13T22:21:53.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.057+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750cdeed0 con 0x561750d8d4d0 2023-07-13T22:21:53.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.058+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750d8d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.058+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750d8d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55212/0 (socket says 172.21.15.26:55212) 2023-07-13T22:21:53.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.058+0000 7f082b7fe700 1 -- 172.21.15.26:0/498255418 learned_addr learned my addr 172.21.15.26:0/498255418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.058+0000 7f082b7fe700 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750d8f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.060+0000 7f082bfff700 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750d8e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.064+0000 7f08297fa700 1 -- 172.21.15.26:0/3171734409 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0824005920 con 0x7f082c10bd60 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.067+0000 7f08297fa700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 0x7f08140a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.067+0000 7f08297fa700 1 -- 172.21.15.26:0/3171734409 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f08240ce370 con 0x7f082c10bd60 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.068+0000 7f082b7fe700 1 -- 172.21.15.26:0/498255418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 msgr2=0x561750d8d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.068+0000 7f082b7fe700 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750d8d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.068+0000 7f082b7fe700 1 -- 172.21.15.26:0/498255418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 msgr2=0x561750d8e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.068+0000 7f082b7fe700 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750d8e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.068+0000 7f082b7fe700 1 -- 172.21.15.26:0/498255418 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561750b0cde0 con 0x561750d8ec70 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.069+0000 7f082b7fe700 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750d8f0e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0820007fb0 tx=0x7f0820004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98edbf84ddbcd3db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.070+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 0x7f08140a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.071+0000 7f083099c700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 0x7f08140a3800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0824097cc0 tx=0x7f0824009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.071+0000 7f08127fc700 1 -- 172.21.15.26:0/498255418 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820018620 con 0x561750d8ec70 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.071+0000 7f08127fc700 1 -- 172.21.15.26:0/498255418 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0820018c40 con 0x561750d8ec70 2023-07-13T22:21:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.071+0000 7f08127fc700 1 -- 172.21.15.26:0/498255418 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820017990 con 0x561750d8ec70 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.072+0000 7f0845d1bb80 1 -- 172.21.15.26:0/498255418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 msgr2=0x561750d8f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.072+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750d8f0e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0820007fb0 tx=0x7f0820004750 comp rx=0 tx=0).stop 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 -- 172.21.15.26:0/498255418 shutdown_connections 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750d8f0e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750d8d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/498255418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750d8e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 -- 172.21.15.26:0/498255418 >> 172.21.15.26:0/498255418 conn(0x561750d82da0 msgr2=0x561750d881d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 -- 172.21.15.26:0/498255418 shutdown_connections 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.074+0000 7f0845d1bb80 1 -- 172.21.15.26:0/498255418 wait complete. 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.075+0000 7f0845d1bb80 1 -- start start 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 Processor -- start 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750db9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8ec70 0x561750dc07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750c7eac0 con 0x561750d8ec70 2023-07-13T22:21:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750c7e7c0 con 0x561750d8de20 2023-07-13T22:21:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.076+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750b62920 con 0x561750d8d4d0 2023-07-13T22:21:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.077+0000 7f0845d1bb80 10 client.0 _ll_register_callbacks cb 0x561750c7c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.077+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.077+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55214/0 (socket says 172.21.15.26:55214) 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.077+0000 7f082b7fe700 1 -- 172.21.15.26:0/1927808184 learned_addr learned my addr 172.21.15.26:0/1927808184 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082bfff700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750db9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082b7fe700 1 -- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 msgr2=0x561750db9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082b7fe700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750db9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082b7fe700 1 -- 172.21.15.26:0/1927808184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8ec70 msgr2=0x561750dc07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082b7fe700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8ec70 0x561750dc07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.078+0000 7f082b7fe700 1 -- 172.21.15.26:0/1927808184 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0820016310 con 0x561750d8d4d0 2023-07-13T22:21:53.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.079+0000 7f082b7fe700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0820006d60 tx=0x7f0820007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b87120f534aedef server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.082+0000 7f080fff7700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.082+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820017910 con 0x561750d8d4d0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.082+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0820000e40 con 0x561750d8d4d0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.083+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561750c384f0 con 0x561750d8d4d0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.083+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0820007390 con 0x561750d8d4d0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.083+0000 7f0845d1bb80 10 client.26731 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.083+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561750af5c60 con 0x561750d8d4d0 2023-07-13T22:21:53.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561750c384f0 con 0x561750d8d4d0 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0820024a50 con 0x561750d8d4d0 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 10 client.26731 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 10 client.26731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 20 client.26731 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750dc4d70 0x561750dc7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 4 client.26731 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.084+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561750cd7eb0 con 0x561750dc4d70 2023-07-13T22:21:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.085+0000 7f082bfff700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750dc4d70 0x561750dc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.087+0000 7f082bfff700 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750dc4d70 0x561750dc7180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40f3da560f386815 server_cookie=da0035f51fb27bc in_seq=0 out_seq=0 2023-07-13T22:21:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.092+0000 7f080fff7700 10 client.26731 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.092+0000 7f080fff7700 1 -- 172.21.15.26:0/1927808184 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561750cd7eb0 con 0x561750dc4d70 2023-07-13T22:21:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.092+0000 7f080fff7700 10 client.26731 handle_command_reply: tid=0 2023-07-13T22:21:53.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.092+0000 7f0845d1bb80 1 client.26731 shutdown 2023-07-13T22:21:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.093+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 msgr2=0x561750db9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.093+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0820006d60 tx=0x7f0820007590 comp rx=0 tx=0).stop 2023-07-13T22:21:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.093+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 shutdown_connections 2023-07-13T22:21:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.093+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8ec70 0x561750dc07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.093+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750dc4d70 0x561750dc7180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.094+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750db9220 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.094+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1927808184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750db9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.094+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 >> 172.21.15.26:0/1927808184 conn(0x561750d82da0 msgr2=0x561750d878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.104+0000 7f080fff7700 0 client.26731 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.104+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 shutdown_connections 2023-07-13T22:21:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.105+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1927808184 wait complete. 2023-07-13T22:21:53.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.124+0000 7f0845d1bb80 20 client.26731 trim_cache size 0 max 16384 2023-07-13T22:21:53.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.156+0000 7f0845d1bb80 1 Processor -- start 2023-07-13T22:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.165+0000 7f0845d1bb80 1 -- start start 2023-07-13T22:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.165+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8d4d0 0x561750dc6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.167+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.167+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8ec70 0x561750d91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.167+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750d894c0 con 0x561750d8d4d0 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.167+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750d89ca0 con 0x561750d8de20 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.167+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750dba250 con 0x561750d8ec70 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.168+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8ec70 0x561750d91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.168+0000 7f082bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.168+0000 7f082bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60474/0 (socket says 172.21.15.26:60474) 2023-07-13T22:21:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.168+0000 7f082bfff700 1 -- 172.21.15.26:0/1481513226 learned_addr learned my addr 172.21.15.26:0/1481513226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.170+0000 7f082bfff700 1 -- 172.21.15.26:0/1481513226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8ec70 msgr2=0x561750d91b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.170+0000 7f082bfff700 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8ec70 0x561750d91b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.170+0000 7f082bfff700 1 -- 172.21.15.26:0/1481513226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8d4d0 msgr2=0x561750dc6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.170+0000 7f082bfff700 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8d4d0 0x561750dc6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.170+0000 7f082bfff700 1 -- 172.21.15.26:0/1481513226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561750b33250 con 0x561750d8de20 2023-07-13T22:21:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.171+0000 7f082bfff700 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f081c009f50 tx=0x7f081c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ba56f1045d392f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.182+0000 7f0812ffd700 1 -- 172.21.15.26:0/1481513226 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c01c070 con 0x561750d8de20 2023-07-13T22:21:53.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.182+0000 7f0812ffd700 1 -- 172.21.15.26:0/1481513226 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f081c016430 con 0x561750d8de20 2023-07-13T22:21:53.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.182+0000 7f0812ffd700 1 -- 172.21.15.26:0/1481513226 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c0153c0 con 0x561750d8de20 2023-07-13T22:21:53.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.184+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1481513226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 msgr2=0x561750dc0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.184+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f081c009f50 tx=0x7f081c008040 comp rx=0 tx=0).stop 2023-07-13T22:21:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1481513226 shutdown_connections 2023-07-13T22:21:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8d4d0 0x561750dc6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8ec70 0x561750d91b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/1481513226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8de20 0x561750dc0760 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1481513226 >> 172.21.15.26:0/1481513226 conn(0x561750d82da0 msgr2=0x561750d95040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1481513226 shutdown_connections 2023-07-13T22:21:53.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.199+0000 7f0845d1bb80 1 -- 172.21.15.26:0/1481513226 wait complete. 2023-07-13T22:21:53.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.200+0000 7f0845d1bb80 1 -- start start 2023-07-13T22:21:53.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.200+0000 7f0845d1bb80 1 Processor -- start 2023-07-13T22:21:53.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750dbcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750dbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750dc01e0 con 0x561750d8de20 2023-07-13T22:21:53.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750dc0350 con 0x561750d8ec70 2023-07-13T22:21:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561750dc0510 con 0x561750d8d4d0 2023-07-13T22:21:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.201+0000 7f0845d1bb80 10 client.0 _ll_register_callbacks cb 0x7f0820007c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.202+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.202+0000 7f082b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55232/0 (socket says 172.21.15.26:55232) 2023-07-13T22:21:53.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.202+0000 7f082b7fe700 1 -- 172.21.15.26:0/85396878 learned_addr learned my addr 172.21.15.26:0/85396878 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:52 smithi117 ceph-mon[129074]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 84 op/s 2023-07-13T22:21:53.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.203+0000 7f082b7fe700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750dbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.211+0000 7f082b7fe700 1 -- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 msgr2=0x561750dbfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.211+0000 7f082b7fe700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750dbfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.211+0000 7f082b7fe700 1 -- 172.21.15.26:0/85396878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 msgr2=0x561750dbcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:53.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.211+0000 7f082b7fe700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750dbcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.211+0000 7f082b7fe700 1 -- 172.21.15.26:0/85396878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c009a50 con 0x561750d8d4d0 2023-07-13T22:21:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.213+0000 7f082b7fe700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0820006d30 tx=0x7f0820000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b730a019d5fe5e9d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.213+0000 7f080fff7700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.213+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082002a710 con 0x561750d8d4d0 2023-07-13T22:21:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.213+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f082003f030 con 0x561750d8d4d0 2023-07-13T22:21:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.214+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0820018c40 con 0x561750d8d4d0 2023-07-13T22:21:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.214+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0820007a50 con 0x561750d8d4d0 2023-07-13T22:21:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.217+0000 7f0845d1bb80 10 client.26737 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.217+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x561750b0b2b0 con 0x561750d8d4d0 2023-07-13T22:21:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.219+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0820018c40 con 0x561750d8d4d0 2023-07-13T22:21:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.220+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f082002c5f0 con 0x561750d8d4d0 2023-07-13T22:21:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.220+0000 7f0845d1bb80 10 client.26737 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.220+0000 7f0845d1bb80 10 client.26737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:21:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.220+0000 7f0845d1bb80 20 client.26737 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.227+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750e19930 0x561750e1bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.227+0000 7f0845d1bb80 4 client.26737 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:53.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.227+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561750c9a050 con 0x561750e19930 2023-07-13T22:21:53.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.236+0000 7f082bfff700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750e19930 0x561750e1bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.239+0000 7f082bfff700 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750e19930 0x561750e1bd40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=428f8aa53171cd91 server_cookie=c2d2ec0cbfc5f971 in_seq=0 out_seq=0 2023-07-13T22:21:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.239+0000 7f080fff7700 10 client.26737 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.239+0000 7f080fff7700 1 -- 172.21.15.26:0/85396878 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561750c9a050 con 0x561750e19930 2023-07-13T22:21:53.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.239+0000 7f080fff7700 10 client.26737 handle_command_reply: tid=0 2023-07-13T22:21:53.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.240+0000 7f0845d1bb80 1 client.26737 shutdown 2023-07-13T22:21:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.242+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 msgr2=0x561750dbc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.242+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0820006d30 tx=0x7f0820000bc0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 shutdown_connections 2023-07-13T22:21:53.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x561750e19930 0x561750e1bd40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561750d8ec70 0x561750dbfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561750d8d4d0 0x561750dbc740 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 --2- 172.21.15.26:0/85396878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561750d8de20 0x561750dbcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.243+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 >> 172.21.15.26:0/85396878 conn(0x561750d82da0 msgr2=0x561750d91b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.244+0000 7f080fff7700 0 client.26737 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:21:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.244+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 shutdown_connections 2023-07-13T22:21:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.244+0000 7f0845d1bb80 1 -- 172.21.15.26:0/85396878 wait complete. 2023-07-13T22:21:53.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.245+0000 7f0845d1bb80 20 client.26737 trim_cache size 0 max 16384 2023-07-13T22:21:53.231 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 msgr2=0x7f08140a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 0x7f08140a3800 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0824097cc0 tx=0x7f0824009040 comp rx=0 tx=0).stop 2023-07-13T22:21:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 msgr2=0x7f082c1b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 0x7f082c1b4960 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f082400b070 tx=0x7f0824004c80 comp rx=0 tx=0).stop 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 shutdown_connections 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082c10bd60 0x7f082c1b4960 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082c070ee0 0x7f082c113c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f08140a1350 0x7f08140a3800 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 --2- 172.21.15.26:0/3171734409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f082c071830 0x7f082c114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.248+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 >> 172.21.15.26:0/3171734409 conn(0x7f082c06ca90 msgr2=0x7f082c06e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.249+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 shutdown_connections 2023-07-13T22:21:53.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:53.250+0000 7f08323ff700 1 -- 172.21.15.26:0/3171734409 wait complete. 2023-07-13T22:21:53.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:52 smithi121 ceph-mon[129187]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 263 MiB/s wr, 84 op/s 2023-07-13T22:21:53.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:21:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.007+0000 7f0979dff700 1 -- 172.21.15.26:0/2197935194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974070ee0 msgr2=0x7f09740712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.007+0000 7f0979dff700 1 --2- 172.21.15.26:0/2197935194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974070ee0 0x7f09740712f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0964009990 tx=0x7f0964009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 -- 172.21.15.26:0/2197935194 shutdown_connections 2023-07-13T22:21:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 --2- 172.21.15.26:0/2197935194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974070ee0 0x7f09740712f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 --2- 172.21.15.26:0/2197935194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097410bd60 0x7f097410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 --2- 172.21.15.26:0/2197935194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0974071830 0x7f097410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 -- 172.21.15.26:0/2197935194 >> 172.21.15.26:0/2197935194 conn(0x7f097406ca90 msgr2=0x7f097406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 -- 172.21.15.26:0/2197935194 shutdown_connections 2023-07-13T22:21:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.008+0000 7f0979dff700 1 -- 172.21.15.26:0/2197935194 wait complete. 2023-07-13T22:21:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.010+0000 7f0979dff700 1 Processor -- start 2023-07-13T22:21:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.010+0000 7f0979dff700 1 -- start start 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 0x7f09741a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097410bd60 0x7f09741abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09741ac3a0 con 0x7f0974071830 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097410ec40 con 0x7f0974070ee0 2023-07-13T22:21:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0979dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097410edd0 con 0x7f097410bd60 2023-07-13T22:21:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f09737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 0x7f09741a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.011+0000 7f0972ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34028/0 (socket says 172.21.15.26:34028) 2023-07-13T22:21:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 -- 172.21.15.26:0/4112350063 learned_addr learned my addr 172.21.15.26:0/4112350063 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 -- 172.21.15.26:0/4112350063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097410bd60 msgr2=0x7f09741abdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097410bd60 0x7f09741abdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 -- 172.21.15.26:0/4112350063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 msgr2=0x7f09741a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 0x7f09741a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f0972ffd700 1 -- 172.21.15.26:0/4112350063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640095f0 con 0x7f0974071830 2023-07-13T22:21:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.012+0000 7f09737fe700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 0x7f09741a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.013+0000 7f0972ffd700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b550 tx=0x7f095c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.013+0000 7f0970ff9700 1 -- 172.21.15.26:0/4112350063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c011830 con 0x7f0974071830 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.013+0000 7f0970ff9700 1 -- 172.21.15.26:0/4112350063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095c011e50 con 0x7f0974071830 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.013+0000 7f0970ff9700 1 -- 172.21.15.26:0/4112350063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00f2e0 con 0x7f0974071830 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.013+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097410f0b0 con 0x7f0974071830 2023-07-13T22:21:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.014+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097410f5f0 con 0x7f0974071830 2023-07-13T22:21:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.021+0000 7f0970ff9700 1 -- 172.21.15.26:0/4112350063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f095c010560 con 0x7f0974071830 2023-07-13T22:21:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.021+0000 7f098d7c6b80 1 Processor -- start 2023-07-13T22:21:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.024+0000 7f0970ff9700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 0x7f09600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.024+0000 7f0970ff9700 1 -- 172.21.15.26:0/4112350063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f095c0cc310 con 0x7f0974071830 2023-07-13T22:21:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 -- start start 2023-07-13T22:21:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabb26cb0 con 0x555aabc214d0 2023-07-13T22:21:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabb79a40 con 0x555aabc21e20 2023-07-13T22:21:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.026+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabb72ed0 con 0x555aabc22c70 2023-07-13T22:21:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.027+0000 7f0973fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.027+0000 7f0973fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55258/0 (socket says 172.21.15.26:55258) 2023-07-13T22:21:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.027+0000 7f0973fff700 1 -- 172.21.15.26:0/1492897315 learned_addr learned my addr 172.21.15.26:0/1492897315 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.027+0000 7f09737fe700 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0972ffd700 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 -- 172.21.15.26:0/1492897315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 msgr2=0x555aabc226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 -- 172.21.15.26:0/1492897315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 msgr2=0x555aabc218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 -- 172.21.15.26:0/1492897315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aab9a0de0 con 0x555aabc22c70 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.028+0000 7f0973fff700 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bcb0 tx=0x7f096c00d050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81762352e03bbdaf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.029+0000 7f0973fff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 0x7f09600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.029+0000 7f0969ffb700 1 -- 172.21.15.26:0/1492897315 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c010070 con 0x555aabc22c70 2023-07-13T22:21:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.029+0000 7f0969ffb700 1 -- 172.21.15.26:0/1492897315 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c00dde0 con 0x555aabc22c70 2023-07-13T22:21:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.029+0000 7f0969ffb700 1 -- 172.21.15.26:0/1492897315 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c014410 con 0x555aabc22c70 2023-07-13T22:21:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.030+0000 7f098d7c6b80 1 -- 172.21.15.26:0/1492897315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 msgr2=0x555aabc230e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.030+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bcb0 tx=0x7f096c00d050 comp rx=0 tx=0).stop 2023-07-13T22:21:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.032+0000 7f098d7c6b80 1 -- 172.21.15.26:0/1492897315 shutdown_connections 2023-07-13T22:21:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.032+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.032+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc230e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.032+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/1492897315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.032+0000 7f098d7c6b80 1 -- 172.21.15.26:0/1492897315 >> 172.21.15.26:0/1492897315 conn(0x555aabc16da0 msgr2=0x555aabc1c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.033+0000 7f0973fff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 0x7f09600a3800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bcb0 tx=0x7f096c00d050 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.034+0000 7f098d7c6b80 1 -- 172.21.15.26:0/1492897315 shutdown_connections 2023-07-13T22:21:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.034+0000 7f098d7c6b80 1 -- 172.21.15.26:0/1492897315 wait complete. 2023-07-13T22:21:54.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.035+0000 7f098d7c6b80 1 -- start start 2023-07-13T22:21:54.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.035+0000 7f098d7c6b80 1 Processor -- start 2023-07-13T22:21:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc214d0 0x555aabc2b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabb12ac0 con 0x555aabc22c70 2023-07-13T22:21:54.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabb127c0 con 0x555aabc21e20 2023-07-13T22:21:54.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aab9f6920 con 0x555aabc214d0 2023-07-13T22:21:54.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.036+0000 7f098d7c6b80 10 client.0 _ll_register_callbacks cb 0x555aabb10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:54.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.037+0000 7f0973fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.037+0000 7f0973fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34048/0 (socket says 172.21.15.26:34048) 2023-07-13T22:21:54.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.037+0000 7f0973fff700 1 -- 172.21.15.26:0/2514795871 learned_addr learned my addr 172.21.15.26:0/2514795871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.038+0000 7f0973fff700 1 -- 172.21.15.26:0/2514795871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc214d0 msgr2=0x555aabc2b1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.038+0000 7f0973fff700 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc214d0 0x555aabc2b1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.038+0000 7f0973fff700 1 -- 172.21.15.26:0/2514795871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 msgr2=0x555aabc2b720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:21:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.038+0000 7f0973fff700 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.038+0000 7f0973fff700 1 -- 172.21.15.26:0/2514795871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c00b9b0 con 0x555aabc22c70 2023-07-13T22:21:54.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.039+0000 7f0973fff700 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f096c0063d0 tx=0x7f096c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bf9db9a11ae3b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.050+0000 7f095affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.050+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c02d070 con 0x555aabc22c70 2023-07-13T22:21:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.050+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f096c011070 con 0x555aabc22c70 2023-07-13T22:21:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.050+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555aabacc4f0 con 0x555aabc22c70 2023-07-13T22:21:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.051+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096c010030 con 0x555aabc22c70 2023-07-13T22:21:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.051+0000 7f098d7c6b80 10 client.16305 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.051+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555aab989c60 con 0x555aabc22c70 2023-07-13T22:21:54.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.052+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555aabacc4f0 con 0x555aabc22c70 2023-07-13T22:21:54.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.052+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f096c030020 con 0x555aabc22c70 2023-07-13T22:21:54.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 10 client.16305 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:21:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 20 client.16305 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc36e80 0x555aabc39290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 4 client.16305 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:54.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.054+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555aabb6beb0 con 0x555aabc36e80 2023-07-13T22:21:54.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.077+0000 7f09737fe700 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc36e80 0x555aabc39290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.128+0000 7f09737fe700 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc36e80 0x555aabc39290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deccbf422d11c5c0 server_cookie=8128934d191d09b in_seq=0 out_seq=0 2023-07-13T22:21:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.130+0000 7f095affd700 10 client.16305 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.130+0000 7f095affd700 1 -- 172.21.15.26:0/2514795871 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555aabb6beb0 con 0x555aabc36e80 2023-07-13T22:21:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.130+0000 7f095affd700 10 client.16305 handle_command_reply: tid=0 2023-07-13T22:21:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.130+0000 7f098d7c6b80 1 client.16305 shutdown 2023-07-13T22:21:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.139+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 msgr2=0x555aabc32760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.139+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f096c0063d0 tx=0x7f096c00b690 comp rx=0 tx=0).stop 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 shutdown_connections 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc36e80 0x555aabc39290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc32760 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc214d0 0x555aabc2b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2514795871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 >> 172.21.15.26:0/2514795871 conn(0x555aabc16da0 msgr2=0x555aabc1b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.149+0000 7f095affd700 0 client.16305 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:54.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.171+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 shutdown_connections 2023-07-13T22:21:54.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.171+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2514795871 wait complete. 2023-07-13T22:21:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.187+0000 7f098d7c6b80 20 client.16305 trim_cache size 0 max 16384 2023-07-13T22:21:54.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.243+0000 7f098d7c6b80 1 Processor -- start 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.244+0000 7f098d7c6b80 1 -- start start 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc214d0 0x555aabc1b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc38e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc2a690 con 0x555aabc22c70 2023-07-13T22:21:54.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc36c60 con 0x555aabc214d0 2023-07-13T22:21:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.245+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc2c360 con 0x555aabc21e20 2023-07-13T22:21:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.247+0000 7f0972ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.247+0000 7f09737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc214d0 0x555aabc1b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.247+0000 7f0972ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55282/0 (socket says 172.21.15.26:55282) 2023-07-13T22:21:54.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.247+0000 7f0972ffd700 1 -- 172.21.15.26:0/2254934442 learned_addr learned my addr 172.21.15.26:0/2254934442 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.248+0000 7f0972ffd700 1 -- 172.21.15.26:0/2254934442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc214d0 msgr2=0x555aabc1b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.248+0000 7f0972ffd700 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc214d0 0x555aabc1b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.248+0000 7f0972ffd700 1 -- 172.21.15.26:0/2254934442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 msgr2=0x555aabc38e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.248+0000 7f0972ffd700 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc38e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.248+0000 7f0972ffd700 1 -- 172.21.15.26:0/2254934442 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555aabc23560 con 0x555aabc21e20 2023-07-13T22:21:54.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.249+0000 7f0972ffd700 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f095c0993d0 tx=0x7f095c099570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d40d7bda32a594 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.261+0000 7f096a7fc700 1 -- 172.21.15.26:0/2254934442 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0996d0 con 0x555aabc21e20 2023-07-13T22:21:54.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.261+0000 7f096a7fc700 1 -- 172.21.15.26:0/2254934442 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095c09d700 con 0x555aabc21e20 2023-07-13T22:21:54.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.261+0000 7f096a7fc700 1 -- 172.21.15.26:0/2254934442 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c098bc0 con 0x555aabc21e20 2023-07-13T22:21:54.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.263+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2254934442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 msgr2=0x555aabc32720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.263+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f095c0993d0 tx=0x7f095c099570 comp rx=0 tx=0).stop 2023-07-13T22:21:54.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2254934442 shutdown_connections 2023-07-13T22:21:54.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc22c70 0x555aabc38e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc214d0 0x555aabc1b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/2254934442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc21e20 0x555aabc32720 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2254934442 >> 172.21.15.26:0/2254934442 conn(0x555aabc16da0 msgr2=0x555aabc29e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.265+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2254934442 shutdown_connections 2023-07-13T22:21:54.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.266+0000 7f098d7c6b80 1 -- 172.21.15.26:0/2254934442 wait complete. 2023-07-13T22:21:54.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.266+0000 7f098d7c6b80 1 -- start start 2023-07-13T22:21:54.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.267+0000 7f098d7c6b80 1 Processor -- start 2023-07-13T22:21:54.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc32370 con 0x555aabc214d0 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc324e0 con 0x555aabc21e20 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555aabc326a0 con 0x555aabc22c70 2023-07-13T22:21:54.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.268+0000 7f098d7c6b80 10 client.0 _ll_register_callbacks cb 0x7f096c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:54.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.269+0000 7f0973fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.275+0000 7f0973fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55286/0 (socket says 172.21.15.26:55286) 2023-07-13T22:21:54.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.275+0000 7f0973fff700 1 -- 172.21.15.26:0/179108966 learned_addr learned my addr 172.21.15.26:0/179108966 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:54.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.275+0000 7f0972ffd700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.277+0000 7f09737fe700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.286+0000 7f0973fff700 1 -- 172.21.15.26:0/179108966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 msgr2=0x555aabc2ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.286+0000 7f0973fff700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.286+0000 7f0973fff700 1 -- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 msgr2=0x555aabc2e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.286+0000 7f0973fff700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc2e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.286+0000 7f0973fff700 1 -- 172.21.15.26:0/179108966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097410f5f0 con 0x555aabc22c70 2023-07-13T22:21:54.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.293+0000 7f0973fff700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f096c0147a0 tx=0x7f096c0147f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bccac73c56a9a4b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.294+0000 7f095affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.294+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c010030 con 0x555aabc22c70 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.295+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f096c011070 con 0x555aabc22c70 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.296+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f096c00dde0 con 0x555aabc22c70 2023-07-13T22:21:54.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.296+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f096c02d070 con 0x555aabc22c70 2023-07-13T22:21:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.302+0000 7f098d7c6b80 10 client.26746 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.302+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555aab9c7250 con 0x555aabc22c70 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.308+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f096c00dde0 con 0x555aabc22c70 2023-07-13T22:21:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.308+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f096c0148b0 con 0x555aabc22c70 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 10 client.26746 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 10 client.26746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 20 client.26746 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc8bca0 0x555aabc8e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 4 client.26746 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555aabb2e050 con 0x555aabc8bca0 2023-07-13T22:21:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.309+0000 7f09737fe700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc8bca0 0x555aabc8e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.312+0000 7f09737fe700 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc8bca0 0x555aabc8e0b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a087f2a3f8787f9d server_cookie=46e2663957c206c8 in_seq=0 out_seq=0 2023-07-13T22:21:54.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.312+0000 7f095affd700 10 client.26746 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.312+0000 7f095affd700 1 -- 172.21.15.26:0/179108966 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555aabb2e050 con 0x555aabc8bca0 2023-07-13T22:21:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.312+0000 7f095affd700 10 client.26746 handle_command_reply: tid=0 2023-07-13T22:21:54.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.312+0000 7f098d7c6b80 1 client.26746 shutdown 2023-07-13T22:21:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.322+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 msgr2=0x555aabc31e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.322+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f096c0147a0 tx=0x7f096c0147f0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 shutdown_connections 2023-07-13T22:21:54.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555aabc8bca0 0x555aabc8e0b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555aabc214d0 0x555aabc2e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555aabc22c70 0x555aabc31e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 --2- 172.21.15.26:0/179108966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555aabc21e20 0x555aabc2ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.323+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 >> 172.21.15.26:0/179108966 conn(0x555aabc16da0 msgr2=0x555aabc1b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.335+0000 7f095affd700 0 client.26746 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:21:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.335+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 shutdown_connections 2023-07-13T22:21:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.335+0000 7f098d7c6b80 1 -- 172.21.15.26:0/179108966 wait complete. 2023-07-13T22:21:54.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.346+0000 7f098d7c6b80 20 client.26746 trim_cache size 0 max 16384 2023-07-13T22:21:54.342 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:54.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.379+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 msgr2=0x7f09600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.379+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 0x7f09600a3800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f096c00bcb0 tx=0x7f096c00d050 comp rx=0 tx=0).stop 2023-07-13T22:21:54.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.379+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 msgr2=0x7f09741a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:54.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.379+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b550 tx=0x7f095c00b910 comp rx=0 tx=0).stop 2023-07-13T22:21:54.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 shutdown_connections 2023-07-13T22:21:54.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f097410bd60 0x7f09741abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0974070ee0 0x7f09741a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f09600a1350 0x7f09600a3800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 --2- 172.21.15.26:0/4112350063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0974071830 0x7f09741a4d60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.388+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 >> 172.21.15.26:0/4112350063 conn(0x7f097406ca90 msgr2=0x7f097406de20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.389+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 shutdown_connections 2023-07-13T22:21:54.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:54.389+0000 7f0979dff700 1 -- 172.21.15.26:0/4112350063 wait complete. 2023-07-13T22:21:54.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:21:54.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:54 smithi117 ceph-mon[129074]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 92 op/s 2023-07-13T22:21:54.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:54 smithi026 ceph-mon[95647]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 92 op/s 2023-07-13T22:21:54.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:54 smithi121 ceph-mon[129187]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 92 op/s 2023-07-13T22:21:55.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.111+0000 7fc01edff700 1 -- 172.21.15.26:0/2631359337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018103670 msgr2=0x7fc018103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.111+0000 7fc01edff700 1 --2- 172.21.15.26:0/2631359337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018103670 0x7fc018103ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc010009990 tx=0x7fc010009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.112+0000 7fc01edff700 1 -- 172.21.15.26:0/2631359337 shutdown_connections 2023-07-13T22:21:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.112+0000 7fc01edff700 1 --2- 172.21.15.26:0/2631359337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018103670 0x7fc018103ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.112+0000 7fc01edff700 1 --2- 172.21.15.26:0/2631359337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0180fe350 0x7fc018100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.112+0000 7fc01edff700 1 --2- 172.21.15.26:0/2631359337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc018100cb0 0x7fc018103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.112+0000 7fc01edff700 1 -- 172.21.15.26:0/2631359337 >> 172.21.15.26:0/2631359337 conn(0x7fc0180fa000 msgr2=0x7fc0180fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.113+0000 7fc01edff700 1 -- 172.21.15.26:0/2631359337 shutdown_connections 2023-07-13T22:21:55.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.113+0000 7fc01edff700 1 -- 172.21.15.26:0/2631359337 wait complete. 2023-07-13T22:21:55.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.114+0000 7fc01edff700 1 Processor -- start 2023-07-13T22:21:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 -- start start 2023-07-13T22:21:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0180fe350 0x7fc01806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc018103670 0x7fc0181abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01806fb70 con 0x7fc0180fe350 2023-07-13T22:21:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018104970 con 0x7fc018103670 2023-07-13T22:21:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.115+0000 7fc01edff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc018104b30 con 0x7fc018100cb0 2023-07-13T22:21:55.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.116+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.116+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55304/0 (socket says 172.21.15.26:55304) 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.116+0000 7fc00ffff700 1 -- 172.21.15.26:0/2862886644 learned_addr learned my addr 172.21.15.26:0/2862886644 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.116+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0180fe350 0x7fc01806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.117+0000 7fc00ffff700 1 -- 172.21.15.26:0/2862886644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc018103670 msgr2=0x7fc0181abf70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.117+0000 7fc00ffff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc018103670 0x7fc0181abf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.117+0000 7fc00ffff700 1 -- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0180fe350 msgr2=0x7fc01806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.117+0000 7fc00ffff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0180fe350 0x7fc01806ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.117+0000 7fc00ffff700 1 -- 172.21.15.26:0/2862886644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100095f0 con 0x7fc018100cb0 2023-07-13T22:21:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.118+0000 7fc00ffff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc00800e9a0 tx=0x7fc00800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.118+0000 7fc00dffb700 1 -- 172.21.15.26:0/2862886644 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008009790 con 0x7fc018100cb0 2023-07-13T22:21:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.118+0000 7fc00dffb700 1 -- 172.21.15.26:0/2862886644 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00800cbd0 con 0x7fc018100cb0 2023-07-13T22:21:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.118+0000 7fc00dffb700 1 -- 172.21.15.26:0/2862886644 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008018680 con 0x7fc018100cb0 2023-07-13T22:21:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.118+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc018104e10 con 0x7fc018100cb0 2023-07-13T22:21:55.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.119+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0181052d0 con 0x7fc018100cb0 2023-07-13T22:21:55.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.124+0000 7fc00dffb700 1 -- 172.21.15.26:0/2862886644 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc008010430 con 0x7fc018100cb0 2023-07-13T22:21:55.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.127+0000 7fc0327b9b80 1 Processor -- start 2023-07-13T22:21:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.127+0000 7fc0327b9b80 1 -- start start 2023-07-13T22:21:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.127+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbe06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cae4cb0 con 0x55d16cbdfe20 2023-07-13T22:21:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cb37a40 con 0x55d16cbe0c70 2023-07-13T22:21:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cb30ed0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc00dffb700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 0x7fc0000a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc01d39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc01d39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55314/0 (socket says 172.21.15.26:55314) 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc01d39c700 1 -- 172.21.15.26:0/2088759650 learned_addr learned my addr 172.21.15.26:0/2088759650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.128+0000 7fc00dffb700 1 -- 172.21.15.26:0/2862886644 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc008014070 con 0x7fc018100cb0 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbe06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc00ffff700 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 -- 172.21.15.26:0/2088759650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 msgr2=0x55d16cbe10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 -- 172.21.15.26:0/2088759650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 msgr2=0x55d16cbe06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbe06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 -- 172.21.15.26:0/2088759650 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d16c95ede0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0100098e0 tx=0x7fc010005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f67b44f85402587 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.129+0000 7fc01d39c700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 0x7fc0000a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.130+0000 7fbffeffd700 1 -- 172.21.15.26:0/2088759650 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000bab0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.130+0000 7fbffeffd700 1 -- 172.21.15.26:0/2088759650 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc010004580 con 0x55d16cbdf4d0 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.130+0000 7fbffeffd700 1 -- 172.21.15.26:0/2088759650 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01000f880 con 0x55d16cbdf4d0 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2088759650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 msgr2=0x55d16cbdf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc01d39c700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 0x7fc0000a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc01000bfd0 tx=0x7fc01000fac0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc0100098e0 tx=0x7fc010005760 comp rx=0 tx=0).stop 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2088759650 shutdown_connections 2023-07-13T22:21:55.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbdf8e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2088759650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbe06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.131+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2088759650 >> 172.21.15.26:0/2088759650 conn(0x55d16cbd4da0 msgr2=0x55d16cbda1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.132+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2088759650 shutdown_connections 2023-07-13T22:21:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.132+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2088759650 wait complete. 2023-07-13T22:21:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.133+0000 7fc0327b9b80 1 -- start start 2023-07-13T22:21:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.133+0000 7fc0327b9b80 1 Processor -- start 2023-07-13T22:21:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 0x55d16cbe9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 0x55d16cbf04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cad0ac0 con 0x55d16cbe0c70 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cad07c0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16c9b4920 con 0x55d16cbdfe20 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc0327b9b80 10 client.0 _ll_register_callbacks cb 0x55d16cace760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc01cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc01cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60582/0 (socket says 172.21.15.26:60582) 2023-07-13T22:21:55.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc01cb9b700 1 -- 172.21.15.26:0/1702101774 learned_addr learned my addr 172.21.15.26:0/1702101774 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:55.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.134+0000 7fc00ffff700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 0x55d16cbe9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01cb9b700 1 -- 172.21.15.26:0/1702101774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 msgr2=0x55d16cbe9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01d39c700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 0x55d16cbf04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 0x55d16cbe9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01cb9b700 1 -- 172.21.15.26:0/1702101774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 msgr2=0x55d16cbf04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 0x55d16cbf04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01cb9b700 1 -- 172.21.15.26:0/1702101774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc018104610 con 0x55d16cbdf4d0 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc00ffff700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 0x55d16cbe9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.135+0000 7fc01d39c700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 0x55d16cbf04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc014007ba0 tx=0x7fc014007f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b99fc70a6b2e4c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fbff7fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:55.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140119c0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc014011b20 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d16ca8a4f0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.136+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc014011ce0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 10 client.26753 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d16c947c60 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d16ca8a4f0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc014014070 con 0x55d16cbdf4d0 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 10 client.26753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 20 client.26753 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cbf4cc0 0x55d16cbf70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 4 client.26753 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:55.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.137+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d16cb29eb0 con 0x55d16cbf4cc0 2023-07-13T22:21:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.138+0000 7fc00ffff700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cbf4cc0 0x55d16cbf70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.138+0000 7fc00ffff700 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cbf4cc0 0x55d16cbf70d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541ebfa58cf67339 server_cookie=99be6f2363edb402 in_seq=0 out_seq=0 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.138+0000 7fbff7fff700 10 client.26753 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.139+0000 7fbff7fff700 1 -- 172.21.15.26:0/1702101774 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d16cb29eb0 con 0x55d16cbf4cc0 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.139+0000 7fbff7fff700 10 client.26753 handle_command_reply: tid=0 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.139+0000 7fc0327b9b80 1 client.26753 shutdown 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 msgr2=0x55d16cbe8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc014007ba0 tx=0x7fc014007f60 comp rx=0 tx=0).stop 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 shutdown_connections 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbe0c70 0x55d16cbf04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cbf4cc0 0x55d16cbf70d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe8f40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/1702101774 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdfe20 0x55d16cbe9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 >> 172.21.15.26:0/1702101774 conn(0x55d16cbd4da0 msgr2=0x55d16cbd98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fbff7fff700 0 client.26753 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:55.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 shutdown_connections 2023-07-13T22:21:55.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.140+0000 7fc0327b9b80 1 -- 172.21.15.26:0/1702101774 wait complete. 2023-07-13T22:21:55.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.141+0000 7fc0327b9b80 20 client.26753 trim_cache size 0 max 16384 2023-07-13T22:21:55.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.169+0000 7fc0327b9b80 1 Processor -- start 2023-07-13T22:21:55.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.169+0000 7fc0327b9b80 1 -- start start 2023-07-13T22:21:55.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbf0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbe8b10 con 0x55d16cbdfe20 2023-07-13T22:21:55.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbdb680 con 0x55d16cbe0c70 2023-07-13T22:21:55.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.170+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbe03a0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.171+0000 7fc01cb9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.171+0000 7fc01d39c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.172+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbf0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.172+0000 7fc00ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbf0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34100/0 (socket says 172.21.15.26:34100) 2023-07-13T22:21:55.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.172+0000 7fc00ffff700 1 -- 172.21.15.26:0/2330431358 learned_addr learned my addr 172.21.15.26:0/2330431358 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:55.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 -- 172.21.15.26:0/2330431358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 msgr2=0x55d16cbe3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 -- 172.21.15.26:0/2330431358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 msgr2=0x55d16cbf0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbf0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 -- 172.21.15.26:0/2330431358 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d16cbe1310 con 0x55d16cbdf4d0 2023-07-13T22:21:55.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.173+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe6fd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc014015520 tx=0x7fc014009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa912ba89750945b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.174+0000 7fbfff7fe700 1 -- 172.21.15.26:0/2330431358 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014015790 con 0x55d16cbdf4d0 2023-07-13T22:21:55.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.175+0000 7fbfff7fe700 1 -- 172.21.15.26:0/2330431358 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc01400f050 con 0x55d16cbdf4d0 2023-07-13T22:21:55.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.175+0000 7fbfff7fe700 1 -- 172.21.15.26:0/2330431358 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014015d70 con 0x55d16cbdf4d0 2023-07-13T22:21:55.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.176+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2330431358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 msgr2=0x55d16cbe6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.176+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe6fd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc014015520 tx=0x7fc014009450 comp rx=0 tx=0).stop 2023-07-13T22:21:55.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.177+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2330431358 shutdown_connections 2023-07-13T22:21:55.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.177+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbe0c70 0x55d16cbe3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.177+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbdf4d0 0x55d16cbe6fd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.177+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/2330431358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbf0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.177+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2330431358 >> 172.21.15.26:0/2330431358 conn(0x55d16cbd4da0 msgr2=0x55d16cbf6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.178+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2330431358 shutdown_connections 2023-07-13T22:21:55.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.178+0000 7fc0327b9b80 1 -- 172.21.15.26:0/2330431358 wait complete. 2023-07-13T22:21:55.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.178+0000 7fc0327b9b80 1 -- start start 2023-07-13T22:21:55.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 Processor -- start 2023-07-13T22:21:55.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbe0c70 0x55d16cbef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbefce0 con 0x55d16cbdfe20 2023-07-13T22:21:55.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbefe50 con 0x55d16cbdf4d0 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.179+0000 7fc0327b9b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16cbf0010 con 0x55d16cbe0c70 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.180+0000 7fc0327b9b80 10 client.0 _ll_register_callbacks cb 0x7fc014009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.180+0000 7fc01cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.180+0000 7fc01cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60592/0 (socket says 172.21.15.26:60592) 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.180+0000 7fc01cb9b700 1 -- 172.21.15.26:0/3920793124 learned_addr learned my addr 172.21.15.26:0/3920793124 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:55.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.180+0000 7fc01d39c700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbe0c70 0x55d16cbef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.182+0000 7fc00ffff700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 -- 172.21.15.26:0/3920793124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbe0c70 msgr2=0x55d16cbef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbe0c70 0x55d16cbef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 -- 172.21.15.26:0/3920793124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 msgr2=0x55d16cbec780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:55.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbec780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 -- 172.21.15.26:0/3920793124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d16c947c60 con 0x55d16cbdf4d0 2023-07-13T22:21:55.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.183+0000 7fc01cb9b700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc014007b70 tx=0x7fc014011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fee7a2943adea1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:55.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.186+0000 7fbff7fff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:55.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.186+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014015ab0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.186+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc014015c10 con 0x55d16cbdf4d0 2023-07-13T22:21:55.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.187+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc010004580 con 0x55d16cbdf4d0 2023-07-13T22:21:55.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.188+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc014015dd0 con 0x55d16cbdf4d0 2023-07-13T22:21:55.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.190+0000 7fc0327b9b80 10 client.26759 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:55.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.190+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d16c985250 con 0x55d16cbdf4d0 2023-07-13T22:21:55.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc010004580 con 0x55d16cbdf4d0 2023-07-13T22:21:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc014007c80 con 0x55d16cbdf4d0 2023-07-13T22:21:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:55.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 20 client.26759 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cc49800 0x55d16cc4bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 4 client.26759 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:21:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.201+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d16caec050 con 0x55d16cc49800 2023-07-13T22:21:55.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.202+0000 7fc00ffff700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cc49800 0x55d16cc4bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:55.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.204+0000 7fc00ffff700 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cc49800 0x55d16cc4bc10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992990148ac08c2 server_cookie=4e26dd49fa7a2c27 in_seq=0 out_seq=0 2023-07-13T22:21:55.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.204+0000 7fbff7fff700 10 client.26759 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:55.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.204+0000 7fbff7fff700 1 -- 172.21.15.26:0/3920793124 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d16caec050 con 0x55d16cc49800 2023-07-13T22:21:55.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.204+0000 7fbff7fff700 10 client.26759 handle_command_reply: tid=0 2023-07-13T22:21:55.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.204+0000 7fc0327b9b80 1 client.26759 shutdown 2023-07-13T22:21:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 msgr2=0x55d16cbec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc014007b70 tx=0x7fc014011e60 comp rx=0 tx=0).stop 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 shutdown_connections 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d16cc49800 0x55d16cc4bc10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d16cbe0c70 0x55d16cbef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d16cbdf4d0 0x55d16cbec240 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc014007b70 tx=0x7fc014011e60 comp rx=0 tx=0).stop 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 --2- 172.21.15.26:0/3920793124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d16cbdfe20 0x55d16cbec780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 >> 172.21.15.26:0/3920793124 conn(0x55d16cbd4da0 msgr2=0x55d16cbe6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.211+0000 7fbff7fff700 0 client.26759 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:55.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.214+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 shutdown_connections 2023-07-13T22:21:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.214+0000 7fc0327b9b80 1 -- 172.21.15.26:0/3920793124 wait complete. 2023-07-13T22:21:55.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.215+0000 7fc0327b9b80 20 client.26759 trim_cache size 0 max 16384 2023-07-13T22:21:55.210 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:21:55.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.234+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 msgr2=0x7fc0000a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.234+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 0x7fc0000a3750 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc01000bfd0 tx=0x7fc01000fac0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.234+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 msgr2=0x7fc01806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:55.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.234+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc00800e9a0 tx=0x7fc00800ed60 comp rx=0 tx=0).stop 2023-07-13T22:21:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 shutdown_connections 2023-07-13T22:21:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc0000a12a0 0x7fc0000a3750 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0180fe350 0x7fc01806ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc018103670 0x7fc0181abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 --2- 172.21.15.26:0/2862886644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc018100cb0 0x7fc01806f4c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.246+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 >> 172.21.15.26:0/2862886644 conn(0x7fc0180fa000 msgr2=0x7fc0180feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.248+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 shutdown_connections 2023-07-13T22:21:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:55.249+0000 7fc01edff700 1 -- 172.21.15.26:0/2862886644 wait complete. 2023-07-13T22:21:56.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:21:56.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:21:56.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.995+0000 7fbc91fff700 1 -- 172.21.15.26:0/3293091320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 msgr2=0x7fbc8c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.995+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3293091320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c10e1f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b0a0 tx=0x7fbc8400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.996+0000 7fbc91fff700 1 -- 172.21.15.26:0/3293091320 shutdown_connections 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.996+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3293091320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c10e1f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.996+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3293091320 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c070ee0 0x7fbc8c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.996+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3293091320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c071830 0x7fbc8c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:56.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:56.996+0000 7fbc91fff700 1 -- 172.21.15.26:0/3293091320 >> 172.21.15.26:0/3293091320 conn(0x7fbc8c06ca90 msgr2=0x7fbc8c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.018+0000 7fbc91fff700 1 -- 172.21.15.26:0/3293091320 shutdown_connections 2023-07-13T22:21:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.019+0000 7fbc91fff700 1 -- 172.21.15.26:0/3293091320 wait complete. 2023-07-13T22:21:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.021+0000 7fbc91fff700 1 Processor -- start 2023-07-13T22:21:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.021+0000 7fbc91fff700 1 -- start start 2023-07-13T22:21:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c070ee0 0x7fbc8c114cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 0x7fbc8c115230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c116840 con 0x7fbc8c070ee0 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c1169b0 con 0x7fbc8c071830 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.022+0000 7fbc91fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c116b70 con 0x7fbc8c10bd60 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.023+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 0x7fbc8c115230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.023+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 0x7fbc8c115230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60610/0 (socket says 172.21.15.26:60610) 2023-07-13T22:21:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.023+0000 7fbc8affd700 1 -- 172.21.15.26:0/3386646173 learned_addr learned my addr 172.21.15.26:0/3386646173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.023+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.024+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c070ee0 0x7fbc8c114cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.025+0000 7fbc8bfff700 1 -- 172.21.15.26:0/3386646173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 msgr2=0x7fbc8c115230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.025+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 0x7fbc8c115230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.025+0000 7fbc8bfff700 1 -- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c070ee0 msgr2=0x7fbc8c114cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.025+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c070ee0 0x7fbc8c114cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.025+0000 7fbc8bfff700 1 -- 172.21.15.26:0/3386646173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84009d00 con 0x7fbc8c10bd60 2023-07-13T22:21:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.027+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c116270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b070 tx=0x7fbc8400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.034+0000 7fbc88ff9700 1 -- 172.21.15.26:0/3386646173 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400e040 con 0x7fbc8c10bd60 2023-07-13T22:21:57.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.034+0000 7fbc88ff9700 1 -- 172.21.15.26:0/3386646173 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc84004e60 con 0x7fbc8c10bd60 2023-07-13T22:21:57.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.034+0000 7fbc88ff9700 1 -- 172.21.15.26:0/3386646173 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8401b750 con 0x7fbc8c10bd60 2023-07-13T22:21:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.037+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c116d90 con 0x7fbc8c10bd60 2023-07-13T22:21:57.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.037+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c1171c0 con 0x7fbc8c10bd60 2023-07-13T22:21:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.045+0000 7fbca58d6b80 1 Processor -- start 2023-07-13T22:21:57.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.049+0000 7fbc88ff9700 1 -- 172.21.15.26:0/3386646173 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc8401da30 con 0x7fbc8c10bd60 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.049+0000 7fbca58d6b80 1 -- start start 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.049+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 0x561306879960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 0x56130687a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130677ed30 con 0x561306879ea0 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613067d1ac0 con 0x561306879550 2023-07-13T22:21:57.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613067caf50 con 0x56130687acf0 2023-07-13T22:21:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 0x561306879960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 0x561306879960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60620/0 (socket says 172.21.15.26:60620) 2023-07-13T22:21:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbc8affd700 1 -- 172.21.15.26:0/17164236 learned_addr learned my addr 172.21.15.26:0/17164236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:57.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.050+0000 7fbc8affd700 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.052+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 0x56130687a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.053+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/17164236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 msgr2=0x56130687b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.053+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.053+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/17164236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 msgr2=0x561306879960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.053+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 0x561306879960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.053+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/17164236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613065f8e60 con 0x561306879ea0 2023-07-13T22:21:57.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.054+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 0x56130687a760 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009c00 tx=0x7fbc7c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71065e156dfdeee5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.054+0000 7fbc88ff9700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 0x7fbc740a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.054+0000 7fbc71ffb700 1 -- 172.21.15.26:0/17164236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c011b70 con 0x561306879ea0 2023-07-13T22:21:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.055+0000 7fbc71ffb700 1 -- 172.21.15.26:0/17164236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7c011cd0 con 0x561306879ea0 2023-07-13T22:21:57.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.055+0000 7fbc71ffb700 1 -- 172.21.15.26:0/17164236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00f6a0 con 0x561306879ea0 2023-07-13T22:21:57.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.055+0000 7fbc88ff9700 1 -- 172.21.15.26:0/3386646173 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc840cd330 con 0x7fbc8c10bd60 2023-07-13T22:21:57.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.058+0000 7fbc8affd700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 0x7fbc740a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.059+0000 7fbca58d6b80 1 -- 172.21.15.26:0/17164236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 msgr2=0x56130687a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.059+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 0x56130687a760 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009c00 tx=0x7fbc7c009f10 comp rx=0 tx=0).stop 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.060+0000 7fbca58d6b80 1 -- 172.21.15.26:0/17164236 shutdown_connections 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.060+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.060+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879550 0x561306879960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.060+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/17164236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561306879ea0 0x56130687a760 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.060+0000 7fbca58d6b80 1 -- 172.21.15.26:0/17164236 >> 172.21.15.26:0/17164236 conn(0x56130686ee20 msgr2=0x561306874250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.061+0000 7fbc8affd700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 0x7fbc740a3800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b290 tx=0x7fbc8000b930 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.066+0000 7fbca58d6b80 1 -- 172.21.15.26:0/17164236 shutdown_connections 2023-07-13T22:21:57.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.066+0000 7fbca58d6b80 1 -- 172.21.15.26:0/17164236 wait complete. 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.067+0000 7fbca58d6b80 1 -- start start 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.067+0000 7fbca58d6b80 1 Processor -- start 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x561306883720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x56130688a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130676ab40 con 0x56130687acf0 2023-07-13T22:21:57.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130676a840 con 0x561306879ea0 2023-07-13T22:21:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130664e9a0 con 0x561306879550 2023-07-13T22:21:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.068+0000 7fbca58d6b80 10 client.0 _ll_register_callbacks cb 0x5613067687e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.069+0000 7fbc8b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.069+0000 7fbc8b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55354/0 (socket says 172.21.15.26:55354) 2023-07-13T22:21:57.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.069+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/1110159653 learned_addr learned my addr 172.21.15.26:0/1110159653 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.070+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x56130688a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.070+0000 7fbc8affd700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x561306883720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 msgr2=0x561306883720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x561306883720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/1110159653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 msgr2=0x56130688a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x56130688a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/1110159653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c009830 con 0x561306879550 2023-07-13T22:21:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.071+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01c040 tx=0x7fbc7c012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=666964ce090e3ed7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.072+0000 7fbc6affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:21:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.072+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c005ce0 con 0x561306879550 2023-07-13T22:21:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.072+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc7c005030 con 0x561306879550 2023-07-13T22:21:57.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.072+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561306724570 con 0x561306879550 2023-07-13T22:21:57.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.077+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc7c005210 con 0x561306879550 2023-07-13T22:21:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.077+0000 7fbca58d6b80 10 client.26773 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:57.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.077+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5613065e1ce0 con 0x561306879550 2023-07-13T22:21:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561306724570 con 0x561306879550 2023-07-13T22:21:57.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc7c022040 con 0x561306879550 2023-07-13T22:21:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 10 client.26773 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 10 client.26773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 20 client.26773 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56130688ee90 0x5613068912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 4 client.26773 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:57.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.078+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613067c3f30 con 0x56130688ee90 2023-07-13T22:21:57.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.079+0000 7fbc8affd700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56130688ee90 0x5613068912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.079+0000 7fbc8affd700 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56130688ee90 0x5613068912a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15551fd2b9e48387 server_cookie=c801f0784ea04855 in_seq=0 out_seq=0 2023-07-13T22:21:57.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.079+0000 7fbc6affd700 10 client.26773 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:57.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.081+0000 7fbc6affd700 1 -- 172.21.15.26:0/1110159653 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5613067c3f30 con 0x56130688ee90 2023-07-13T22:21:57.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.081+0000 7fbc6affd700 10 client.26773 handle_command_reply: tid=0 2023-07-13T22:21:57.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.081+0000 7fbca58d6b80 1 client.26773 shutdown 2023-07-13T22:21:57.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.081+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 msgr2=0x5613068831e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.081+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01c040 tx=0x7fbc7c012680 comp rx=0 tx=0).stop 2023-07-13T22:21:57.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 shutdown_connections 2023-07-13T22:21:57.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x56130688a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56130688ee90 0x5613068912a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879550 0x5613068831e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/1110159653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x561306883720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 >> 172.21.15.26:0/1110159653 conn(0x56130686ee20 msgr2=0x561306873930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbc6affd700 0 client.26773 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:57.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 shutdown_connections 2023-07-13T22:21:57.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.082+0000 7fbca58d6b80 1 -- 172.21.15.26:0/1110159653 wait complete. 2023-07-13T22:21:57.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.084+0000 7fbca58d6b80 20 client.26773 trim_cache size 0 max 16384 2023-07-13T22:21:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.137+0000 7fbca58d6b80 1 Processor -- start 2023-07-13T22:21:57.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 -- start start 2023-07-13T22:21:57.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130688a720 0x561306882f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130687b5e0 con 0x56130688a720 2023-07-13T22:21:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130687a270 con 0x561306879ea0 2023-07-13T22:21:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.138+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x561306875d80 con 0x56130687acf0 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60632/0 (socket says 172.21.15.26:60632) 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/910004444 learned_addr learned my addr 172.21.15.26:0/910004444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8affd700 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/910004444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 msgr2=0x56130687e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/910004444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130688a720 msgr2=0x561306882f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130688a720 0x561306882f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.139+0000 7fbc8b7fe700 1 -- 172.21.15.26:0/910004444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56130661f2d0 con 0x561306879ea0 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.140+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009f50 tx=0x7fbc7c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d411cae467fafd92 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.140+0000 7fbc727fc700 1 -- 172.21.15.26:0/910004444 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c024420 con 0x561306879ea0 2023-07-13T22:21:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.140+0000 7fbc727fc700 1 -- 172.21.15.26:0/910004444 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7c009260 con 0x561306879ea0 2023-07-13T22:21:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.140+0000 7fbc727fc700 1 -- 172.21.15.26:0/910004444 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c005370 con 0x561306879ea0 2023-07-13T22:21:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 -- 172.21.15.26:0/910004444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 msgr2=0x56130687dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c009f50 tx=0x7fbc7c009b00 comp rx=0 tx=0).stop 2023-07-13T22:21:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 -- 172.21.15.26:0/910004444 shutdown_connections 2023-07-13T22:21:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130688a720 0x561306882f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561306879ea0 0x56130687dc40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/910004444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56130687acf0 0x56130687e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 -- 172.21.15.26:0/910004444 >> 172.21.15.26:0/910004444 conn(0x56130686ee20 msgr2=0x561306879550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.142+0000 7fbca58d6b80 1 -- 172.21.15.26:0/910004444 shutdown_connections 2023-07-13T22:21:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.143+0000 7fbca58d6b80 1 -- 172.21.15.26:0/910004444 wait complete. 2023-07-13T22:21:57.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.143+0000 7fbca58d6b80 1 -- start start 2023-07-13T22:21:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.144+0000 7fbca58d6b80 1 Processor -- start 2023-07-13T22:21:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879ea0 0x56130688a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x561306886ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613068878f0 con 0x56130687acf0 2023-07-13T22:21:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613068e2d40 con 0x56130688a720 2023-07-13T22:21:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613068e2eb0 con 0x561306879ea0 2023-07-13T22:21:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.145+0000 7fbca58d6b80 10 client.0 _ll_register_callbacks cb 0x7fbc8000bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.147+0000 7fbc8affd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x561306886ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.147+0000 7fbc8bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.147+0000 7fbc8bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60648/0 (socket says 172.21.15.26:60648) 2023-07-13T22:21:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.147+0000 7fbc8bfff700 1 -- 172.21.15.26:0/2855091237 learned_addr learned my addr 172.21.15.26:0/2855091237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:57.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.147+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879ea0 0x56130688a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.148+0000 7fbc8bfff700 1 -- 172.21.15.26:0/2855091237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879ea0 msgr2=0x56130688a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.148+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879ea0 0x56130688a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.148+0000 7fbc8bfff700 1 -- 172.21.15.26:0/2855091237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 msgr2=0x561306886ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.148+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x561306886ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.148+0000 7fbc8bfff700 1 -- 172.21.15.26:0/2855091237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613065e1ce0 con 0x56130688a720 2023-07-13T22:21:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.149+0000 7fbc8bfff700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009fd0 tx=0x7fbc8409a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b9b250cbef7707 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.149+0000 7fbc6affd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.149+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc840a2c00 con 0x56130688a720 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.149+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fbc840ba0b0 con 0x56130688a720 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.149+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc7c011cd0 con 0x56130688a720 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.150+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc8409aaa0 con 0x56130688a720 2023-07-13T22:21:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.150+0000 7fbca58d6b80 10 client.26777 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.150+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5613065f7330 con 0x56130688a720 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.150+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc7c011cd0 con 0x56130688a720 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbc840a2770 con 0x56130688a720 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 10 client.26777 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 20 client.26777 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5613068e4660 0x5613068e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 4 client.26777 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5613067860d0 con 0x5613068e4660 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.152+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5613068e4660 0x5613068e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.153+0000 7fbc8b7fe700 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5613068e4660 0x5613068e6a40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da075e58cb6d478 server_cookie=f7b44b27dc9701c8 in_seq=0 out_seq=0 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.153+0000 7fbc6affd700 10 client.26777 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.154+0000 7fbc6affd700 1 -- 172.21.15.26:0/2855091237 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5613067860d0 con 0x5613068e4660 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.154+0000 7fbc6affd700 10 client.26777 handle_command_reply: tid=0 2023-07-13T22:21:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.154+0000 7fbca58d6b80 1 client.26777 shutdown 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 msgr2=0x5613068871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009fd0 tx=0x7fbc8409a3d0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 shutdown_connections 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5613068e4660 0x5613068e6a40 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56130688a720 0x5613068871e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561306879ea0 0x56130688a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 --2- 172.21.15.26:0/2855091237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56130687acf0 0x561306886ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 >> 172.21.15.26:0/2855091237 conn(0x56130686ee20 msgr2=0x56130687dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.155+0000 7fbc6affd700 0 client.26777 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.156+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 shutdown_connections 2023-07-13T22:21:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.156+0000 7fbca58d6b80 1 -- 172.21.15.26:0/2855091237 wait complete. 2023-07-13T22:21:57.145 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:21:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "5cbf14a6-db63-4b60-9689-934c22d6f39c", 2023-07-13T22:21:57.146 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:21:57.146 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.156+0000 7fbca58d6b80 20 client.26777 trim_cache size 0 max 16384 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 msgr2=0x7fbc740a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 0x7fbc740a3800 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000b290 tx=0x7fbc8000b930 comp rx=0 tx=0).stop 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 msgr2=0x7fbc8c116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c116270 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b070 tx=0x7fbc8400bce0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 shutdown_connections 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fbc740a1350 0x7fbc740a3800 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc8c070ee0 0x7fbc8c114cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbc8c10bd60 0x7fbc8c116270 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 --2- 172.21.15.26:0/3386646173 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc8c071830 0x7fbc8c115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.157+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 >> 172.21.15.26:0/3386646173 conn(0x7fbc8c06ca90 msgr2=0x7fbc8c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.158+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 shutdown_connections 2023-07-13T22:21:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:57.158+0000 7fbc91fff700 1 -- 172.21.15.26:0/3386646173 wait complete. 2023-07-13T22:21:57.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:21:57.453 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:57 smithi026 ceph-mon[95647]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 265 MiB/s wr, 81 op/s 2023-07-13T22:21:57.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:57 smithi117 ceph-mon[129074]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 265 MiB/s wr, 81 op/s 2023-07-13T22:21:57.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:57 smithi121 ceph-mon[129187]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 265 MiB/s wr, 81 op/s 2023-07-13T22:21:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.036+0000 7effbdfff700 1 -- 172.21.15.26:0/2310207547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 msgr2=0x7effb810e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.036+0000 7effbdfff700 1 --2- 172.21.15.26:0/2310207547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb810e1f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7effa8009990 tx=0x7effa8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.046+0000 7effbdfff700 1 -- 172.21.15.26:0/2310207547 shutdown_connections 2023-07-13T22:21:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.046+0000 7effbdfff700 1 --2- 172.21.15.26:0/2310207547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb810e1f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.046+0000 7effbdfff700 1 --2- 172.21.15.26:0/2310207547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8070ee0 0x7effb80712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.046+0000 7effbdfff700 1 --2- 172.21.15.26:0/2310207547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.046+0000 7effbdfff700 1 -- 172.21.15.26:0/2310207547 >> 172.21.15.26:0/2310207547 conn(0x7effb806ca90 msgr2=0x7effb806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.047+0000 7effbdfff700 1 -- 172.21.15.26:0/2310207547 shutdown_connections 2023-07-13T22:21:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.048+0000 7effbdfff700 1 -- 172.21.15.26:0/2310207547 wait complete. 2023-07-13T22:21:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.049+0000 7effbdfff700 1 Processor -- start 2023-07-13T22:21:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 -- start start 2023-07-13T22:21:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8070ee0 0x7effb8113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb8113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb81b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81170d0 con 0x7effb810bd60 2023-07-13T22:21:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb8117240 con 0x7effb8070ee0 2023-07-13T22:21:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.050+0000 7effbdfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb81b4ec0 con 0x7effb8071830 2023-07-13T22:21:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.051+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb8113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.051+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb8113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55394/0 (socket says 172.21.15.26:55394) 2023-07-13T22:21:58.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.051+0000 7effb6ffd700 1 -- 172.21.15.26:0/1286546965 learned_addr learned my addr 172.21.15.26:0/1286546965 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.051+0000 7effb7fff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb81b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 -- 172.21.15.26:0/1286546965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 msgr2=0x7effb8113ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb8113ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 -- 172.21.15.26:0/1286546965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8070ee0 msgr2=0x7effb8113ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:21:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8070ee0 0x7effb8113ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 -- 172.21.15.26:0/1286546965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80095f0 con 0x7effb810bd60 2023-07-13T22:21:58.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.052+0000 7effb7fff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb81b4980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7effa80098e0 tx=0x7effa8003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.053+0000 7effb4ff9700 1 -- 172.21.15.26:0/1286546965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa801d070 con 0x7effb810bd60 2023-07-13T22:21:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.053+0000 7effb4ff9700 1 -- 172.21.15.26:0/1286546965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa800f460 con 0x7effb810bd60 2023-07-13T22:21:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.053+0000 7effb4ff9700 1 -- 172.21.15.26:0/1286546965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8017650 con 0x7effb810bd60 2023-07-13T22:21:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.054+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb81b50c0 con 0x7effb810bd60 2023-07-13T22:21:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.054+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb81b5510 con 0x7effb810bd60 2023-07-13T22:21:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.061+0000 7effd1919b80 1 Processor -- start 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 -- start start 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bcf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcfc70 0x556059bd00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059ad3cb0 con 0x556059bcfc70 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059b26a40 con 0x556059bcee20 2023-07-13T22:21:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.065+0000 7effd1919b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059b1fed0 con 0x556059bce4d0 2023-07-13T22:21:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.066+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.066+0000 7effb77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bcf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.066+0000 7effb77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bcf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60674/0 (socket says 172.21.15.26:60674) 2023-07-13T22:21:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.066+0000 7effb77fe700 1 -- 172.21.15.26:0/659008612 learned_addr learned my addr 172.21.15.26:0/659008612 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:58.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.066+0000 7effb4ff9700 1 -- 172.21.15.26:0/1286546965 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7effa8005390 con 0x7effb810bd60 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.068+0000 7effb6ffd700 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcfc70 0x556059bd00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 -- 172.21.15.26:0/659008612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 msgr2=0x556059bcf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bcf6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 -- 172.21.15.26:0/659008612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcfc70 msgr2=0x556059bd00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcfc70 0x556059bd00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 -- 172.21.15.26:0/659008612 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55605994dde0 con 0x556059bce4d0 2023-07-13T22:21:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.069+0000 7effb6ffd700 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bce8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7effac0153e0 tx=0x7effac009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5caac3bd2b87f2ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.071+0000 7effb4ff9700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 0x7effa00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.072+0000 7effb4ff9700 1 -- 172.21.15.26:0/1286546965 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effa80cda00 con 0x7effb810bd60 2023-07-13T22:21:58.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.074+0000 7effb77fe700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 0x7effa00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.075+0000 7eff9dffb700 1 -- 172.21.15.26:0/659008612 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac015e80 con 0x556059bce4d0 2023-07-13T22:21:58.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.075+0000 7eff9dffb700 1 -- 172.21.15.26:0/659008612 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effac017930 con 0x556059bce4d0 2023-07-13T22:21:58.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.075+0000 7eff9dffb700 1 -- 172.21.15.26:0/659008612 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0138b0 con 0x556059bce4d0 2023-07-13T22:21:58.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effb77fe700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 0x7effa00a3800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7effb0009f10 tx=0x7effb0009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 -- 172.21.15.26:0/659008612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 msgr2=0x556059bce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bce8e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7effac0153e0 tx=0x7effac009570 comp rx=0 tx=0).stop 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 -- 172.21.15.26:0/659008612 shutdown_connections 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcfc70 0x556059bd00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bce8e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 --2- 172.21.15.26:0/659008612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bcf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.078+0000 7effd1919b80 1 -- 172.21.15.26:0/659008612 >> 172.21.15.26:0/659008612 conn(0x556059bc3da0 msgr2=0x556059bc91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.079+0000 7effd1919b80 1 -- 172.21.15.26:0/659008612 shutdown_connections 2023-07-13T22:21:58.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.079+0000 7effd1919b80 1 -- 172.21.15.26:0/659008612 wait complete. 2023-07-13T22:21:58.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.080+0000 7effd1919b80 1 -- start start 2023-07-13T22:21:58.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.080+0000 7effd1919b80 1 Processor -- start 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bd8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bdf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059abfac0 con 0x556059bcee20 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059abf7c0 con 0x556059bcfc70 2023-07-13T22:21:58.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560599a3920 con 0x556059bce4d0 2023-07-13T22:21:58.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.081+0000 7effd1919b80 10 client.0 _ll_register_callbacks cb 0x556059abd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:58.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb77fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bd8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34150/0 (socket says 172.21.15.26:34150) 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 -- 172.21.15.26:0/45284280 learned_addr learned my addr 172.21.15.26:0/45284280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 -- 172.21.15.26:0/45284280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 msgr2=0x556059bd8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bd8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 -- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 msgr2=0x556059bdf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bdf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.083+0000 7effb6ffd700 1 -- 172.21.15.26:0/45284280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac009d10 con 0x556059bcee20 2023-07-13T22:21:58.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.084+0000 7effb6ffd700 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7effac0094c0 tx=0x7effac017d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece151cbc6008c70 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.085+0000 7eff96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:58.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.085+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0135f0 con 0x556059bcee20 2023-07-13T22:21:58.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.085+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effac013750 con 0x556059bcee20 2023-07-13T22:21:58.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.085+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556059a794f0 con 0x556059bcee20 2023-07-13T22:21:58.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.086+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7effac013950 con 0x556059bcee20 2023-07-13T22:21:58.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.086+0000 7effd1919b80 10 client.16350 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:58.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.086+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556059936c60 con 0x556059bcee20 2023-07-13T22:21:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556059a794f0 con 0x556059bcee20 2023-07-13T22:21:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7effac020ba0 con 0x556059bcee20 2023-07-13T22:21:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:58.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 20 client.16350 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059be3e70 0x556059be62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 4 client.16350 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:21:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.087+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556059b18eb0 con 0x556059be3e70 2023-07-13T22:21:58.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.088+0000 7effb77fe700 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059be3e70 0x556059be62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.090+0000 7effb77fe700 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059be3e70 0x556059be62e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f300c085a0c9812 server_cookie=a3077631f1a63339 in_seq=0 out_seq=0 2023-07-13T22:21:58.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.090+0000 7eff96ffd700 10 client.16350 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.090+0000 7eff96ffd700 1 -- 172.21.15.26:0/45284280 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556059b18eb0 con 0x556059be3e70 2023-07-13T22:21:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.091+0000 7eff96ffd700 10 client.16350 handle_command_reply: tid=0 2023-07-13T22:21:58.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.091+0000 7effd1919b80 1 client.16350 shutdown 2023-07-13T22:21:58.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.092+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 msgr2=0x556059bd8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.092+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7effac0094c0 tx=0x7effac017d80 comp rx=0 tx=0).stop 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 shutdown_connections 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059be3e70 0x556059be62e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bdf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059bd8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 --2- 172.21.15.26:0/45284280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bd8740 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 >> 172.21.15.26:0/45284280 conn(0x556059bc3da0 msgr2=0x556059bc88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.096+0000 7eff96ffd700 0 client.16350 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:58.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.097+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 shutdown_connections 2023-07-13T22:21:58.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.097+0000 7effd1919b80 1 -- 172.21.15.26:0/45284280 wait complete. 2023-07-13T22:21:58.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.098+0000 7effd1919b80 20 client.16350 trim_cache size 0 max 16384 2023-07-13T22:21:58.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.118+0000 7effd1919b80 1 Processor -- start 2023-07-13T22:21:58.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 -- start start 2023-07-13T22:21:58.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059be5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bd2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bd7dd0 con 0x556059bcee20 2023-07-13T22:21:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bc5ef0 con 0x556059bcfc70 2023-07-13T22:21:58.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.119+0000 7effd1919b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bcf440 con 0x556059bce4d0 2023-07-13T22:21:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.121+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.121+0000 7effb6ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34158/0 (socket says 172.21.15.26:34158) 2023-07-13T22:21:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.121+0000 7effb6ffd700 1 -- 172.21.15.26:0/861775720 learned_addr learned my addr 172.21.15.26:0/861775720 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:58.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb7fff700 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bd2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb77fe700 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059be5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb6ffd700 1 -- 172.21.15.26:0/861775720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 msgr2=0x556059be5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb6ffd700 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059be5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb6ffd700 1 -- 172.21.15.26:0/861775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 msgr2=0x556059bd2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb6ffd700 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bd2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.122+0000 7effb6ffd700 1 -- 172.21.15.26:0/861775720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556059bd0310 con 0x556059bcee20 2023-07-13T22:21:58.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.123+0000 7effb6ffd700 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7effac0092f0 tx=0x7effac015a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77de9e9d92242d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.124+0000 7eff9e7fc700 1 -- 172.21.15.26:0/861775720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac017ea0 con 0x556059bcee20 2023-07-13T22:21:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.124+0000 7eff9e7fc700 1 -- 172.21.15.26:0/861775720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effac01f4d0 con 0x556059bcee20 2023-07-13T22:21:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.124+0000 7eff9e7fc700 1 -- 172.21.15.26:0/861775720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac020840 con 0x556059bcee20 2023-07-13T22:21:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.125+0000 7effd1919b80 1 -- 172.21.15.26:0/861775720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 msgr2=0x556059bdf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.125+0000 7effd1919b80 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7effac0092f0 tx=0x7effac015a80 comp rx=0 tx=0).stop 2023-07-13T22:21:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.136+0000 7effd1919b80 1 -- 172.21.15.26:0/861775720 shutdown_connections 2023-07-13T22:21:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.136+0000 7effd1919b80 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcfc70 0x556059bd2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.136+0000 7effd1919b80 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bce4d0 0x556059be5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.136+0000 7effd1919b80 1 --2- 172.21.15.26:0/861775720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bcee20 0x556059bdf740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.136+0000 7effd1919b80 1 -- 172.21.15.26:0/861775720 >> 172.21.15.26:0/861775720 conn(0x556059bc3da0 msgr2=0x556059bd6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.145+0000 7effd1919b80 1 -- 172.21.15.26:0/861775720 shutdown_connections 2023-07-13T22:21:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.145+0000 7effd1919b80 1 -- 172.21.15.26:0/861775720 wait complete. 2023-07-13T22:21:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.146+0000 7effd1919b80 1 -- start start 2023-07-13T22:21:58.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.146+0000 7effd1919b80 1 Processor -- start 2023-07-13T22:21:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bdd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bcfc70 0x556059bde560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bdeaa0 con 0x556059bce4d0 2023-07-13T22:21:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bdec10 con 0x556059bcee20 2023-07-13T22:21:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556059bdedd0 con 0x556059bcfc70 2023-07-13T22:21:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.147+0000 7effd1919b80 10 client.0 _ll_register_callbacks cb 0x7effac009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:21:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.152+0000 7effb77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.152+0000 7effb77fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34166/0 (socket says 172.21.15.26:34166) 2023-07-13T22:21:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.152+0000 7effb77fe700 1 -- 172.21.15.26:0/79080684 learned_addr learned my addr 172.21.15.26:0/79080684 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:21:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.159+0000 7effb7fff700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bcfc70 0x556059bde560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.159+0000 7effb6ffd700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bdd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.171+0000 7effb77fe700 1 -- 172.21.15.26:0/79080684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bcfc70 msgr2=0x556059bde560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.171+0000 7effb77fe700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bcfc70 0x556059bde560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.171+0000 7effb77fe700 1 -- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 msgr2=0x556059bdd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:21:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.171+0000 7effb77fe700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bdd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.171+0000 7effb77fe700 1 -- 172.21.15.26:0/79080684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556059936c60 con 0x556059bce4d0 2023-07-13T22:21:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.186+0000 7effb77fe700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7effb0007e00 tx=0x7effb0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d8634a3f5e4abf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:21:58.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.187+0000 7eff96ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:21:58.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.198+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00071b0 con 0x556059bce4d0 2023-07-13T22:21:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.198+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7effb0023400 con 0x556059bce4d0 2023-07-13T22:21:58.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.208+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effac017930 con 0x556059bce4d0 2023-07-13T22:21:58.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.233+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7effb0023600 con 0x556059bce4d0 2023-07-13T22:21:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.236+0000 7effd1919b80 10 client.16356 fetch_fsmap learned FSMap version 29 2023-07-13T22:21:58.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.236+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556059974250 con 0x556059bce4d0 2023-07-13T22:21:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effac017930 con 0x556059bce4d0 2023-07-13T22:21:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7effb0004190 con 0x556059bce4d0 2023-07-13T22:21:58.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:21:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:21:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 20 client.16356 populate_metadata read hostname 'smithi026' 2023-07-13T22:21:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059c38920 0x556059c3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:21:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 4 client.16356 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:21:58.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.262+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556059adb050 con 0x556059c38920 2023-07-13T22:21:58.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.275+0000 7effb6ffd700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059c38920 0x556059c3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:21:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.299+0000 7effb6ffd700 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059c38920 0x556059c3ad30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e27e450995d78bb5 server_cookie=c02d2ecdac060584 in_seq=0 out_seq=0 2023-07-13T22:21:58.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.300+0000 7eff96ffd700 10 client.16356 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.300+0000 7eff96ffd700 1 -- 172.21.15.26:0/79080684 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556059adb050 con 0x556059c38920 2023-07-13T22:21:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.300+0000 7eff96ffd700 10 client.16356 handle_command_reply: tid=0 2023-07-13T22:21:58.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.300+0000 7effd1919b80 1 client.16356 shutdown 2023-07-13T22:21:58.351 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "5cbf14a6-db63-4b60-9689-934c22d6f39c": { 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "5cbf14a6-db63-4b60-9689-934c22d6f39c", 2023-07-13T22:21:58.352 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:21:58.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:21:58.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:21:58.353 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:21:58.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.306+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 msgr2=0x556059bdd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.306+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7effb0007e00 tx=0x7effb0011000 comp rx=0 tx=0).stop 2023-07-13T22:21:58.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 shutdown_connections 2023-07-13T22:21:58.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556059bce4d0 0x556059bdd010 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556059c38920 0x556059c3ad30 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556059bcfc70 0x556059bde560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 --2- 172.21.15.26:0/79080684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556059bcee20 0x556059bdd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 >> 172.21.15.26:0/79080684 conn(0x556059bc3da0 msgr2=0x556059be5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7eff96ffd700 0 client.16356 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:21:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 shutdown_connections 2023-07-13T22:21:58.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.317+0000 7effd1919b80 1 -- 172.21.15.26:0/79080684 wait complete. 2023-07-13T22:21:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.366+0000 7effd1919b80 20 client.16356 trim_cache size 0 max 16384 2023-07-13T22:21:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.367+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 msgr2=0x7effa00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.367+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 0x7effa00a3800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7effb0009f10 tx=0x7effb0009510 comp rx=0 tx=0).stop 2023-07-13T22:21:58.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.367+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 msgr2=0x7effb81b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:21:58.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.367+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb81b4980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7effa80098e0 tx=0x7effa8003f00 comp rx=0 tx=0).stop 2023-07-13T22:21:58.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 shutdown_connections 2023-07-13T22:21:58.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7effa00a1350 0x7effa00a3800 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effb810bd60 0x7effb81b4980 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effb8070ee0 0x7effb8113ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 --2- 172.21.15.26:0/1286546965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effb8071830 0x7effb8113ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:21:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 >> 172.21.15.26:0/1286546965 conn(0x7effb806ca90 msgr2=0x7effb806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:21:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 shutdown_connections 2023-07-13T22:21:58.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:21:58.368+0000 7effbdfff700 1 -- 172.21.15.26:0/1286546965 wait complete. 2023-07-13T22:21:58.449 INFO:tasks.cephfs.filesystem:scrub status for tag:5cbf14a6-db63-4b60-9689-934c22d6f39c - {'path': '/', 'tag': '5cbf14a6-db63-4b60-9689-934c22d6f39c', 'options': 'recursive,force'} 2023-07-13T22:21:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:21:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:21:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:21:58.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:58 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:58 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:21:58.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:21:58.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:21:58.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:58.739 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:58 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:21:59.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:21:59 smithi117 ceph-mon[129074]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:21:59.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:21:59 smithi026 ceph-mon[95647]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:21:59.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:21:59 smithi121 ceph-mon[129187]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 283 MiB/s wr, 91 op/s 2023-07-13T22:22:01.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:01 smithi117 ceph-mon[129074]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 308 MiB/s wr, 96 op/s 2023-07-13T22:22:01.733 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:01 smithi026 ceph-mon[95647]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 308 MiB/s wr, 96 op/s 2023-07-13T22:22:01.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:01 smithi121 ceph-mon[129187]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 308 MiB/s wr, 96 op/s 2023-07-13T22:22:02.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:02 smithi117 ceph-mon[129074]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-07-13T22:22:02.711 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:02 smithi026 ceph-mon[95647]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-07-13T22:22:02.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:02 smithi121 ceph-mon[129187]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 275 MiB/s wr, 90 op/s 2023-07-13T22:22:05.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:05 smithi117 ceph-mon[129074]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:22:05.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:05 smithi026 ceph-mon[95647]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:22:05.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:05 smithi121 ceph-mon[129187]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 321 MiB/s wr, 102 op/s 2023-07-13T22:22:06.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:06 smithi117 ceph-mon[129074]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:22:06.704 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:06 smithi026 ceph-mon[95647]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:22:06.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:06 smithi121 ceph-mon[129187]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 292 MiB/s wr, 93 op/s 2023-07-13T22:22:07.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:07 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:22:07.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:07 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:22:07.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:07 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:22:08.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:08 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:22:08.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:08 smithi117 ceph-mon[129074]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 308 MiB/s wr, 103 op/s 2023-07-13T22:22:08.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:08 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:22:08.698 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:08 smithi026 ceph-mon[95647]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 308 MiB/s wr, 103 op/s 2023-07-13T22:22:08.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:08 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:22:08.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:08 smithi121 ceph-mon[129187]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 308 MiB/s wr, 103 op/s 2023-07-13T22:22:11.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:11 smithi117 ceph-mon[129074]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:22:11.712 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:11 smithi026 ceph-mon[95647]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:22:11.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:11 smithi121 ceph-mon[129187]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 315 MiB/s wr, 101 op/s 2023-07-13T22:22:12.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:12 smithi117 ceph-mon[129074]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 274 MiB/s wr, 91 op/s 2023-07-13T22:22:12.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:12 smithi026 ceph-mon[95647]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 274 MiB/s wr, 91 op/s 2023-07-13T22:22:12.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:12 smithi121 ceph-mon[129187]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 274 MiB/s wr, 91 op/s 2023-07-13T22:22:15.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:15 smithi117 ceph-mon[129074]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:22:15.700 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:15 smithi026 ceph-mon[95647]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:22:15.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:15 smithi121 ceph-mon[129187]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 315 MiB/s wr, 103 op/s 2023-07-13T22:22:17.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:17 smithi117 ceph-mon[129074]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:22:17.706 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:17 smithi026 ceph-mon[95647]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:22:17.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:17 smithi121 ceph-mon[129187]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 278 MiB/s wr, 92 op/s 2023-07-13T22:22:18.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:18 smithi026 ceph-mon[95647]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 100 op/s 2023-07-13T22:22:18.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:18 smithi121 ceph-mon[129187]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 100 op/s 2023-07-13T22:22:18.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:18 smithi117 ceph-mon[129074]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 100 op/s 2023-07-13T22:22:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:21 smithi117 ceph-mon[129074]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:22:21.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:21 smithi026 ceph-mon[95647]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:22:21.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:21 smithi121 ceph-mon[129187]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 305 MiB/s wr, 100 op/s 2023-07-13T22:22:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:23 smithi117 ceph-mon[129074]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 95 op/s 2023-07-13T22:22:23.711 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:23 smithi026 ceph-mon[95647]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 95 op/s 2023-07-13T22:22:23.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:23 smithi121 ceph-mon[129187]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 280 MiB/s wr, 95 op/s 2023-07-13T22:22:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:25 smithi117 ceph-mon[129074]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 330 MiB/s wr, 108 op/s 2023-07-13T22:22:25.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:25 smithi026 ceph-mon[95647]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 330 MiB/s wr, 108 op/s 2023-07-13T22:22:25.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:25 smithi121 ceph-mon[129187]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 330 MiB/s wr, 108 op/s 2023-07-13T22:22:26.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:26 smithi121 ceph-mon[129187]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:26 smithi117 ceph-mon[129074]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:26.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:26 smithi026 ceph-mon[95647]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:28.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:22:29.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.319+0000 7f658ffff700 1 -- 172.21.15.26:0/4074327469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588070ee0 msgr2=0x7f65880712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.319+0000 7f658ffff700 1 --2- 172.21.15.26:0/4074327469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588070ee0 0x7f65880712f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6584009990 tx=0x7f6584009ca0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 -- 172.21.15.26:0/4074327469 shutdown_connections 2023-07-13T22:22:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 --2- 172.21.15.26:0/4074327469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588070ee0 0x7f65880712f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 --2- 172.21.15.26:0/4074327469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f658810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 --2- 172.21.15.26:0/4074327469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588071830 0x7f658810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 -- 172.21.15.26:0/4074327469 >> 172.21.15.26:0/4074327469 conn(0x7f658806ca90 msgr2=0x7f658806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 -- 172.21.15.26:0/4074327469 shutdown_connections 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.331+0000 7f658ffff700 1 -- 172.21.15.26:0/4074327469 wait complete. 2023-07-13T22:22:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.333+0000 7f658ffff700 1 Processor -- start 2023-07-13T22:22:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.334+0000 7f658ffff700 1 -- start start 2023-07-13T22:22:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 0x7f65881a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588071830 0x7f65881a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f65881abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881ac290 con 0x7f6588071830 2023-07-13T22:22:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658810ec40 con 0x7f6588070ee0 2023-07-13T22:22:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.335+0000 7f658ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658810edd0 con 0x7f658810bd60 2023-07-13T22:22:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.336+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 0x7f65881a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.337+0000 7f658e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f65881abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.337+0000 7f658e59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f65881abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60424/0 (socket says 172.21.15.26:60424) 2023-07-13T22:22:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.337+0000 7f658e59c700 1 -- 172.21.15.26:0/1242187615 learned_addr learned my addr 172.21.15.26:0/1242187615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.344+0000 7f658dd9b700 1 -- 172.21.15.26:0/1242187615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 msgr2=0x7f65881abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.344+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f65881abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.344+0000 7f658dd9b700 1 -- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588071830 msgr2=0x7f65881a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.344+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588071830 0x7f65881a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.344+0000 7f658dd9b700 1 -- 172.21.15.26:0/1242187615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65840095f0 con 0x7f6588070ee0 2023-07-13T22:22:29.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.345+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 0x7f65881a4710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6584000c00 tx=0x7f6584003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.366+0000 7f657effd700 1 -- 172.21.15.26:0/1242187615 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658401d070 con 0x7f6588070ee0 2023-07-13T22:22:29.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.367+0000 7f657effd700 1 -- 172.21.15.26:0/1242187615 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658400f460 con 0x7f6588070ee0 2023-07-13T22:22:29.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.367+0000 7f657effd700 1 -- 172.21.15.26:0/1242187615 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584017650 con 0x7f6588070ee0 2023-07-13T22:22:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.367+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658810f050 con 0x7f6588070ee0 2023-07-13T22:22:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.367+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658810f530 con 0x7f6588070ee0 2023-07-13T22:22:29.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.374+0000 7f65a39fab80 1 Processor -- start 2023-07-13T22:22:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.381+0000 7f65a39fab80 1 -- start start 2023-07-13T22:22:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.383+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36faa68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.383+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36faa76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.383+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.384+0000 7f65a39fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f9abcb0 con 0x55d36faa6e20 2023-07-13T22:22:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.384+0000 7f65a39fab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f9fea40 con 0x55d36faa64d0 2023-07-13T22:22:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.384+0000 7f65a39fab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f9f7ed0 con 0x55d36faa7c70 2023-07-13T22:22:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.385+0000 7f658d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.385+0000 7f658d59a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60428/0 (socket says 172.21.15.26:60428) 2023-07-13T22:22:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.385+0000 7f658d59a700 1 -- 172.21.15.26:0/756469447 learned_addr learned my addr 172.21.15.26:0/756469447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.385+0000 7f658e59c700 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36faa68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.386+0000 7f658d59a700 1 -- 172.21.15.26:0/756469447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 msgr2=0x55d36faa68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.386+0000 7f658d59a700 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36faa68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.386+0000 7f658d59a700 1 -- 172.21.15.26:0/756469447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 msgr2=0x55d36faa76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.386+0000 7f658d59a700 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36faa76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.386+0000 7f658d59a700 1 -- 172.21.15.26:0/756469447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d36f825de0 con 0x55d36faa7c70 2023-07-13T22:22:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.389+0000 7f657effd700 1 -- 172.21.15.26:0/1242187615 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f65840177b0 con 0x7f6588070ee0 2023-07-13T22:22:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.392+0000 7f658d59a700 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f657800bba0 tx=0x7f657800beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38ba9aefe36f2a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.392+0000 7f657effd700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 0x7f65740a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.392+0000 7f657effd700 1 -- 172.21.15.26:0/1242187615 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65840ce980 con 0x7f6588070ee0 2023-07-13T22:22:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.393+0000 7f6573fff700 1 -- 172.21.15.26:0/756469447 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657800da90 con 0x55d36faa7c70 2023-07-13T22:22:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.393+0000 7f6573fff700 1 -- 172.21.15.26:0/756469447 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f657800b680 con 0x55d36faa7c70 2023-07-13T22:22:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.393+0000 7f6573fff700 1 -- 172.21.15.26:0/756469447 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657800f620 con 0x55d36faa7c70 2023-07-13T22:22:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.394+0000 7f65a39fab80 1 -- 172.21.15.26:0/756469447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 msgr2=0x55d36faa80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.394+0000 7f65a39fab80 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f657800bba0 tx=0x7f657800beb0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.394+0000 7f658e59c700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 0x7f65740a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 -- 172.21.15.26:0/756469447 shutdown_connections 2023-07-13T22:22:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36faa80e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36faa68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 --2- 172.21.15.26:0/756469447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36faa76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 -- 172.21.15.26:0/756469447 >> 172.21.15.26:0/756469447 conn(0x55d36fa9bda0 msgr2=0x55d36faa11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 -- 172.21.15.26:0/756469447 shutdown_connections 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.395+0000 7f65a39fab80 1 -- 172.21.15.26:0/756469447 wait complete. 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.397+0000 7f65a39fab80 1 -- start start 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 Processor -- start 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa6e20 0x55d36fab0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36fab77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f997ac0 con 0x55d36faa7c70 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f9977c0 con 0x55d36faa64d0 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36f87b920 con 0x55d36faa6e20 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.398+0000 7f65a39fab80 10 client.0 _ll_register_callbacks cb 0x55d36f995760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.399+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.399+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38278/0 (socket says 172.21.15.26:38278) 2023-07-13T22:22:29.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.399+0000 7f658dd9b700 1 -- 172.21.15.26:0/1497873457 learned_addr learned my addr 172.21.15.26:0/1497873457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.399+0000 7f658d59a700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa6e20 0x55d36fab0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.401+0000 7f658dd9b700 1 -- 172.21.15.26:0/1497873457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa6e20 msgr2=0x55d36fab0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.401+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa6e20 0x55d36fab0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.402+0000 7f658dd9b700 1 -- 172.21.15.26:0/1497873457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 msgr2=0x55d36fab77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.402+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36fab77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.402+0000 7f658dd9b700 1 -- 172.21.15.26:0/1497873457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657800b800 con 0x55d36faa64d0 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f658dd9b700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f658409bd60 tx=0x7f658409bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f59d670e8b527a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f65717fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f658e59c700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 0x7f65740a3800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6580007580 tx=0x7f6580007890 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840b3070 con 0x55d36faa64d0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6584029340 con 0x55d36faa64d0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.403+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d36f9f0eb0 con 0x55d36faa64d0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.404+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658409b730 con 0x55d36faa64d0 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.404+0000 7f65a39fab80 10 client.26798 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:29.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.404+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d36f80ec60 con 0x55d36faa64d0 2023-07-13T22:22:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.404+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d36f9f0eb0 con 0x55d36faa64d0 2023-07-13T22:22:29.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f658409b3e0 con 0x55d36faa64d0 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 10 client.26798 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 10 client.26798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 20 client.26798 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fabbc10 0x55d36fabe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 4 client.26798 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d36f9b3050 con 0x55d36fabbc10 2023-07-13T22:22:29.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.406+0000 7f658d59a700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fabbc10 0x55d36fabe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.409+0000 7f658d59a700 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fabbc10 0x55d36fabe020 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a8e2846cbc98ee server_cookie=3b7145a725639931 in_seq=0 out_seq=0 2023-07-13T22:22:29.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.409+0000 7f65717fa700 10 client.26798 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.418+0000 7f65717fa700 1 -- 172.21.15.26:0/1497873457 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d36f9b3050 con 0x55d36fabbc10 2023-07-13T22:22:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.418+0000 7f65717fa700 10 client.26798 handle_command_reply: tid=0 2023-07-13T22:22:29.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.419+0000 7f65a39fab80 1 client.26798 shutdown 2023-07-13T22:22:29.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 msgr2=0x55d36fab0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f658409bd60 tx=0x7f658409bba0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 shutdown_connections 2023-07-13T22:22:29.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36fab77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fabbc10 0x55d36fabe020 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab0220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 --2- 172.21.15.26:0/1497873457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa6e20 0x55d36fab0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 >> 172.21.15.26:0/1497873457 conn(0x55d36fa9bda0 msgr2=0x55d36faa08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.421+0000 7f65717fa700 0 client.26798 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:29.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.422+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 shutdown_connections 2023-07-13T22:22:29.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.422+0000 7f65a39fab80 1 -- 172.21.15.26:0/1497873457 wait complete. 2023-07-13T22:22:29.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.423+0000 7f65a39fab80 20 client.26798 trim_cache size 0 max 16384 2023-07-13T22:22:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 Processor -- start 2023-07-13T22:22:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 -- start start 2023-07-13T22:22:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa64d0 0x55d36fabdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 0x55d36fab7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36faae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36fa9dec0 con 0x55d36faa7c70 2023-07-13T22:22:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36faa2070 con 0x55d36faa6e20 2023-07-13T22:22:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.456+0000 7f65a39fab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36fab86e0 con 0x55d36faa64d0 2023-07-13T22:22:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.458+0000 7f658d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 0x55d36fab7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.458+0000 7f658d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 0x55d36fab7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38292/0 (socket says 172.21.15.26:38292) 2023-07-13T22:22:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.458+0000 7f658d59a700 1 -- 172.21.15.26:0/2756134963 learned_addr learned my addr 172.21.15.26:0/2756134963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.459+0000 7f658dd9b700 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa64d0 0x55d36fabdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.459+0000 7f658e59c700 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36faae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.460+0000 7f658e59c700 1 -- 172.21.15.26:0/2756134963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa64d0 msgr2=0x55d36fabdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.460+0000 7f658e59c700 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa64d0 0x55d36fabdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.460+0000 7f658e59c700 1 -- 172.21.15.26:0/2756134963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 msgr2=0x55d36fab7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.460+0000 7f658e59c700 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 0x55d36fab7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.460+0000 7f658e59c700 1 -- 172.21.15.26:0/2756134963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d36f84c250 con 0x55d36faa7c70 2023-07-13T22:22:29.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.461+0000 7f658e59c700 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36faae140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f658001c4a0 tx=0x7f6580015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc3048971b25c6f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.461+0000 7f657cff9700 1 -- 172.21.15.26:0/2756134963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580025480 con 0x55d36faa7c70 2023-07-13T22:22:29.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.461+0000 7f657cff9700 1 -- 172.21.15.26:0/2756134963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65800198d0 con 0x55d36faa7c70 2023-07-13T22:22:29.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.461+0000 7f657cff9700 1 -- 172.21.15.26:0/2756134963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580017a60 con 0x55d36faa7c70 2023-07-13T22:22:29.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.462+0000 7f65a39fab80 1 -- 172.21.15.26:0/2756134963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 msgr2=0x55d36faae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.462+0000 7f65a39fab80 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36faae140 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f658001c4a0 tx=0x7f6580015000 comp rx=0 tx=0).stop 2023-07-13T22:22:29.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 -- 172.21.15.26:0/2756134963 shutdown_connections 2023-07-13T22:22:29.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa7c70 0x55d36faae140 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa64d0 0x55d36fabdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 --2- 172.21.15.26:0/2756134963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa6e20 0x55d36fab7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 -- 172.21.15.26:0/2756134963 >> 172.21.15.26:0/2756134963 conn(0x55d36fa9bda0 msgr2=0x55d36fab7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 -- 172.21.15.26:0/2756134963 shutdown_connections 2023-07-13T22:22:29.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 -- 172.21.15.26:0/2756134963 wait complete. 2023-07-13T22:22:29.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.463+0000 7f65a39fab80 1 -- start start 2023-07-13T22:22:29.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.464+0000 7f65a39fab80 1 Processor -- start 2023-07-13T22:22:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.464+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36fab59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36fab5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36fab6430 con 0x55d36faa6e20 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36fab65a0 con 0x55d36faa64d0 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d36fab6760 con 0x55d36faa7c70 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f65a39fab80 10 client.0 _ll_register_callbacks cb 0x7f65840979f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2023-07-13T22:22:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f658dd9b700 1 -- 172.21.15.26:0/388893556 learned_addr learned my addr 172.21.15.26:0/388893556 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.465+0000 7f658d59a700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36fab59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 -- 172.21.15.26:0/388893556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 msgr2=0x55d36fab5ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36fab5ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 -- 172.21.15.26:0/388893556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 msgr2=0x55d36fab59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36fab59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 -- 172.21.15.26:0/388893556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6580011040 con 0x55d36faa64d0 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f658dd9b700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f65840a3c60 tx=0x7f65840a3fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2adf5e9731f94bec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f65717fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65840b3070 con 0x55d36faa64d0 2023-07-13T22:22:29.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.466+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6584029340 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.467+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d36f9514f0 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.467+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65840cc2a0 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.467+0000 7f65a39fab80 10 client.26804 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.467+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d36f8242b0 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65840bf020 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f65840b4230 con 0x55d36faa64d0 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 10 client.26804 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:29.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 10 client.26804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 20 client.26804 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fb10c10 0x55d36fb13020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 4 client.26804 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:22:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.468+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d36f846f70 con 0x55d36fb10c10 2023-07-13T22:22:29.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.470+0000 7f658d59a700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fb10c10 0x55d36fb13020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.470+0000 7f658d59a700 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fb10c10 0x55d36fb13020 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2b77f4aef41cda server_cookie=e62f024297c26cf8 in_seq=0 out_seq=0 2023-07-13T22:22:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.471+0000 7f65717fa700 10 client.26804 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:29.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.471+0000 7f65717fa700 1 -- 172.21.15.26:0/388893556 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d36f846f70 con 0x55d36fb10c10 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.471+0000 7f65717fa700 10 client.26804 handle_command_reply: tid=0 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.471+0000 7f65a39fab80 1 client.26804 shutdown 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.472+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 msgr2=0x55d36fab73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.472+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f65840a3c60 tx=0x7f65840a3fe0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 shutdown_connections 2023-07-13T22:22:29.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55d36fb10c10 0x55d36fb13020 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d36faa7c70 0x55d36fab5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d36faa64d0 0x55d36fab73a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 --2- 172.21.15.26:0/388893556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d36faa6e20 0x55d36fab59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 >> 172.21.15.26:0/388893556 conn(0x55d36fa9bda0 msgr2=0x55d36faae140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65717fa700 0 client.26804 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 shutdown_connections 2023-07-13T22:22:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.473+0000 7f65a39fab80 1 -- 172.21.15.26:0/388893556 wait complete. 2023-07-13T22:22:29.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.474+0000 7f65a39fab80 20 client.26804 trim_cache size 0 max 16384 2023-07-13T22:22:29.463 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:22:29.463 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:22:29.464 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:22:29.464 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:22:29.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.474+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 msgr2=0x7f65740a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.474+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 0x7f65740a3800 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6580007580 tx=0x7f6580007890 comp rx=0 tx=0).stop 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.474+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 msgr2=0x7f65881a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.474+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 0x7f65881a4710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6584000c00 tx=0x7f6584003c40 comp rx=0 tx=0).stop 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 shutdown_connections 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f658810bd60 0x7f65881abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6588070ee0 0x7f65881a4710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65740a1350 0x7f65740a3800 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 --2- 172.21.15.26:0/1242187615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6588071830 0x7f65881a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 >> 172.21.15.26:0/1242187615 conn(0x7f658806ca90 msgr2=0x7f658806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 shutdown_connections 2023-07-13T22:22:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.475+0000 7f658ffff700 1 -- 172.21.15.26:0/1242187615 wait complete. 2023-07-13T22:22:29.484 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:22:29.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:22:29.609 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:29 smithi026 ceph-mon[95647]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 317 MiB/s wr, 105 op/s 2023-07-13T22:22:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:29 smithi117 ceph-mon[129074]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 317 MiB/s wr, 105 op/s 2023-07-13T22:22:29.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:29 smithi121 ceph-mon[129187]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 317 MiB/s wr, 105 op/s 2023-07-13T22:22:29.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 -- 172.21.15.26:0/1017095484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 msgr2=0x7fd99010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 --2- 172.21.15.26:0/1017095484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd99010e1e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd984009990 tx=0x7fd984009ca0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 -- 172.21.15.26:0/1017095484 shutdown_connections 2023-07-13T22:22:29.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 --2- 172.21.15.26:0/1017095484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9900712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 --2- 172.21.15.26:0/1017095484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd99010e1e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 --2- 172.21.15.26:0/1017095484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd99010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.979+0000 7fd9955ff700 1 -- 172.21.15.26:0/1017095484 >> 172.21.15.26:0/1017095484 conn(0x7fd99006ca40 msgr2=0x7fd99006ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:29.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.980+0000 7fd9955ff700 1 -- 172.21.15.26:0/1017095484 shutdown_connections 2023-07-13T22:22:29.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.980+0000 7fd9955ff700 1 -- 172.21.15.26:0/1017095484 wait complete. 2023-07-13T22:22:29.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.982+0000 7fd9955ff700 1 Processor -- start 2023-07-13T22:22:29.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.982+0000 7fd9955ff700 1 -- start start 2023-07-13T22:22:29.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.982+0000 7fd9955ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9901a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.982+0000 7fd9955ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd9901a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.983+0000 7fd98effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9901a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.983+0000 7fd98effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9901a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44718/0 (socket says 172.21.15.26:44718) 2023-07-13T22:22:29.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd9901a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd9955ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd9901abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd9955ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901ac110 con 0x7fd990070ed0 2023-07-13T22:22:29.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd9955ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901ac250 con 0x7fd99010bd60 2023-07-13T22:22:29.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd9955ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901ac3e0 con 0x7fd990071820 2023-07-13T22:22:29.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98effd700 1 -- 172.21.15.26:0/2914782412 learned_addr learned my addr 172.21.15.26:0/2914782412 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:29.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd9901abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 -- 172.21.15.26:0/2914782412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 msgr2=0x7fd9901abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd9901abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 -- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 msgr2=0x7fd9901a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:29.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9901a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:29.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.984+0000 7fd98e7fc700 1 -- 172.21.15.26:0/2914782412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840095f0 con 0x7fd990071820 2023-07-13T22:22:29.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd98e7fc700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd9901a4ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd98800eee0 tx=0x7fd98800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:29.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988018070 con 0x7fd990071820 2023-07-13T22:22:29.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98800f040 con 0x7fd990071820 2023-07-13T22:22:29.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988004280 con 0x7fd990071820 2023-07-13T22:22:29.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9901ac660 con 0x7fd990071820 2023-07-13T22:22:29.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.985+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9901acb20 con 0x7fd990071820 2023-07-13T22:22:29.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.987+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99010af60 con 0x7fd990071820 2023-07-13T22:22:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.991+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd98800cad0 con 0x7fd990071820 2023-07-13T22:22:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.994+0000 7fd977fff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 0x7fd9780a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.994+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9880cc6f0 con 0x7fd990071820 2023-07-13T22:22:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.995+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd988099410 con 0x7fd990071820 2023-07-13T22:22:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.996+0000 7fd98effd700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 0x7fd9780a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:29.997+0000 7fd98effd700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 0x7fd9780a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd980009cb0 tx=0x7fd980009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.710+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd99002c850 con 0x7fd990071820 2023-07-13T22:22:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.716+0000 7fd977fff700 1 -- 172.21.15.26:0/2914782412 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fd9880995f0 con 0x7fd990071820 2023-07-13T22:22:30.736 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:22:30.736 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:22:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.817+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 msgr2=0x7fd9780a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.817+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 0x7fd9780a3750 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd980009cb0 tx=0x7fd980009260 comp rx=0 tx=0).stop 2023-07-13T22:22:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.817+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 msgr2=0x7fd9901a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.817+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd9901a4ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd98800eee0 tx=0x7fd98800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:22:30.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 shutdown_connections 2023-07-13T22:22:30.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a12a0 0x7fd9780a3750 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:30.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd990070ed0 0x7fd9901a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:30.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010bd60 0x7fd9901abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:30.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 --2- 172.21.15.26:0/2914782412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd990071820 0x7fd9901a4ad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:30.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.861+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 >> 172.21.15.26:0/2914782412 conn(0x7fd99006ca40 msgr2=0x7fd99006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:30.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.862+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 shutdown_connections 2023-07-13T22:22:30.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:30.862+0000 7fd9955ff700 1 -- 172.21.15.26:0/2914782412 wait complete. 2023-07-13T22:22:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:22:30.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:22:31.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.616+0000 7f8f117ff700 1 -- 172.21.15.26:0/255575470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c1080d0 msgr2=0x7f8f0c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.616+0000 7f8f117ff700 1 --2- 172.21.15.26:0/255575470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c10e970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009990 tx=0x7f8efc009ca0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.617+0000 7f8f117ff700 1 -- 172.21.15.26:0/255575470 shutdown_connections 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.617+0000 7f8f117ff700 1 --2- 172.21.15.26:0/255575470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c10e970 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.617+0000 7f8f117ff700 1 --2- 172.21.15.26:0/255575470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.617+0000 7f8f117ff700 1 --2- 172.21.15.26:0/255575470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c107720 0x7f8f0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.617+0000 7f8f117ff700 1 -- 172.21.15.26:0/255575470 >> 172.21.15.26:0/255575470 conn(0x7f8f0c06cd80 msgr2=0x7f8f0c06f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.618+0000 7f8f117ff700 1 -- 172.21.15.26:0/255575470 shutdown_connections 2023-07-13T22:22:31.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.618+0000 7f8f117ff700 1 -- 172.21.15.26:0/255575470 wait complete. 2023-07-13T22:22:31.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.620+0000 7f8f117ff700 1 Processor -- start 2023-07-13T22:22:31.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.620+0000 7f8f117ff700 1 -- start start 2023-07-13T22:22:31.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.620+0000 7f8f117ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.620+0000 7f8f117ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c107720 0x7f8f0c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f117ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f117ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1b48c0 con 0x7f8f0c0716f0 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f117ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1b4a00 con 0x7f8f0c1080d0 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f117ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c1b4b90 con 0x7f8f0c107720 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38344/0 (socket says 172.21.15.26:38344) 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/1222883493 learned_addr learned my addr 172.21.15.26:0/1222883493 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.621+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c107720 0x7f8f0c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.622+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/1222883493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c107720 msgr2=0x7f8f0c19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.622+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c107720 0x7f8f0c19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.622+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c0716f0 msgr2=0x7f8f0c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.622+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.622+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/1222883493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0095f0 con 0x7f8f0c1080d0 2023-07-13T22:22:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.623+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005f50 tx=0x7f8efc003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.623+0000 7f8ef3fff700 1 -- 172.21.15.26:0/1222883493 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0052c0 con 0x7f8f0c1080d0 2023-07-13T22:22:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.623+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c072740 con 0x7f8f0c1080d0 2023-07-13T22:22:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.623+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c072bf0 con 0x7f8f0c1080d0 2023-07-13T22:22:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.624+0000 7f8ef3fff700 1 -- 172.21.15.26:0/1222883493 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8efc03f070 con 0x7f8f0c1080d0 2023-07-13T22:22:31.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.624+0000 7f8ef3fff700 1 -- 172.21.15.26:0/1222883493 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00fdc0 con 0x7f8f0c1080d0 2023-07-13T22:22:31.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.632+0000 7f8f2508db80 1 Processor -- start 2023-07-13T22:22:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 -- start start 2023-07-13T22:22:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570d4d0 0x55c41570d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570ec70 0x55c41570f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c415612cb0 con 0x55c41570d4d0 2023-07-13T22:22:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c415665a40 con 0x55c41570de20 2023-07-13T22:22:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.635+0000 7f8f2508db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c41565eed0 con 0x55c41570ec70 2023-07-13T22:22:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.639+0000 7f8f0affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.639+0000 7f8f0affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38358/0 (socket says 172.21.15.26:38358) 2023-07-13T22:22:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.639+0000 7f8f0affd700 1 -- 172.21.15.26:0/1956329558 learned_addr learned my addr 172.21.15.26:0/1956329558 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.643+0000 7f8ef3fff700 1 -- 172.21.15.26:0/1222883493 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8efc041030 con 0x7f8f0c1080d0 2023-07-13T22:22:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.646+0000 7f8f0affd700 1 -- 172.21.15.26:0/1956329558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570ec70 msgr2=0x55c41570f0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.646+0000 7f8f0affd700 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570ec70 0x55c41570f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.646+0000 7f8f0affd700 1 -- 172.21.15.26:0/1956329558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570d4d0 msgr2=0x55c41570d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.646+0000 7f8f0affd700 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570d4d0 0x55c41570d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.646+0000 7f8f0affd700 1 -- 172.21.15.26:0/1956329558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c41548cde0 con 0x55c41570de20 2023-07-13T22:22:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.647+0000 7f8f0affd700 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b4a0 tx=0x7f8f0400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c65f57e9a18f41f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.648+0000 7f8ef3fff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 0x7f8ef40a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.648+0000 7f8ef3fff700 1 -- 172.21.15.26:0/1222883493 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8efc0ef810 con 0x7f8f0c1080d0 2023-07-13T22:22:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.649+0000 7f8ef17fa700 1 -- 172.21.15.26:0/1956329558 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04010070 con 0x55c41570de20 2023-07-13T22:22:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.649+0000 7f8ef17fa700 1 -- 172.21.15.26:0/1956329558 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f040148c0 con 0x55c41570de20 2023-07-13T22:22:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.649+0000 7f8ef17fa700 1 -- 172.21.15.26:0/1956329558 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040097c0 con 0x55c41570de20 2023-07-13T22:22:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.650+0000 7f8f2508db80 1 -- 172.21.15.26:0/1956329558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 msgr2=0x55c41570e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.650+0000 7f8f2508db80 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8f0400b4a0 tx=0x7f8f0400b860 comp rx=0 tx=0).stop 2023-07-13T22:22:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.651+0000 7f8f2508db80 1 -- 172.21.15.26:0/1956329558 shutdown_connections 2023-07-13T22:22:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.651+0000 7f8f2508db80 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570d4d0 0x55c41570d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.651+0000 7f8f2508db80 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570ec70 0x55c41570f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.651+0000 7f8f2508db80 1 --2- 172.21.15.26:0/1956329558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c41570e6e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.651+0000 7f8f2508db80 1 -- 172.21.15.26:0/1956329558 >> 172.21.15.26:0/1956329558 conn(0x55c415702da0 msgr2=0x55c4157081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.652+0000 7f8f2508db80 1 -- 172.21.15.26:0/1956329558 shutdown_connections 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.652+0000 7f8f2508db80 1 -- 172.21.15.26:0/1956329558 wait complete. 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.653+0000 7f8f2508db80 1 -- start start 2023-07-13T22:22:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.653+0000 7f8f2508db80 1 Processor -- start 2023-07-13T22:22:31.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.653+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 0x7f8ef40a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c415717370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c41571c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4155feac0 con 0x55c415717d20 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4155fe7c0 con 0x55c41570de20 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4154e2920 con 0x55c4157178b0 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.654+0000 7f8f2508db80 10 client.0 _ll_register_callbacks cb 0x55c4155fc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.655+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.655+0000 7f8f0b7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44750/0 (socket says 172.21.15.26:44750) 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.655+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/936740371 learned_addr learned my addr 172.21.15.26:0/936740371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.656+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/936740371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 msgr2=0x55c41571c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.656+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c41571c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.656+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 msgr2=0x55c415717370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.656+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c415717370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.656+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/936740371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f04008d20 con 0x55c415717d20 2023-07-13T22:22:31.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.657+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8efc011040 tx=0x7f8efc0bc7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abed5f3e2a71836 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.659+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 0x7f8ef40a3730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005950 tx=0x7f8f0000a380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.660+0000 7f8eeeff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:22:31.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.660+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0bcc60 con 0x55c415717d20 2023-07-13T22:22:31.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.660+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8efc0c0c90 con 0x55c415717d20 2023-07-13T22:22:31.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.661+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4155b84f0 con 0x55c415717d20 2023-07-13T22:22:31.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.662+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8efc0c5020 con 0x55c415717d20 2023-07-13T22:22:31.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.662+0000 7f8f2508db80 10 client.16374 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:31.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.662+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c415475c60 con 0x55c415717d20 2023-07-13T22:22:31.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c4155b84f0 con 0x55c415717d20 2023-07-13T22:22:31.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8efc0bc150 con 0x55c415717d20 2023-07-13T22:22:31.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:31.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 10 client.16374 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:22:31.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 20 client.16374 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:31.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415723510 0x55c415725920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 4 client.16374 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:31.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.663+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c415657eb0 con 0x55c415723510 2023-07-13T22:22:31.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.667+0000 7f8f0affd700 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415723510 0x55c415725920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.669+0000 7f8f0affd700 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415723510 0x55c415725920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97477149c9a2041c server_cookie=5f94d7ec4b85d881 in_seq=0 out_seq=0 2023-07-13T22:22:31.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.669+0000 7f8eeeff5700 10 client.16374 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:22:31.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.669+0000 7f8eeeff5700 1 -- 172.21.15.26:0/936740371 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c415657eb0 con 0x55c415723510 2023-07-13T22:22:31.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.669+0000 7f8eeeff5700 10 client.16374 handle_command_reply: tid=0 2023-07-13T22:22:31.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.670+0000 7f8f2508db80 1 client.16374 shutdown 2023-07-13T22:22:31.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.671+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 msgr2=0x55c41571ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.671+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8efc011040 tx=0x7f8efc0bc7a0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 shutdown_connections 2023-07-13T22:22:31.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415723510 0x55c415725920 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c415717d20 0x55c41571ee70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c41570de20 0x55c415717370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 --2- 172.21.15.26:0/936740371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c41571c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 >> 172.21.15.26:0/936740371 conn(0x55c415702da0 msgr2=0x55c4157078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.673+0000 7f8eeeff5700 0 client.16374 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:22:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.674+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 shutdown_connections 2023-07-13T22:22:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.674+0000 7f8f2508db80 1 -- 172.21.15.26:0/936740371 wait complete. 2023-07-13T22:22:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.674+0000 7f8f2508db80 20 client.16374 trim_cache size 0 max 16384 2023-07-13T22:22:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.693+0000 7f8f2508db80 1 Processor -- start 2023-07-13T22:22:31.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 -- start start 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570de20 0x55c41570e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415715270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c415704ec0 con 0x55c4157178b0 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4157180f0 con 0x55c415717d20 2023-07-13T22:22:31.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.694+0000 7f8f2508db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c415709670 con 0x55c41570de20 2023-07-13T22:22:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.695+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.695+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44766/0 (socket says 172.21.15.26:44766) 2023-07-13T22:22:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/2189747792 learned_addr learned my addr 172.21.15.26:0/2189747792 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0affd700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570de20 0x55c41570e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415715270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/2189747792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570de20 msgr2=0x55c41570e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570de20 0x55c41570e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/2189747792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 msgr2=0x55c41571c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0b7fe700 1 -- 172.21.15.26:0/2189747792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4154b3250 con 0x55c415717d20 2023-07-13T22:22:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.696+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.697+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415715270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b96d0 tx=0x7f8efc005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8586885170a77091 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.697+0000 7f8ef1ffb700 1 -- 172.21.15.26:0/2189747792 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0c0c90 con 0x55c415717d20 2023-07-13T22:22:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.697+0000 7f8ef1ffb700 1 -- 172.21.15.26:0/2189747792 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8efc0bcec0 con 0x55c415717d20 2023-07-13T22:22:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.697+0000 7f8ef1ffb700 1 -- 172.21.15.26:0/2189747792 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0d5200 con 0x55c415717d20 2023-07-13T22:22:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.698+0000 7f8f2508db80 1 -- 172.21.15.26:0/2189747792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 msgr2=0x55c415715270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.698+0000 7f8f2508db80 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415715270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8efc0b96d0 tx=0x7f8efc005c70 comp rx=0 tx=0).stop 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 -- 172.21.15.26:0/2189747792 shutdown_connections 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415715270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c41570de20 0x55c41570e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 --2- 172.21.15.26:0/2189747792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c4157178b0 0x55c41571c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 -- 172.21.15.26:0/2189747792 >> 172.21.15.26:0/2189747792 conn(0x55c415702da0 msgr2=0x55c41572a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 -- 172.21.15.26:0/2189747792 shutdown_connections 2023-07-13T22:22:31.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 -- 172.21.15.26:0/2189747792 wait complete. 2023-07-13T22:22:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.699+0000 7f8f2508db80 1 -- start start 2023-07-13T22:22:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 Processor -- start 2023-07-13T22:22:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570de20 0x55c415771390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415775e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c41571ce40 con 0x55c41570de20 2023-07-13T22:22:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c415776370 con 0x55c415717d20 2023-07-13T22:22:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4157764b0 con 0x55c4157178b0 2023-07-13T22:22:31.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.700+0000 7f8f2508db80 10 client.0 _ll_register_callbacks cb 0x55c4155fc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.701+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.701+0000 7f8f0a7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60522/0 (socket says 172.21.15.26:60522) 2023-07-13T22:22:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.701+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3410985848 learned_addr learned my addr 172.21.15.26:0/3410985848 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.701+0000 7f8f0b7fe700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415775e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0affd700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570de20 0x55c415771390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 msgr2=0x55c415775e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415775e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3410985848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570de20 msgr2=0x55c415771390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570de20 0x55c415771390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 -- 172.21.15.26:0/3410985848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0000f7b0 con 0x55c4157178b0 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.702+0000 7f8f0a7fc700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001d9b0 tx=0x7f8f000047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70c1dcb1cc246a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:31.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.703+0000 7f8eeeff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:22:31.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:31 smithi117 ceph-mon[129074]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 324 MiB/s wr, 103 op/s 2023-07-13T22:22:31.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:31 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2914782412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.703+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f00004220 con 0x55c4157178b0 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.703+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8f00004380 con 0x55c4157178b0 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.703+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f040148c0 con 0x55c4157178b0 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.704+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f00004540 con 0x55c4157178b0 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.704+0000 7f8f2508db80 10 client.26815 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:31.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.704+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c415475c60 con 0x55c4157178b0 2023-07-13T22:22:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.706+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f040148c0 con 0x55c4157178b0 2023-07-13T22:22:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.706+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f00010ba0 con 0x55c4157178b0 2023-07-13T22:22:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 10 client.26815 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:31.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 10 client.26815 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:22:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 20 client.26815 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415777f90 0x55c41577a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 4 client.26815 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:22:31.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.707+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c41561a050 con 0x55c415777f90 2023-07-13T22:22:31.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.709+0000 7f8f0affd700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415777f90 0x55c41577a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:31.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:31 smithi026 ceph-mon[95647]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 324 MiB/s wr, 103 op/s 2023-07-13T22:22:31.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:31 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2914782412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:22:31.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8f0affd700 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415777f90 0x55c41577a3a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65e45622cc163e3b server_cookie=d9f590f5aa2d392c in_seq=0 out_seq=0 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8eeeff5700 10 client.26815 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8eeeff5700 1 -- 172.21.15.26:0/3410985848 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c41561a050 con 0x55c415777f90 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8eeeff5700 10 client.26815 handle_command_reply: tid=0 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8f2508db80 1 client.26815 shutdown 2023-07-13T22:22:31.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 msgr2=0x55c4157738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.713+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001d9b0 tx=0x7f8f000047c0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 shutdown_connections 2023-07-13T22:22:31.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55c415777f90 0x55c41577a3a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c41570de20 0x55c415771390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c415717d20 0x55c415775e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 --2- 172.21.15.26:0/3410985848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c4157178b0 0x55c4157738e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 >> 172.21.15.26:0/3410985848 conn(0x55c415702da0 msgr2=0x55c415715270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8eeeff5700 0 client.26815 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.714+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 shutdown_connections 2023-07-13T22:22:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.715+0000 7f8f2508db80 1 -- 172.21.15.26:0/3410985848 wait complete. 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.715+0000 7f8f2508db80 20 client.26815 trim_cache size 0 max 16384 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:22:31.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.717+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 msgr2=0x7f8ef40a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.717+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 0x7f8ef40a3730 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8f00005950 tx=0x7f8f0000a380 comp rx=0 tx=0).stop 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.717+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 msgr2=0x7f8f0c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.717+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8efc005f50 tx=0x7f8efc003cd0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 shutdown_connections 2023-07-13T22:22:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8ef40a1280 0x7f8ef40a3730 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f0c0716f0 0x7f8f0c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f0c1080d0 0x7f8f0c19c3b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 --2- 172.21.15.26:0/1222883493 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f0c107720 0x7f8f0c19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.718+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 >> 172.21.15.26:0/1222883493 conn(0x7f8f0c06cd80 msgr2=0x7f8f0c06eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.719+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 shutdown_connections 2023-07-13T22:22:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:31.719+0000 7f8f117ff700 1 -- 172.21.15.26:0/1222883493 wait complete. 2023-07-13T22:22:31.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:22:31.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:31 smithi121 ceph-mon[129187]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 324 MiB/s wr, 103 op/s 2023-07-13T22:22:31.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:31 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2914782412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:22:32.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.399+0000 7f924339d700 1 -- 172.21.15.26:0/2198356363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c071b20 msgr2=0x7f923c10b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.399+0000 7f924339d700 1 --2- 172.21.15.26:0/2198356363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c071b20 0x7f923c10b530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238009ca0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 -- 172.21.15.26:0/2198356363 shutdown_connections 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 --2- 172.21.15.26:0/2198356363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c0711d0 0x7f923c0715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 --2- 172.21.15.26:0/2198356363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f923c10bba0 0x7f923c10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 --2- 172.21.15.26:0/2198356363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c071b20 0x7f923c10b530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 -- 172.21.15.26:0/2198356363 >> 172.21.15.26:0/2198356363 conn(0x7f923c06c8b0 msgr2=0x7f923c06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 -- 172.21.15.26:0/2198356363 shutdown_connections 2023-07-13T22:22:32.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.400+0000 7f924339d700 1 -- 172.21.15.26:0/2198356363 wait complete. 2023-07-13T22:22:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.401+0000 7f924339d700 1 Processor -- start 2023-07-13T22:22:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 -- start start 2023-07-13T22:22:32.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f923c0711d0 0x7f923c1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c071b20 0x7f923c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c1abf80 con 0x7f923c071b20 2023-07-13T22:22:32.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c1ac0c0 con 0x7f923c0711d0 2023-07-13T22:22:32.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.402+0000 7f924339d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c1ac250 con 0x7f923c10bba0 2023-07-13T22:22:32.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60544/0 (socket says 172.21.15.26:60544) 2023-07-13T22:22:32.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 -- 172.21.15.26:0/1352993278 learned_addr learned my addr 172.21.15.26:0/1352993278 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:32.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9241b9a700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c071b20 0x7f923c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 -- 172.21.15.26:0/1352993278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f923c0711d0 msgr2=0x7f923c1a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f923c0711d0 0x7f923c1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 -- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c071b20 msgr2=0x7f923c1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.403+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c071b20 0x7f923c1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.404+0000 7f9242b9c700 1 -- 172.21.15.26:0/1352993278 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380095f0 con 0x7f923c10bba0 2023-07-13T22:22:32.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.404+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f922c00cab0 tx=0x7f922c00ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.404+0000 7f92337fe700 1 -- 172.21.15.26:0/1352993278 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c004c40 con 0x7f923c10bba0 2023-07-13T22:22:32.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.404+0000 7f92337fe700 1 -- 172.21.15.26:0/1352993278 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f922c007e40 con 0x7f923c10bba0 2023-07-13T22:22:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.404+0000 7f92337fe700 1 -- 172.21.15.26:0/1352993278 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c00f450 con 0x7f923c10bba0 2023-07-13T22:22:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.405+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c1ac530 con 0x7f923c10bba0 2023-07-13T22:22:32.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.405+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c1ac9f0 con 0x7f923c10bba0 2023-07-13T22:22:32.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.412+0000 7f9257ebab80 1 Processor -- start 2023-07-13T22:22:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.412+0000 7f9257ebab80 1 -- start start 2023-07-13T22:22:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d7f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355c82cb0 con 0x55c355d7ee30 2023-07-13T22:22:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355cd5a40 con 0x55c355d7e070 2023-07-13T22:22:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.413+0000 7f9257ebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355cceed0 con 0x55c355d7d720 2023-07-13T22:22:32.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.414+0000 7f9241b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.414+0000 7f9241b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60554/0 (socket says 172.21.15.26:60554) 2023-07-13T22:22:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.414+0000 7f9241b9a700 1 -- 172.21.15.26:0/1638325307 learned_addr learned my addr 172.21.15.26:0/1638325307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:32.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.415+0000 7f924239b700 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.418+0000 7f9241b9a700 1 -- 172.21.15.26:0/1638325307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 msgr2=0x55c355d7e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.418+0000 7f9241b9a700 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d7e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.418+0000 7f9241b9a700 1 -- 172.21.15.26:0/1638325307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 msgr2=0x55c355d7f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.418+0000 7f9241b9a700 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d7f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.418+0000 7f9241b9a700 1 -- 172.21.15.26:0/1638325307 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c355afcde0 con 0x55c355d7d720 2023-07-13T22:22:32.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.419+0000 7f92337fe700 1 -- 172.21.15.26:0/1352993278 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f922c0050b0 con 0x7f923c10bba0 2023-07-13T22:22:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.422+0000 7f92337fe700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 0x7f92280a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.422+0000 7f92337fe700 1 -- 172.21.15.26:0/1352993278 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f922c0cc9a0 con 0x7f923c10bba0 2023-07-13T22:22:32.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.423+0000 7f924239b700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 0x7f92280a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.424+0000 7f9241b9a700 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238005530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f343c7f601075ab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.425+0000 7f924239b700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 0x7f92280a3890 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f923c072710 tx=0x7f923400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.426+0000 7f9230bfe700 1 -- 172.21.15.26:0/1638325307 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800bcb0 con 0x55c355d7d720 2023-07-13T22:22:32.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.426+0000 7f9230bfe700 1 -- 172.21.15.26:0/1638325307 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f923801e070 con 0x55c355d7d720 2023-07-13T22:22:32.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.426+0000 7f9230bfe700 1 -- 172.21.15.26:0/1638325307 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238017650 con 0x55c355d7d720 2023-07-13T22:22:32.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 -- 172.21.15.26:0/1638325307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 msgr2=0x55c355d7db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238005530 comp rx=0 tx=0).stop 2023-07-13T22:22:32.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 -- 172.21.15.26:0/1638325307 shutdown_connections 2023-07-13T22:22:32.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d7f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d7db30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1638325307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d7e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 -- 172.21.15.26:0/1638325307 >> 172.21.15.26:0/1638325307 conn(0x55c355d72ff0 msgr2=0x55c355d78460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.427+0000 7f9257ebab80 1 -- 172.21.15.26:0/1638325307 shutdown_connections 2023-07-13T22:22:32.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.428+0000 7f9257ebab80 1 -- 172.21.15.26:0/1638325307 wait complete. 2023-07-13T22:22:32.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.428+0000 7f9257ebab80 1 -- start start 2023-07-13T22:22:32.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.429+0000 7f9257ebab80 1 Processor -- start 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7e070 0x55c355d87890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355c6eac0 con 0x55c355d7ee30 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355c6e7c0 con 0x55c355d7d720 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355b52920 con 0x55c355d7e070 2023-07-13T22:22:32.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.430+0000 7f9257ebab80 10 client.0 _ll_register_callbacks cb 0x55c355c6c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.431+0000 7f924239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.431+0000 7f924239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:38420/0 (socket says 172.21.15.26:38420) 2023-07-13T22:22:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.431+0000 7f924239b700 1 -- 172.21.15.26:0/358145454 learned_addr learned my addr 172.21.15.26:0/358145454 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:32.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.431+0000 7f9241b9a700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7e070 0x55c355d87890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.432+0000 7f924239b700 1 -- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7e070 msgr2=0x55c355d87890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.432+0000 7f924239b700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7e070 0x55c355d87890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.432+0000 7f924239b700 1 -- 172.21.15.26:0/358145454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 msgr2=0x55c355d8e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.432+0000 7f924239b700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.432+0000 7f924239b700 1 -- 172.21.15.26:0/358145454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c117590 con 0x55c355d7d720 2023-07-13T22:22:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.433+0000 7f924239b700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d50 tx=0x7f9234009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec8c774e41e5e4d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.434+0000 7f9225ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.434+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400e050 con 0x55c355d7d720 2023-07-13T22:22:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.434+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9234016d90 con 0x55c355d7d720 2023-07-13T22:22:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.434+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c355c284f0 con 0x55c355d7d720 2023-07-13T22:22:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.435+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9234007eb0 con 0x55c355d7d720 2023-07-13T22:22:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.435+0000 7f9257ebab80 10 client.26828 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.435+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c355ae5c60 con 0x55c355d7d720 2023-07-13T22:22:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c355c284f0 con 0x55c355d7d720 2023-07-13T22:22:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9234020540 con 0x55c355d7d720 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 10 client.26828 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 10 client.26828 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 20 client.26828 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355da4210 0x55c355da6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 4 client.26828 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.436+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c355cc7eb0 con 0x55c355da4210 2023-07-13T22:22:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.437+0000 7f9241b9a700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355da4210 0x55c355da6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.438+0000 7f9241b9a700 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355da4210 0x55c355da6620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7bb5b4b3be3741a server_cookie=5c512393b4aa6a13 in_seq=0 out_seq=0 2023-07-13T22:22:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.438+0000 7f9225ffb700 10 client.26828 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:22:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.439+0000 7f9225ffb700 1 -- 172.21.15.26:0/358145454 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c355cc7eb0 con 0x55c355da4210 2023-07-13T22:22:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.439+0000 7f9225ffb700 10 client.26828 handle_command_reply: tid=0 2023-07-13T22:22:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.439+0000 7f9257ebab80 1 client.26828 shutdown 2023-07-13T22:22:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.440+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 msgr2=0x55c355d87350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.440+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9234009d50 tx=0x7f9234009920 comp rx=0 tx=0).stop 2023-07-13T22:22:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 shutdown_connections 2023-07-13T22:22:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355da4210 0x55c355da6620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d87350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 --2- 172.21.15.26:0/358145454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7e070 0x55c355d87890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 >> 172.21.15.26:0/358145454 conn(0x55c355d72ff0 msgr2=0x55c355d77b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9225ffb700 0 client.26828 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:22:32.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.441+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 shutdown_connections 2023-07-13T22:22:32.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.442+0000 7f9257ebab80 1 -- 172.21.15.26:0/358145454 wait complete. 2023-07-13T22:22:32.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.442+0000 7f9257ebab80 20 client.26828 trim_cache size 0 max 16384 2023-07-13T22:22:32.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.471+0000 7f9257ebab80 1 Processor -- start 2023-07-13T22:22:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.471+0000 7f9257ebab80 1 -- start start 2023-07-13T22:22:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d85250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7e070 0x55c355d8e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355d72bc0 con 0x55c355d7e070 2023-07-13T22:22:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355d88380 con 0x55c355d7d720 2023-07-13T22:22:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.472+0000 7f9257ebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355da15d0 con 0x55c355d7ee30 2023-07-13T22:22:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.473+0000 7f924239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d85250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.473+0000 7f9242b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.473+0000 7f9242b9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60578/0 (socket says 172.21.15.26:60578) 2023-07-13T22:22:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.473+0000 7f9242b9c700 1 -- 172.21.15.26:0/1981491134 learned_addr learned my addr 172.21.15.26:0/1981491134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 -- 172.21.15.26:0/1981491134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 msgr2=0x55c355d85250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d85250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 -- 172.21.15.26:0/1981491134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7e070 msgr2=0x55c355d8e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7e070 0x55c355d8e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 -- 172.21.15.26:0/1981491134 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c355b23250 con 0x55c355d7ee30 2023-07-13T22:22:32.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.476+0000 7f9242b9c700 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f922c099ae0 tx=0x7f922c099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f7e0f94fc7725b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.490+0000 7f92313ff700 1 -- 172.21.15.26:0/1981491134 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c099de0 con 0x55c355d7ee30 2023-07-13T22:22:32.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.490+0000 7f92313ff700 1 -- 172.21.15.26:0/1981491134 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f922c09de10 con 0x55c355d7ee30 2023-07-13T22:22:32.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.490+0000 7f92313ff700 1 -- 172.21.15.26:0/1981491134 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c007be0 con 0x55c355d7ee30 2023-07-13T22:22:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 -- 172.21.15.26:0/1981491134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 msgr2=0x55c355da6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f922c099ae0 tx=0x7f922c099920 comp rx=0 tx=0).stop 2023-07-13T22:22:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 -- 172.21.15.26:0/1981491134 shutdown_connections 2023-07-13T22:22:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7ee30 0x55c355da6220 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7d720 0x55c355d85250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 --2- 172.21.15.26:0/1981491134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7e070 0x55c355d8e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.494+0000 7f9257ebab80 1 -- 172.21.15.26:0/1981491134 >> 172.21.15.26:0/1981491134 conn(0x55c355d72ff0 msgr2=0x55c355d86010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.495+0000 7f9257ebab80 1 -- 172.21.15.26:0/1981491134 shutdown_connections 2023-07-13T22:22:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.495+0000 7f9257ebab80 1 -- 172.21.15.26:0/1981491134 wait complete. 2023-07-13T22:22:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.495+0000 7f9257ebab80 1 -- start start 2023-07-13T22:22:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 Processor -- start 2023-07-13T22:22:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d8a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d8abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355d8e120 con 0x55c355d7ee30 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355d8e290 con 0x55c355d7e070 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c355d8e450 con 0x55c355d7d720 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.496+0000 7f9257ebab80 10 client.0 _ll_register_callbacks cb 0x7f92340157e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.497+0000 7f924239b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d8a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.497+0000 7f924239b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d8a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:60586/0 (socket says 172.21.15.26:60586) 2023-07-13T22:22:32.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.497+0000 7f924239b700 1 -- 172.21.15.26:0/585859911 learned_addr learned my addr 172.21.15.26:0/585859911 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d8abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 -- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 msgr2=0x55c355d8a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d8a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 -- 172.21.15.26:0/585859911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 msgr2=0x55c355d8dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.498+0000 7f9241b9a700 1 -- 172.21.15.26:0/585859911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c1ac9f0 con 0x55c355d7e070 2023-07-13T22:22:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.499+0000 7f9241b9a700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d8abc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f92380098e0 tx=0x7f9238005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37416d60f3e2fd04 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.500+0000 7f9225ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.500+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238010ee0 con 0x55c355d7e070 2023-07-13T22:22:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.500+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f923800bc30 con 0x55c355d7e070 2023-07-13T22:22:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.501+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f923801e070 con 0x55c355d7e070 2023-07-13T22:22:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.501+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f923800bdf0 con 0x55c355d7e070 2023-07-13T22:22:32.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.501+0000 7f9257ebab80 10 client.26834 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:32.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.502+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c355afb2b0 con 0x55c355d7e070 2023-07-13T22:22:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.502+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f923801e070 con 0x55c355d7e070 2023-07-13T22:22:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.502+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f923802f960 con 0x55c355d7e070 2023-07-13T22:22:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 10 client.26834 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 10 client.26834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:22:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 20 client.26834 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355df8b00 0x55c355dfaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 4 client.26834 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:22:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.503+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c355c8a050 con 0x55c355df8b00 2023-07-13T22:22:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.504+0000 7f924239b700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355df8b00 0x55c355dfaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.505+0000 7f924239b700 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355df8b00 0x55c355dfaee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b88627cf7d297af2 server_cookie=3a89427ba8b36b30 in_seq=0 out_seq=0 2023-07-13T22:22:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.505+0000 7f9225ffb700 10 client.26834 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:22:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.505+0000 7f9225ffb700 1 -- 172.21.15.26:0/585859911 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c355c8a050 con 0x55c355df8b00 2023-07-13T22:22:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.505+0000 7f9225ffb700 10 client.26834 handle_command_reply: tid=0 2023-07-13T22:22:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.505+0000 7f9257ebab80 1 client.26834 shutdown 2023-07-13T22:22:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 msgr2=0x55c355d8abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d8abc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f92380098e0 tx=0x7f9238005bc0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 shutdown_connections 2023-07-13T22:22:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55c355df8b00 0x55c355dfaee0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c355d7ee30 0x55c355d8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c355d7d720 0x55c355d8a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 --2- 172.21.15.26:0/585859911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c355d7e070 0x55c355d8abc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 >> 172.21.15.26:0/585859911 conn(0x55c355d72ff0 msgr2=0x55c355d85250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9225ffb700 0 client.26834 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:22:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 shutdown_connections 2023-07-13T22:22:32.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.506+0000 7f9257ebab80 1 -- 172.21.15.26:0/585859911 wait complete. 2023-07-13T22:22:32.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.507+0000 7f9257ebab80 20 client.26834 trim_cache size 0 max 16384 2023-07-13T22:22:32.495 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:22:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.508+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 msgr2=0x7f92280a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.508+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 0x7f92280a3890 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f923c072710 tx=0x7f923400b040 comp rx=0 tx=0).stop 2023-07-13T22:22:32.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.508+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 msgr2=0x7f923c1ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:32.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.508+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f922c00cab0 tx=0x7f922c00ce70 comp rx=0 tx=0).stop 2023-07-13T22:22:32.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 shutdown_connections 2023-07-13T22:22:32.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c10bba0 0x7f923c1ab9b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f923c0711d0 0x7f923c1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f92280a13e0 0x7f92280a3890 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 --2- 172.21.15.26:0/1352993278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f923c071b20 0x7f923c1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:32.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.509+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 >> 172.21.15.26:0/1352993278 conn(0x7f923c06c8b0 msgr2=0x7f923c06db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.510+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 shutdown_connections 2023-07-13T22:22:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:32.510+0000 7f924339d700 1 -- 172.21.15.26:0/1352993278 wait complete. 2023-07-13T22:22:32.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:22:33.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 -- 172.21.15.26:0/1873862080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1518103670 msgr2=0x7f1518103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 --2- 172.21.15.26:0/1873862080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1518103670 0x7f1518103ae0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f151000b0a0 tx=0x7f151000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 -- 172.21.15.26:0/1873862080 shutdown_connections 2023-07-13T22:22:33.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 --2- 172.21.15.26:0/1873862080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1518103670 0x7f1518103ae0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 --2- 172.21.15.26:0/1873862080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15180fe350 0x7f1518100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 --2- 172.21.15.26:0/1873862080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 0x7f1518103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.120+0000 7f151f7ff700 1 -- 172.21.15.26:0/1873862080 >> 172.21.15.26:0/1873862080 conn(0x7f15180fa000 msgr2=0x7f15180fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.121+0000 7f151f7ff700 1 -- 172.21.15.26:0/1873862080 shutdown_connections 2023-07-13T22:22:33.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.121+0000 7f151f7ff700 1 -- 172.21.15.26:0/1873862080 wait complete. 2023-07-13T22:22:33.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.123+0000 7f151f7ff700 1 Processor -- start 2023-07-13T22:22:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.123+0000 7f151f7ff700 1 -- start start 2023-07-13T22:22:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15180fe350 0x7f15181098b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 0x7f1518109df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151810a3c0 con 0x7f1518100cb0 2023-07-13T22:22:33.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151810a500 con 0x7f1518103670 2023-07-13T22:22:33.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.124+0000 7f151f7ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518069f60 con 0x7f15180fe350 2023-07-13T22:22:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.125+0000 7f151dd9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59886/0 (socket says 172.21.15.26:59886) 2023-07-13T22:22:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 -- 172.21.15.26:0/946265398 learned_addr learned my addr 172.21.15.26:0/946265398 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:33.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151cd9a700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 0x7f1518109df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 -- 172.21.15.26:0/946265398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15180fe350 msgr2=0x7f15181098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15180fe350 0x7f15181098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 -- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 msgr2=0x7f1518109df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 0x7f1518109df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.126+0000 7f151dd9c700 1 -- 172.21.15.26:0/946265398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009d00 con 0x7f1518103670 2023-07-13T22:22:33.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f151dd9c700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f151000b780 tx=0x7f151000bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f150e7fc700 1 -- 172.21.15.26:0/946265398 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151000e040 con 0x7f1518103670 2023-07-13T22:22:33.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f150e7fc700 1 -- 172.21.15.26:0/946265398 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1510004e60 con 0x7f1518103670 2023-07-13T22:22:33.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f150e7fc700 1 -- 172.21.15.26:0/946265398 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510005390 con 0x7f1518103670 2023-07-13T22:22:33.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151806a1e0 con 0x7f1518103670 2023-07-13T22:22:33.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.127+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151806a640 con 0x7f1518103670 2023-07-13T22:22:33.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.139+0000 7f1533164b80 1 Processor -- start 2023-07-13T22:22:33.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.140+0000 7f150e7fc700 1 -- 172.21.15.26:0/946265398 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f15100054f0 con 0x7f1518103670 2023-07-13T22:22:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 -- start start 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1c4d0 0x55f79ab1c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1dc70 0x55f79ab1e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa21cb0 con 0x55f79ab1c4d0 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa74a40 con 0x55f79ab1dc70 2023-07-13T22:22:33.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.142+0000 7f1533164b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa6ded0 con 0x55f79ab1ce20 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.144+0000 7f150e7fc700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 0x7f15040a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.144+0000 7f151d59b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.144+0000 7f151d59b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44176/0 (socket says 172.21.15.26:44176) 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.144+0000 7f151d59b700 1 -- 172.21.15.26:0/2169671053 learned_addr learned my addr 172.21.15.26:0/2169671053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.144+0000 7f150e7fc700 1 -- 172.21.15.26:0/946265398 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f15100cd720 con 0x7f1518103670 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 -- 172.21.15.26:0/2169671053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1dc70 msgr2=0x55f79ab1e0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1dc70 0x55f79ab1e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 -- 172.21.15.26:0/2169671053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1c4d0 msgr2=0x55f79ab1c8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1c4d0 0x55f79ab1c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 -- 172.21.15.26:0/2169671053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79a89bde0 con 0x55f79ab1ce20 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f151d59b700 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1514009cd0 tx=0x7f151400e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731bfbc617eed78d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.146+0000 7f15037fe700 1 -- 172.21.15.26:0/2169671053 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400ec80 con 0x55f79ab1ce20 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.147+0000 7f15037fe700 1 -- 172.21.15.26:0/2169671053 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151400ede0 con 0x55f79ab1ce20 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.147+0000 7f15037fe700 1 -- 172.21.15.26:0/2169671053 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514010430 con 0x55f79ab1ce20 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.148+0000 7f1533164b80 1 -- 172.21.15.26:0/2169671053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 msgr2=0x55f79ab1d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.148+0000 7f1533164b80 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1514009cd0 tx=0x7f151400e350 comp rx=0 tx=0).stop 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f151dd9c700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 0x7f15040a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f1533164b80 1 -- 172.21.15.26:0/2169671053 shutdown_connections 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f1533164b80 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1c4d0 0x55f79ab1c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f1533164b80 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1dc70 0x55f79ab1e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f1533164b80 1 --2- 172.21.15.26:0/2169671053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab1d6e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.149+0000 7f1533164b80 1 -- 172.21.15.26:0/2169671053 >> 172.21.15.26:0/2169671053 conn(0x55f79ab11da0 msgr2=0x55f79ab171d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.152+0000 7f1533164b80 1 -- 172.21.15.26:0/2169671053 shutdown_connections 2023-07-13T22:22:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.175+0000 7f1533164b80 1 -- 172.21.15.26:0/2169671053 wait complete. 2023-07-13T22:22:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.176+0000 7f1533164b80 1 -- start start 2023-07-13T22:22:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.176+0000 7f1533164b80 1 Processor -- start 2023-07-13T22:22:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab26370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa0dac0 con 0x55f79ab1ce20 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa0d7c0 con 0x55f79ab268b0 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79a8f1920 con 0x55f79ab26d20 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.177+0000 7f1533164b80 10 client.0 _ll_register_callbacks cb 0x55f79aa0b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.178+0000 7f151cd9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.178+0000 7f151cd9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44182/0 (socket says 172.21.15.26:44182) 2023-07-13T22:22:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.178+0000 7f151cd9a700 1 -- 172.21.15.26:0/3228202204 learned_addr learned my addr 172.21.15.26:0/3228202204 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab26370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151d59b700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 -- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 msgr2=0x55f79ab2b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 -- 172.21.15.26:0/3228202204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 msgr2=0x55f79ab26370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab26370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.180+0000 7f151cd9a700 1 -- 172.21.15.26:0/3228202204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1514009980 con 0x55f79ab26d20 2023-07-13T22:22:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.184+0000 7f151cd9a700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15080129f0 tx=0x7f1508009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd047ba90e969ca3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.186+0000 7f1500ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:22:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.186+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508004460 con 0x55f79ab26d20 2023-07-13T22:22:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.186+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1508010760 con 0x55f79ab26d20 2023-07-13T22:22:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.193+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f79a9c74f0 con 0x55f79ab26d20 2023-07-13T22:22:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.193+0000 7f151dd9c700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 0x7f15040a37d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f151009ac10 tx=0x7f151000b3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.202+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1508010920 con 0x55f79ab26d20 2023-07-13T22:22:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.202+0000 7f1533164b80 10 client.26842 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:33.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.202+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f79a884c60 con 0x55f79ab26d20 2023-07-13T22:22:33.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.203+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f79a9c74f0 con 0x55f79ab26d20 2023-07-13T22:22:33.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1508003c80 con 0x55f79ab26d20 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 10 client.26842 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 10 client.26842 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 20 client.26842 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab329a0 0x55f79ab34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 4 client.26842 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.204+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f79aa66eb0 con 0x55f79ab329a0 2023-07-13T22:22:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.206+0000 7f151d59b700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab329a0 0x55f79ab34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.206+0000 7f151d59b700 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab329a0 0x55f79ab34db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2330cd3d25223207 server_cookie=73019cd791bb5b01 in_seq=0 out_seq=0 2023-07-13T22:22:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.207+0000 7f1500ff9700 10 client.26842 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:22:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.207+0000 7f1500ff9700 1 -- 172.21.15.26:0/3228202204 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f79aa66eb0 con 0x55f79ab329a0 2023-07-13T22:22:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.207+0000 7f1500ff9700 10 client.26842 handle_command_reply: tid=0 2023-07-13T22:22:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.207+0000 7f1533164b80 1 client.26842 shutdown 2023-07-13T22:22:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 msgr2=0x55f79ab2de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15080129f0 tx=0x7f1508009710 comp rx=0 tx=0).stop 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 shutdown_connections 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab26370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab329a0 0x55f79ab34db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab26d20 0x55f79ab2de70 secure :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f15080129f0 tx=0x7f1508009710 comp rx=0 tx=0).stop 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 --2- 172.21.15.26:0/3228202204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.208+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 >> 172.21.15.26:0/3228202204 conn(0x55f79ab11da0 msgr2=0x55f79ab168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.209+0000 7f1500ff9700 0 client.26842 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:22:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.209+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 shutdown_connections 2023-07-13T22:22:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.209+0000 7f1533164b80 1 -- 172.21.15.26:0/3228202204 wait complete. 2023-07-13T22:22:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.218+0000 7f1533164b80 20 client.26842 trim_cache size 0 max 16384 2023-07-13T22:22:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.243+0000 7f1533164b80 1 Processor -- start 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 -- start start 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab25a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1d460 0x55f79ab2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79aa66eb0 con 0x55f79ab268b0 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79a9c74f0 con 0x55f79ab1d460 2023-07-13T22:22:33.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.244+0000 7f1533164b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ab25870 con 0x55f79ab1ce20 2023-07-13T22:22:33.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.252+0000 7f151d59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.252+0000 7f151d59b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41596/0 (socket says 172.21.15.26:41596) 2023-07-13T22:22:33.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.252+0000 7f151d59b700 1 -- 172.21.15.26:0/375593027 learned_addr learned my addr 172.21.15.26:0/375593027 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:33.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.253+0000 7f151cd9a700 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab25a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.253+0000 7f151cd9a700 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1d460 0x55f79ab2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.263+0000 7f151d59b700 1 -- 172.21.15.26:0/375593027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 msgr2=0x55f79ab25a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.263+0000 7f151d59b700 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab25a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.263+0000 7f151d59b700 1 -- 172.21.15.26:0/375593027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1d460 msgr2=0x55f79ab2e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.263+0000 7f151d59b700 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1d460 0x55f79ab2e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.263+0000 7f151d59b700 1 -- 172.21.15.26:0/375593027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f79a8c2250 con 0x55f79ab268b0 2023-07-13T22:22:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.266+0000 7f151d59b700 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1514005450 tx=0x7f1514009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0fb3c24587f32a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.268+0000 7f1503fff700 1 -- 172.21.15.26:0/375593027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400ea80 con 0x55f79ab268b0 2023-07-13T22:22:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.268+0000 7f1503fff700 1 -- 172.21.15.26:0/375593027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151400ebe0 con 0x55f79ab268b0 2023-07-13T22:22:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.268+0000 7f1503fff700 1 -- 172.21.15.26:0/375593027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514027590 con 0x55f79ab268b0 2023-07-13T22:22:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.269+0000 7f1533164b80 1 -- 172.21.15.26:0/375593027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 msgr2=0x55f79ab2b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.269+0000 7f1533164b80 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1514005450 tx=0x7f1514009260 comp rx=0 tx=0).stop 2023-07-13T22:22:33.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.275+0000 7f1533164b80 1 -- 172.21.15.26:0/375593027 shutdown_connections 2023-07-13T22:22:33.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.275+0000 7f1533164b80 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab1d460 0x55f79ab2e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.275+0000 7f1533164b80 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1ce20 0x55f79ab25a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.275+0000 7f1533164b80 1 --2- 172.21.15.26:0/375593027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab268b0 0x55f79ab2b800 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.275+0000 7f1533164b80 1 -- 172.21.15.26:0/375593027 >> 172.21.15.26:0/375593027 conn(0x55f79ab11da0 msgr2=0x55f79ab241d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.292+0000 7f1533164b80 1 -- 172.21.15.26:0/375593027 shutdown_connections 2023-07-13T22:22:33.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.306+0000 7f1533164b80 1 -- 172.21.15.26:0/375593027 wait complete. 2023-07-13T22:22:33.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.307+0000 7f1533164b80 1 -- start start 2023-07-13T22:22:33.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.307+0000 7f1533164b80 1 Processor -- start 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab29780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ab2b210 con 0x55f79ab1ce20 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ab2b380 con 0x55f79ab268b0 2023-07-13T22:22:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f79ab2b540 con 0x55f79ab1d460 2023-07-13T22:22:33.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.309+0000 7f1533164b80 10 client.0 _ll_register_callbacks cb 0x7f1508012520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:33.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.315+0000 7f151d59b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.315+0000 7f151d59b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44202/0 (socket says 172.21.15.26:44202) 2023-07-13T22:22:33.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.315+0000 7f151d59b700 1 -- 172.21.15.26:0/768098264 learned_addr learned my addr 172.21.15.26:0/768098264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:33.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151cd9a700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151d59b700 1 -- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 msgr2=0x55f79ab2acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151d59b700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151d59b700 1 -- 172.21.15.26:0/768098264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 msgr2=0x55f79ab29780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151d59b700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab29780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.319+0000 7f151d59b700 1 -- 172.21.15.26:0/768098264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f79a89bde0 con 0x55f79ab1d460 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.322+0000 7f151d59b700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15140059d0 tx=0x7f15140092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79dff402d4d37ce server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.323+0000 7f1500ff9700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.323+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514036070 con 0x55f79ab1d460 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.323+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1514014070 con 0x55f79ab1d460 2023-07-13T22:22:33.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.324+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f151400ede0 con 0x55f79ab1d460 2023-07-13T22:22:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.326+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1514005ce0 con 0x55f79ab1d460 2023-07-13T22:22:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.326+0000 7f1533164b80 10 client.26848 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.326+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f79a89a2b0 con 0x55f79ab1d460 2023-07-13T22:22:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.327+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151400ede0 con 0x55f79ab1d460 2023-07-13T22:22:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.327+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1514039020 con 0x55f79ab1d460 2023-07-13T22:22:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 10 client.26848 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 10 client.26848 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:22:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 20 client.26848 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab872b0 0x55f79ab896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 4 client.26848 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:22:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.328+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f79aa29050 con 0x55f79ab872b0 2023-07-13T22:22:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.332+0000 7f151cd9a700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab872b0 0x55f79ab896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.377+0000 7f151cd9a700 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab872b0 0x55f79ab896c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61aca7d415cbabb7 server_cookie=c87caa033f0c0c01 in_seq=0 out_seq=0 2023-07-13T22:22:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.377+0000 7f1500ff9700 10 client.26848 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:22:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.388+0000 7f1500ff9700 1 -- 172.21.15.26:0/768098264 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f79aa29050 con 0x55f79ab872b0 2023-07-13T22:22:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.388+0000 7f1500ff9700 10 client.26848 handle_command_reply: tid=0 2023-07-13T22:22:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.391+0000 7f1533164b80 1 client.26848 shutdown 2023-07-13T22:22:33.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.422+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 msgr2=0x55f79ab29cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.422+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15140059d0 tx=0x7f15140092d0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.423+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 shutdown_connections 2023-07-13T22:22:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.423+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f79ab1ce20 0x55f79ab29780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.424+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55f79ab872b0 0x55f79ab896c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.424+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f79ab268b0 0x55f79ab2acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.424+0000 7f1533164b80 1 --2- 172.21.15.26:0/768098264 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f79ab1d460 0x55f79ab29cc0 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15140059d0 tx=0x7f15140092d0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.424+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 >> 172.21.15.26:0/768098264 conn(0x55f79ab11da0 msgr2=0x55f79ab349b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.428+0000 7f1500ff9700 0 client.26848 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.428+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 shutdown_connections 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.428+0000 7f1533164b80 1 -- 172.21.15.26:0/768098264 wait complete. 2023-07-13T22:22:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.429+0000 7f1533164b80 20 client.26848 trim_cache size 0 max 16384 2023-07-13T22:22:33.438 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 msgr2=0x7f15040a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 0x7f15040a37d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f151009ac10 tx=0x7f151000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 msgr2=0x7f15181abff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f151000b780 tx=0x7f151000bb60 comp rx=0 tx=0).stop 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 shutdown_connections 2023-07-13T22:22:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1518103670 0x7f15181abff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15180fe350 0x7f15181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f15040a1320 0x7f15040a37d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 --2- 172.21.15.26:0/946265398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1518100cb0 0x7f1518109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.466+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 >> 172.21.15.26:0/946265398 conn(0x7f15180fa000 msgr2=0x7f15180feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.467+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 shutdown_connections 2023-07-13T22:22:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:33.467+0000 7f151f7ff700 1 -- 172.21.15.26:0/946265398 wait complete. 2023-07-13T22:22:33.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:22:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:33 smithi117 ceph-mon[129074]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:33.709 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:33 smithi026 ceph-mon[95647]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:33.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:33 smithi121 ceph-mon[129187]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:22:34.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.354+0000 7f6eebfff700 1 -- 172.21.15.26:0/2655603431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4106110 msgr2=0x7f6ee4106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.354+0000 7f6eebfff700 1 --2- 172.21.15.26:0/2655603431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4106110 0x7f6ee4106520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009f90 tx=0x7f6edc00b330 comp rx=0 tx=0).stop 2023-07-13T22:22:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 -- 172.21.15.26:0/2655603431 shutdown_connections 2023-07-13T22:22:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 --2- 172.21.15.26:0/2655603431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4106110 0x7f6ee4106520 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 --2- 172.21.15.26:0/2655603431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4107c90 0x7f6ee410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 --2- 172.21.15.26:0/2655603431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 -- 172.21.15.26:0/2655603431 >> 172.21.15.26:0/2655603431 conn(0x7f6ee4075160 msgr2=0x7f6ee4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 -- 172.21.15.26:0/2655603431 shutdown_connections 2023-07-13T22:22:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.355+0000 7f6eebfff700 1 -- 172.21.15.26:0/2655603431 wait complete. 2023-07-13T22:22:34.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.357+0000 7f6eebfff700 1 Processor -- start 2023-07-13T22:22:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.358+0000 7f6eebfff700 1 -- start start 2023-07-13T22:22:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4106110 0x7f6ee407dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4107c90 0x7f6ee41ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee407e830 con 0x7f6ee4107c90 2023-07-13T22:22:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee407e970 con 0x7f6ee4107300 2023-07-13T22:22:34.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6eebfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41ac690 con 0x7f6ee4106110 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6ee959a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6ee959a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59932/0 (socket says 172.21.15.26:59932) 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6ee959a700 1 -- 172.21.15.26:0/1699372843 learned_addr learned my addr 172.21.15.26:0/1699372843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6ee959a700 1 -- 172.21.15.26:0/1699372843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4106110 msgr2=0x7f6ee407dce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.359+0000 7f6ee959a700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4106110 0x7f6ee407dce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.360+0000 7f6ee959a700 1 -- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4107c90 msgr2=0x7f6ee41ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.360+0000 7f6ee959a700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4107c90 0x7f6ee41ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.360+0000 7f6ee959a700 1 -- 172.21.15.26:0/1699372843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009c40 con 0x7f6ee4107300 2023-07-13T22:22:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.361+0000 7f6ee959a700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400e9a0 tx=0x7f6ed400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.361+0000 7f6edaffd700 1 -- 172.21.15.26:0/1699372843 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400cbd0 con 0x7f6ee4107300 2023-07-13T22:22:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.361+0000 7f6edaffd700 1 -- 172.21.15.26:0/1699372843 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed400cd30 con 0x7f6ee4107300 2023-07-13T22:22:34.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.361+0000 7f6edaffd700 1 -- 172.21.15.26:0/1699372843 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4018680 con 0x7f6ee4107300 2023-07-13T22:22:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.362+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee41ac970 con 0x7f6ee4107300 2023-07-13T22:22:34.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.363+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41ace00 con 0x7f6ee4107300 2023-07-13T22:22:34.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.369+0000 7f6edaffd700 1 -- 172.21.15.26:0/1699372843 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ed40187e0 con 0x7f6ee4107300 2023-07-13T22:22:34.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.370+0000 7f6eff892b80 1 Processor -- start 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 -- start start 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6ae20 0x55f5cda6b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd96fcb0 con 0x55f5cda6bc70 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd9c2a40 con 0x55f5cda6ae20 2023-07-13T22:22:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.371+0000 7f6eff892b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd9bbed0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.372+0000 7f6ee9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6ae20 0x55f5cda6b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.372+0000 7f6edaffd700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 0x7f6ed00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.372+0000 7f6eea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.373+0000 7f6edaffd700 1 -- 172.21.15.26:0/1699372843 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ed4014070 con 0x7f6ee4107300 2023-07-13T22:22:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.373+0000 7f6eea59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44222/0 (socket says 172.21.15.26:44222) 2023-07-13T22:22:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.373+0000 7f6eea59c700 1 -- 172.21.15.26:0/2871341753 learned_addr learned my addr 172.21.15.26:0/2871341753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:34.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.373+0000 7f6ee959a700 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.373+0000 7f6eea59c700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 0x7f6ed00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 -- 172.21.15.26:0/2871341753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6ae20 msgr2=0x55f5cda6b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6ae20 0x55f5cda6b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 -- 172.21.15.26:0/2871341753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 msgr2=0x55f5cda6c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 -- 172.21.15.26:0/2871341753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5cd7e9de0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6ee959a700 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00095a0 tx=0x7f6ee0012db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390ccd210aabec5b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.374+0000 7f6eea59c700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 0x7f6ed00a3800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0010650 tx=0x7f6ee001c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.375+0000 7f6ecffff700 1 -- 172.21.15.26:0/2871341753 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0004230 con 0x55f5cda6a4d0 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.375+0000 7f6ecffff700 1 -- 172.21.15.26:0/2871341753 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee0012510 con 0x55f5cda6a4d0 2023-07-13T22:22:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.375+0000 7f6ecffff700 1 -- 172.21.15.26:0/2871341753 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0005020 con 0x55f5cda6a4d0 2023-07-13T22:22:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 -- 172.21.15.26:0/2871341753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 msgr2=0x55f5cda6a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00095a0 tx=0x7f6ee0012db0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 -- 172.21.15.26:0/2871341753 shutdown_connections 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6a4d0 0x55f5cda6a8e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 --2- 172.21.15.26:0/2871341753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6ae20 0x55f5cda6b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 -- 172.21.15.26:0/2871341753 >> 172.21.15.26:0/2871341753 conn(0x55f5cda5fda0 msgr2=0x55f5cda651d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 -- 172.21.15.26:0/2871341753 shutdown_connections 2023-07-13T22:22:34.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.376+0000 7f6eff892b80 1 -- 172.21.15.26:0/2871341753 wait complete. 2023-07-13T22:22:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.377+0000 7f6eff892b80 1 -- start start 2023-07-13T22:22:34.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.378+0000 7f6eff892b80 1 Processor -- start 2023-07-13T22:22:34.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda74710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd95bac0 con 0x55f5cda6bc70 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd95b7c0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cd83f920 con 0x55f5cda6ae20 2023-07-13T22:22:34.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.380+0000 7f6eff892b80 10 client.0 _ll_register_callbacks cb 0x55f5cd959760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.381+0000 7f6ee9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.381+0000 7f6ee9d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59958/0 (socket says 172.21.15.26:59958) 2023-07-13T22:22:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.381+0000 7f6ee9d9b700 1 -- 172.21.15.26:0/3976201899 learned_addr learned my addr 172.21.15.26:0/3976201899 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:34.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6eea59c700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda74710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 -- 172.21.15.26:0/3976201899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 msgr2=0x55f5cda74710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda74710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 -- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 msgr2=0x55f5cda7b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda7b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 -- 172.21.15.26:0/3976201899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41ace00 con 0x55f5cda6a4d0 2023-07-13T22:22:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4107160 tx=0x7f6edc009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9a0a9861f48031 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.382+0000 7f6ecd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.383+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00e040 con 0x55f5cda6a4d0 2023-07-13T22:22:34.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.383+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6edc003c00 con 0x55f5cda6a4d0 2023-07-13T22:22:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.383+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5cd9154f0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.383+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6edc003dc0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.384+0000 7f6eff892b80 10 client.26855 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.384+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f6ee00126b0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.384+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5cd9154f0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6edc019030 con 0x55f5cda6a4d0 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 10 client.26855 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 10 client.26855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 20 client.26855 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cda7ff80 0x55f5cda82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 4 client.26855 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:34.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.385+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5cd9b4eb0 con 0x55f5cda7ff80 2023-07-13T22:22:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.386+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cda7ff80 0x55f5cda82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.387+0000 7f6ecd7fa700 10 client.26855 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:22:34.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.387+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cda7ff80 0x55f5cda82390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9d76372d02a1898 server_cookie=391ae58ec53f4636 in_seq=0 out_seq=0 2023-07-13T22:22:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.392+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/3976201899 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f5cd9b4eb0 con 0x55f5cda7ff80 2023-07-13T22:22:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.392+0000 7f6ecd7fa700 10 client.26855 handle_command_reply: tid=0 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.392+0000 7f6eff892b80 1 client.26855 shutdown 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 msgr2=0x55f5cda741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4107160 tx=0x7f6edc009510 comp rx=0 tx=0).stop 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 shutdown_connections 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cda7ff80 0x55f5cda82390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda7b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda741d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3976201899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda74710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 >> 172.21.15.26:0/3976201899 conn(0x55f5cda5fda0 msgr2=0x55f5cda648b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.393+0000 7f6ecd7fa700 0 client.26855 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:22:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.394+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 shutdown_connections 2023-07-13T22:22:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.394+0000 7f6eff892b80 1 -- 172.21.15.26:0/3976201899 wait complete. 2023-07-13T22:22:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.395+0000 7f6eff892b80 20 client.26855 trim_cache size 0 max 16384 2023-07-13T22:22:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.423+0000 7f6eff892b80 1 Processor -- start 2023-07-13T22:22:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.423+0000 7f6eff892b80 1 -- start start 2023-07-13T22:22:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda73680 con 0x55f5cda6bc70 2023-07-13T22:22:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda6b1f0 con 0x55f5cda6a4d0 2023-07-13T22:22:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6eff892b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda5f9a0 con 0x55f5cda6ae20 2023-07-13T22:22:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.424+0000 7f6ee959a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.425+0000 7f6ee959a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2023-07-13T22:22:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.425+0000 7f6ee959a700 1 -- 172.21.15.26:0/3687574557 learned_addr learned my addr 172.21.15.26:0/3687574557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.425+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.431+0000 7f6ee959a700 1 -- 172.21.15.26:0/3687574557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 msgr2=0x55f5cda720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.431+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.431+0000 7f6ee959a700 1 -- 172.21.15.26:0/3687574557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 msgr2=0x55f5cda6ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.431+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.431+0000 7f6ee959a700 1 -- 172.21.15.26:0/3687574557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5cd7d2c60 con 0x55f5cda6ae20 2023-07-13T22:22:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.432+0000 7f6ee959a700 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6ed409a120 tx=0x7f6ed401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f4b0d7b3399e304 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.432+0000 7f6ed8ff9700 1 -- 172.21.15.26:0/3687574557 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed409e1c0 con 0x55f5cda6ae20 2023-07-13T22:22:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.432+0000 7f6ed8ff9700 1 -- 172.21.15.26:0/3687574557 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed40ade90 con 0x55f5cda6ae20 2023-07-13T22:22:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.432+0000 7f6ed8ff9700 1 -- 172.21.15.26:0/3687574557 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40998e0 con 0x55f5cda6ae20 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.433+0000 7f6eff892b80 1 -- 172.21.15.26:0/3687574557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 msgr2=0x55f5cda7b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.433+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6ed409a120 tx=0x7f6ed401a040 comp rx=0 tx=0).stop 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 -- 172.21.15.26:0/3687574557 shutdown_connections 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6bc70 0x55f5cda6ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6a4d0 0x55f5cda720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 --2- 172.21.15.26:0/3687574557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda7b710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 -- 172.21.15.26:0/3687574557 >> 172.21.15.26:0/3687574557 conn(0x55f5cda5fda0 msgr2=0x55f5cda72df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 -- 172.21.15.26:0/3687574557 shutdown_connections 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.434+0000 7f6eff892b80 1 -- 172.21.15.26:0/3687574557 wait complete. 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.435+0000 7f6eff892b80 1 -- start start 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.435+0000 7f6eff892b80 1 Processor -- start 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 0x55f5cda7b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda79910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6bc70 0x55f5cda79e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda7a390 con 0x55f5cda6a4d0 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda7a500 con 0x55f5cda6bc70 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5cda7a6c0 con 0x55f5cda6ae20 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.436+0000 7f6eff892b80 10 client.0 _ll_register_callbacks cb 0x7f6edc009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.437+0000 7f6eea59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6bc70 0x55f5cda79e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.439+0000 7f6ee9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 0x55f5cda7b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.439+0000 7f6ee9d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 0x55f5cda7b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41676/0 (socket says 172.21.15.26:41676) 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.439+0000 7f6ee9d9b700 1 -- 172.21.15.26:0/4184443577 learned_addr learned my addr 172.21.15.26:0/4184443577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.439+0000 7f6ee959a700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda79910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 -- 172.21.15.26:0/4184443577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6bc70 msgr2=0x55f5cda79e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6bc70 0x55f5cda79e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 -- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 msgr2=0x55f5cda7b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 0x55f5cda7b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 -- 172.21.15.26:0/4184443577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41ac970 con 0x55f5cda6ae20 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.440+0000 7f6ee959a700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda79910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55f5cd915510 tx=0x7f6ed4098530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f3127648b859fb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.442+0000 7f6ecd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.442+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40b8070 con 0x55f5cda6ae20 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.442+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6ed40982d0 con 0x55f5cda6ae20 2023-07-13T22:22:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.442+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee0012510 con 0x55f5cda6ae20 2023-07-13T22:22:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.443+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed409e1c0 con 0x55f5cda6ae20 2023-07-13T22:22:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.443+0000 7f6eff892b80 10 client.26863 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.443+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f5cd810250 con 0x55f5cda6ae20 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee0012510 con 0x55f5cda6ae20 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ed4095f80 con 0x55f5cda6ae20 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 10 client.26863 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 10 client.26863 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 20 client.26863 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cdad4fa0 0x55f5cdad73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:34.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 4 client.26863 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.444+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5cd977050 con 0x55f5cdad4fa0 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.445+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cdad4fa0 0x55f5cdad73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.446+0000 7f6ee9d9b700 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cdad4fa0 0x55f5cdad73b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe7ff8078ceda181 server_cookie=3f1f74cbd7dceb04 in_seq=0 out_seq=0 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.446+0000 7f6ecd7fa700 10 client.26863 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.446+0000 7f6ecd7fa700 1 -- 172.21.15.26:0/4184443577 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5cd977050 con 0x55f5cdad4fa0 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.446+0000 7f6ecd7fa700 10 client.26863 handle_command_reply: tid=0 2023-07-13T22:22:34.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.446+0000 7f6eff892b80 1 client.26863 shutdown 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 msgr2=0x55f5cda79910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda79910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55f5cd915510 tx=0x7f6ed4098530 comp rx=0 tx=0).stop 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 shutdown_connections 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55f5cdad4fa0 0x55f5cdad73b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f5cda6a4d0 0x55f5cda7b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5cda6bc70 0x55f5cda79e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 --2- 172.21.15.26:0/4184443577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5cda6ae20 0x55f5cda79910 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.447+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 >> 172.21.15.26:0/4184443577 conn(0x55f5cda5fda0 msgr2=0x55f5cda720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.448+0000 7f6ecd7fa700 0 client.26863 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:22:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.448+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 shutdown_connections 2023-07-13T22:22:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.448+0000 7f6eff892b80 1 -- 172.21.15.26:0/4184443577 wait complete. 2023-07-13T22:22:34.460 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:22:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.468+0000 7f6eff892b80 20 client.26863 trim_cache size 0 max 16384 2023-07-13T22:22:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.480+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 msgr2=0x7f6ed00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.480+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 0x7f6ed00a3800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0010650 tx=0x7f6ee001c000 comp rx=0 tx=0).stop 2023-07-13T22:22:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.480+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 msgr2=0x7f6ee407e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.480+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400e9a0 tx=0x7f6ed400ed60 comp rx=0 tx=0).stop 2023-07-13T22:22:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 shutdown_connections 2023-07-13T22:22:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6ed00a1350 0x7f6ed00a3800 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ee4107c90 0x7f6ee41ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ee4106110 0x7f6ee407dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 --2- 172.21.15.26:0/1699372843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ee4107300 0x7f6ee407e220 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.481+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 >> 172.21.15.26:0/1699372843 conn(0x7f6ee4075160 msgr2=0x7f6ee40773f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.483+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 shutdown_connections 2023-07-13T22:22:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:34.483+0000 7f6eebfff700 1 -- 172.21.15.26:0/1699372843 wait complete. 2023-07-13T22:22:34.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:22:35.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.163+0000 7f172cbff700 1 -- 172.21.15.26:0/1159561364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 msgr2=0x7f17280712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.163+0000 7f172cbff700 1 --2- 172.21.15.26:0/1159561364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 0x7f17280712f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f172000b0a0 tx=0x7f172000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.164+0000 7f172cbff700 1 -- 172.21.15.26:0/1159561364 shutdown_connections 2023-07-13T22:22:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.164+0000 7f172cbff700 1 --2- 172.21.15.26:0/1159561364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f172810bd60 0x7f172810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.164+0000 7f172cbff700 1 --2- 172.21.15.26:0/1159561364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 0x7f17280712f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.164+0000 7f172cbff700 1 --2- 172.21.15.26:0/1159561364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1728071830 0x7f172810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.164+0000 7f172cbff700 1 -- 172.21.15.26:0/1159561364 >> 172.21.15.26:0/1159561364 conn(0x7f172806ca90 msgr2=0x7f172806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.168+0000 7f172cbff700 1 -- 172.21.15.26:0/1159561364 shutdown_connections 2023-07-13T22:22:35.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.174+0000 7f172cbff700 1 -- 172.21.15.26:0/1159561364 wait complete. 2023-07-13T22:22:35.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.176+0000 7f172cbff700 1 Processor -- start 2023-07-13T22:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.183+0000 7f172cbff700 1 -- start start 2023-07-13T22:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 0x7f1728114c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1728071830 0x7f1728115180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728116790 con 0x7f1728071830 2023-07-13T22:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728116900 con 0x7f172810bd60 2023-07-13T22:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.184+0000 7f172cbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728116ac0 con 0x7f1728070ee0 2023-07-13T22:22:35.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.185+0000 7f1726d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59978/0 (socket says 172.21.15.26:59978) 2023-07-13T22:22:35.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 -- 172.21.15.26:0/1157325239 learned_addr learned my addr 172.21.15.26:0/1157325239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 -- 172.21.15.26:0/1157325239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 msgr2=0x7f1728114c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:22:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 0x7f1728114c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 -- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1728071830 msgr2=0x7f1728115180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:22:35.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.186+0000 7f1726d9d700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1728071830 0x7f1728115180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.187+0000 7f1726d9d700 1 -- 172.21.15.26:0/1157325239 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720009d00 con 0x7f172810bd60 2023-07-13T22:22:35.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.187+0000 7f1726d9d700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b550 tx=0x7f171c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.187+0000 7f17177fe700 1 -- 172.21.15.26:0/1157325239 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c011830 con 0x7f172810bd60 2023-07-13T22:22:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.187+0000 7f17177fe700 1 -- 172.21.15.26:0/1157325239 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171c011e50 con 0x7f172810bd60 2023-07-13T22:22:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.187+0000 7f17177fe700 1 -- 172.21.15.26:0/1157325239 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c010ea0 con 0x7f172810bd60 2023-07-13T22:22:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.188+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1728116d40 con 0x7f172810bd60 2023-07-13T22:22:35.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.188+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1728117280 con 0x7f172810bd60 2023-07-13T22:22:35.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.196+0000 7f1740620b80 1 Processor -- start 2023-07-13T22:22:35.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.200+0000 7f1740620b80 1 -- start start 2023-07-13T22:22:35.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a02746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0274c70 0x5627a02750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a0178cb0 con 0x5627a0273e20 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a01cba40 con 0x5627a0274c70 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.201+0000 7f1740620b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a01c4ed0 con 0x5627a02734d0 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.202+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.202+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44268/0 (socket says 172.21.15.26:44268) 2023-07-13T22:22:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.202+0000 7f1725d9b700 1 -- 172.21.15.26:0/3180167239 learned_addr learned my addr 172.21.15.26:0/3180167239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0274c70 0x5627a02750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 -- 172.21.15.26:0/3180167239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0274c70 msgr2=0x5627a02750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0274c70 0x5627a02750e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 -- 172.21.15.26:0/3180167239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 msgr2=0x5627a02746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a02746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.203+0000 7f1725d9b700 1 -- 172.21.15.26:0/3180167239 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56279fff2de0 con 0x5627a02734d0 2023-07-13T22:22:35.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.204+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17180121e0 tx=0x7f171800b500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a594cc6501a218f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.215+0000 7f17177fe700 1 -- 172.21.15.26:0/1157325239 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f171c011990 con 0x7f172810bd60 2023-07-13T22:22:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.215+0000 7f1714ff9700 1 -- 172.21.15.26:0/3180167239 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718012c80 con 0x5627a02734d0 2023-07-13T22:22:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.215+0000 7f1714ff9700 1 -- 172.21.15.26:0/3180167239 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1718012de0 con 0x5627a02734d0 2023-07-13T22:22:35.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.215+0000 7f1714ff9700 1 -- 172.21.15.26:0/3180167239 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180173c0 con 0x5627a02734d0 2023-07-13T22:22:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.216+0000 7f1740620b80 1 -- 172.21.15.26:0/3180167239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 msgr2=0x5627a02738e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.216+0000 7f1740620b80 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17180121e0 tx=0x7f171800b500 comp rx=0 tx=0).stop 2023-07-13T22:22:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.218+0000 7f1740620b80 1 -- 172.21.15.26:0/3180167239 shutdown_connections 2023-07-13T22:22:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.218+0000 7f1740620b80 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0274c70 0x5627a02750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.218+0000 7f1740620b80 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a02738e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.218+0000 7f1740620b80 1 --2- 172.21.15.26:0/3180167239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a02746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.218+0000 7f1740620b80 1 -- 172.21.15.26:0/3180167239 >> 172.21.15.26:0/3180167239 conn(0x5627a0268da0 msgr2=0x5627a026e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.219+0000 7f17177fe700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 0x7f17100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.219+0000 7f1740620b80 1 -- 172.21.15.26:0/3180167239 shutdown_connections 2023-07-13T22:22:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.219+0000 7f17177fe700 1 -- 172.21.15.26:0/1157325239 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f171c0cc2e0 con 0x7f172810bd60 2023-07-13T22:22:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.219+0000 7f1740620b80 1 -- 172.21.15.26:0/3180167239 wait complete. 2023-07-13T22:22:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.220+0000 7f1740620b80 1 -- start start 2023-07-13T22:22:35.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.220+0000 7f172659c700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 0x7f17100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.220+0000 7f1740620b80 1 Processor -- start 2023-07-13T22:22:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a027d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a0284760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a0164ac0 con 0x5627a0274c70 2023-07-13T22:22:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a01647c0 con 0x5627a0273e20 2023-07-13T22:22:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a0048920 con 0x5627a02734d0 2023-07-13T22:22:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.221+0000 7f1740620b80 10 client.0 _ll_register_callbacks cb 0x5627a0162760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:35.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59994/0 (socket says 172.21.15.26:59994) 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 -- 172.21.15.26:0/3752670981 learned_addr learned my addr 172.21.15.26:0/3752670981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 -- 172.21.15.26:0/3752670981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 msgr2=0x5627a027d1e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a027d1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 -- 172.21.15.26:0/3752670981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 msgr2=0x5627a0284760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a0284760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.222+0000 7f1725d9b700 1 -- 172.21.15.26:0/3752670981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1718013040 con 0x5627a0273e20 2023-07-13T22:22:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.223+0000 7f1725d9b700 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f17180121b0 tx=0x7f1718004ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77946378ffcd6ee1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.224+0000 7f170dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:35.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.224+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718012c80 con 0x5627a0273e20 2023-07-13T22:22:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.224+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f171800b840 con 0x5627a0273e20 2023-07-13T22:22:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.224+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627a011e4f0 con 0x5627a0273e20 2023-07-13T22:22:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.225+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1718004e00 con 0x5627a0273e20 2023-07-13T22:22:35.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.225+0000 7f1740620b80 10 client.26867 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.225+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56279ffdbc60 con 0x5627a0273e20 2023-07-13T22:22:35.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.226+0000 7f172659c700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 0x7f17100a3710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f172000b070 tx=0x7f172000b3b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.226+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627a011e4f0 con 0x5627a0273e20 2023-07-13T22:22:35.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f171802a070 con 0x5627a0273e20 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f1740620b80 10 client.26867 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f1740620b80 10 client.26867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f1740620b80 20 client.26867 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a0288db0 0x5627a028b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.227+0000 7f1740620b80 4 client.26867 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.228+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627a01bdeb0 con 0x5627a0288db0 2023-07-13T22:22:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.228+0000 7f172659c700 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a0288db0 0x5627a028b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.230+0000 7f172659c700 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a0288db0 0x5627a028b1c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adddd028c3a5a4bd server_cookie=82c6dec5917dc747 in_seq=0 out_seq=0 2023-07-13T22:22:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.230+0000 7f170dffb700 10 client.26867 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.230+0000 7f170dffb700 1 -- 172.21.15.26:0/3752670981 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5627a01bdeb0 con 0x5627a0288db0 2023-07-13T22:22:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.230+0000 7f170dffb700 10 client.26867 handle_command_reply: tid=0 2023-07-13T22:22:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.231+0000 7f1740620b80 1 client.26867 shutdown 2023-07-13T22:22:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.231+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 msgr2=0x5627a027d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.231+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f17180121b0 tx=0x7f1718004ae0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 shutdown_connections 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a0284760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a0288db0 0x5627a028b1c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a02734d0 0x5627a027d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 --2- 172.21.15.26:0/3752670981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a0273e20 0x5627a027d720 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 >> 172.21.15.26:0/3752670981 conn(0x5627a0268da0 msgr2=0x5627a026d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f170dffb700 0 client.26867 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 shutdown_connections 2023-07-13T22:22:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.232+0000 7f1740620b80 1 -- 172.21.15.26:0/3752670981 wait complete. 2023-07-13T22:22:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.233+0000 7f1740620b80 20 client.26867 trim_cache size 0 max 16384 2023-07-13T22:22:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.261+0000 7f1740620b80 1 Processor -- start 2023-07-13T22:22:35.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 -- start start 2023-07-13T22:22:35.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a027b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a027d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a02689c0 con 0x5627a0274c70 2023-07-13T22:22:35.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a026f330 con 0x5627a02734d0 2023-07-13T22:22:35.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.268+0000 7f1740620b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a027e270 con 0x5627a0273e20 2023-07-13T22:22:35.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.269+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.270+0000 7f1725d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44274/0 (socket says 172.21.15.26:44274) 2023-07-13T22:22:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.270+0000 7f1725d9b700 1 -- 172.21.15.26:0/1290146461 learned_addr learned my addr 172.21.15.26:0/1290146461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:35.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.270+0000 7f172659c700 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a027b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.273+0000 7f1725d9b700 1 -- 172.21.15.26:0/1290146461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 msgr2=0x5627a027b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.273+0000 7f1725d9b700 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a027b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.273+0000 7f1725d9b700 1 -- 172.21.15.26:0/1290146461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 msgr2=0x5627a027d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.273+0000 7f1725d9b700 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a027d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.273+0000 7f1725d9b700 1 -- 172.21.15.26:0/1290146461 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627a0019250 con 0x5627a0273e20 2023-07-13T22:22:35.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.274+0000 7f1725d9b700 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f17180069c0 tx=0x7f1718022990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b8c091b9f52be6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.285+0000 7f17157fa700 1 -- 172.21.15.26:0/1290146461 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718022a60 con 0x5627a0273e20 2023-07-13T22:22:35.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.285+0000 7f17157fa700 1 -- 172.21.15.26:0/1290146461 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17180062d0 con 0x5627a0273e20 2023-07-13T22:22:35.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.285+0000 7f17157fa700 1 -- 172.21.15.26:0/1290146461 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171800b660 con 0x5627a0273e20 2023-07-13T22:22:35.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.286+0000 7f1740620b80 1 -- 172.21.15.26:0/1290146461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 msgr2=0x5627a0284720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.286+0000 7f1740620b80 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f17180069c0 tx=0x7f1718022990 comp rx=0 tx=0).stop 2023-07-13T22:22:35.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 -- 172.21.15.26:0/1290146461 shutdown_connections 2023-07-13T22:22:35.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0274c70 0x5627a027d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a027b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 --2- 172.21.15.26:0/1290146461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0273e20 0x5627a0284720 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 -- 172.21.15.26:0/1290146461 >> 172.21.15.26:0/1290146461 conn(0x5627a0268da0 msgr2=0x5627a027bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 -- 172.21.15.26:0/1290146461 shutdown_connections 2023-07-13T22:22:35.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.288+0000 7f1740620b80 1 -- 172.21.15.26:0/1290146461 wait complete. 2023-07-13T22:22:35.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.289+0000 7f1740620b80 1 -- start start 2023-07-13T22:22:35.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.291+0000 7f1740620b80 1 Processor -- start 2023-07-13T22:22:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.291+0000 7f1740620b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a0280950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a0280f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 0x5627a0283c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a0284190 con 0x5627a0273e20 2023-07-13T22:22:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a0284300 con 0x5627a02734d0 2023-07-13T22:22:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627a02844c0 con 0x5627a0274c70 2023-07-13T22:22:35.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.292+0000 7f1740620b80 10 client.0 _ll_register_callbacks cb 0x7f171800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f1726d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 0x5627a0283c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f1726d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 0x5627a0283c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44284/0 (socket says 172.21.15.26:44284) 2023-07-13T22:22:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f1726d9d700 1 -- 172.21.15.26:0/332186216 learned_addr learned my addr 172.21.15.26:0/332186216 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f172659c700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a0280950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f172659c700 1 -- 172.21.15.26:0/332186216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 msgr2=0x5627a0283c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f172659c700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 0x5627a0283c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f172659c700 1 -- 172.21.15.26:0/332186216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 msgr2=0x5627a0280f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.293+0000 7f172659c700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a0280f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.294+0000 7f172659c700 1 -- 172.21.15.26:0/332186216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56279ffdbc60 con 0x5627a02734d0 2023-07-13T22:22:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.295+0000 7f172659c700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a0280950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5627a028baf0 tx=0x7f17200099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9e82486c8098be server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.296+0000 7f170dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.296+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000e040 con 0x5627a02734d0 2023-07-13T22:22:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.296+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1720028e20 con 0x5627a02734d0 2023-07-13T22:22:35.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.296+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1718012de0 con 0x5627a02734d0 2023-07-13T22:22:35.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.296+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1720028620 con 0x5627a02734d0 2023-07-13T22:22:35.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.301+0000 7f1740620b80 10 client.26873 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:35.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.301+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56279fff12b0 con 0x5627a02734d0 2023-07-13T22:22:35.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1718012de0 con 0x5627a02734d0 2023-07-13T22:22:35.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f172001b810 con 0x5627a02734d0 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 10 client.26873 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 20 client.26873 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a02dde30 0x5627a02e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 4 client.26873 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:22:35.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.305+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5627a0180050 con 0x5627a02dde30 2023-07-13T22:22:35.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.313+0000 7f1725d9b700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a02dde30 0x5627a02e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:35.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.315+0000 7f1725d9b700 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a02dde30 0x5627a02e0240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68f187f9ee9af84 server_cookie=3cc1b4622f4d4d98 in_seq=0 out_seq=0 2023-07-13T22:22:35.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.315+0000 7f170dffb700 10 client.26873 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.318+0000 7f170dffb700 1 -- 172.21.15.26:0/332186216 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5627a0180050 con 0x5627a02dde30 2023-07-13T22:22:35.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.318+0000 7f170dffb700 10 client.26873 handle_command_reply: tid=0 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.321+0000 7f1740620b80 1 client.26873 shutdown 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 msgr2=0x5627a0280950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a0280950 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x5627a028baf0 tx=0x7f17200099e0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 shutdown_connections 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5627a02dde30 0x5627a02e0240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5627a0274c70 0x5627a0283c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5627a02734d0 0x5627a0280950 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 --2- 172.21.15.26:0/332186216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5627a0273e20 0x5627a0280f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 >> 172.21.15.26:0/332186216 conn(0x5627a0268da0 msgr2=0x5627a027b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.322+0000 7f170dffb700 0 client.26873 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.323+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 shutdown_connections 2023-07-13T22:22:35.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.323+0000 7f1740620b80 1 -- 172.21.15.26:0/332186216 wait complete. 2023-07-13T22:22:35.310 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:22:35.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.326+0000 7f1740620b80 20 client.26873 trim_cache size 0 max 16384 2023-07-13T22:22:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 msgr2=0x7f17100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 0x7f17100a3710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f172000b070 tx=0x7f172000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 msgr2=0x7f17281161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b550 tx=0x7f171c00b910 comp rx=0 tx=0).stop 2023-07-13T22:22:35.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 shutdown_connections 2023-07-13T22:22:35.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f172810bd60 0x7f17281161c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1728070ee0 0x7f1728114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f17100a1260 0x7f17100a3710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 --2- 172.21.15.26:0/1157325239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1728071830 0x7f1728115180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:35.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.328+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 >> 172.21.15.26:0/1157325239 conn(0x7f172806ca90 msgr2=0x7f172806de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.329+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 shutdown_connections 2023-07-13T22:22:35.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:35.329+0000 7f172cbff700 1 -- 172.21.15.26:0/1157325239 wait complete. 2023-07-13T22:22:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:35 smithi117 ceph-mon[129074]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 108 op/s 2023-07-13T22:22:35.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:35 smithi026 ceph-mon[95647]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 108 op/s 2023-07-13T22:22:35.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:35 smithi121 ceph-mon[129187]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 337 MiB/s wr, 108 op/s 2023-07-13T22:22:36.352 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:22:36.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:22:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.816+0000 7f96c59ff700 1 -- 172.21.15.26:0/1341051522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c010bd60 msgr2=0x7f96c010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.816+0000 7f96c59ff700 1 --2- 172.21.15.26:0/1341051522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c010bd60 0x7f96c010e1f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009f90 tx=0x7f96b800b330 comp rx=0 tx=0).stop 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 -- 172.21.15.26:0/1341051522 shutdown_connections 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 --2- 172.21.15.26:0/1341051522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c0070ee0 0x7f96c00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 --2- 172.21.15.26:0/1341051522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c010bd60 0x7f96c010e1f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 --2- 172.21.15.26:0/1341051522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0071830 0x7f96c010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 -- 172.21.15.26:0/1341051522 >> 172.21.15.26:0/1341051522 conn(0x7f96c006ca90 msgr2=0x7f96c006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 -- 172.21.15.26:0/1341051522 shutdown_connections 2023-07-13T22:22:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.817+0000 7f96c59ff700 1 -- 172.21.15.26:0/1341051522 wait complete. 2023-07-13T22:22:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.819+0000 7f96c59ff700 1 Processor -- start 2023-07-13T22:22:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.819+0000 7f96c59ff700 1 -- start start 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 0x7f96c01a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 0x7f96c01abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01ac320 con 0x7f96c010bd60 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01ac460 con 0x7f96c0070ee0 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96c59ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01ac5f0 con 0x7f96c0071830 2023-07-13T22:22:36.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.820+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.821+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60026/0 (socket says 172.21.15.26:60026) 2023-07-13T22:22:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.821+0000 7f96beffd700 1 -- 172.21.15.26:0/57702592 learned_addr learned my addr 172.21.15.26:0/57702592 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.821+0000 7f96be7fc700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 0x7f96c01a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.821+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 0x7f96c01abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96be7fc700 1 -- 172.21.15.26:0/57702592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 msgr2=0x7f96c01a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96be7fc700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96be7fc700 1 -- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 msgr2=0x7f96c01abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96be7fc700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 0x7f96c01abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96be7fc700 1 -- 172.21.15.26:0/57702592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009c40 con 0x7f96c0071830 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96beffd700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:36.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.823+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 0x7f96c01abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:22:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.824+0000 7f96be7fc700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 0x7f96c01a4ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005fd0 tx=0x7f96b400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.825+0000 7f96c49fd700 1 -- 172.21.15.26:0/57702592 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4004220 con 0x7f96c0071830 2023-07-13T22:22:36.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.825+0000 7f96c49fd700 1 -- 172.21.15.26:0/57702592 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b4004380 con 0x7f96c0071830 2023-07-13T22:22:36.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.825+0000 7f96c49fd700 1 -- 172.21.15.26:0/57702592 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4003b10 con 0x7f96c0071830 2023-07-13T22:22:36.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.826+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c01ac8d0 con 0x7f96c0071830 2023-07-13T22:22:36.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.826+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c01ace10 con 0x7f96c0071830 2023-07-13T22:22:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.836+0000 7f96d9410b80 1 Processor -- start 2023-07-13T22:22:36.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.840+0000 7f96d9410b80 1 -- start start 2023-07-13T22:22:36.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 0x5568c3e9e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3e9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3da3d30 con 0x5568c3e9fcf0 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3df6ac0 con 0x5568c3e9eea0 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.841+0000 7f96d9410b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3deff50 con 0x5568c3e9e550 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.842+0000 7f96c49fd700 1 -- 172.21.15.26:0/57702592 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f96b4003c70 con 0x7f96c0071830 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.845+0000 7f96c49fd700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 0x7f96a80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.845+0000 7f96c49fd700 1 -- 172.21.15.26:0/57702592 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96b4014070 con 0x7f96c0071830 2023-07-13T22:22:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.846+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3e9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.846+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3e9f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60042/0 (socket says 172.21.15.26:60042) 2023-07-13T22:22:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.846+0000 7f96beffd700 1 -- 172.21.15.26:0/1800591053 learned_addr learned my addr 172.21.15.26:0/1800591053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:36.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.847+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 0x5568c3e9e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 0x7f96a80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 -- 172.21.15.26:0/1800591053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 msgr2=0x5568c3e9f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3e9f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 -- 172.21.15.26:0/1800591053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9fcf0 msgr2=0x5568c3ea0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.848+0000 7f96bf7fe700 1 -- 172.21.15.26:0/1800591053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c3c1de60 con 0x5568c3e9e550 2023-07-13T22:22:36.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.870+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 0x5568c3e9e960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f96b8007900 tx=0x7f96b80077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d77e19dfd427c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:36.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.881+0000 7f96a5ffb700 1 -- 172.21.15.26:0/1800591053 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800e040 con 0x5568c3e9e550 2023-07-13T22:22:36.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.881+0000 7f96a5ffb700 1 -- 172.21.15.26:0/1800591053 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8021460 con 0x5568c3e9e550 2023-07-13T22:22:36.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.881+0000 7f96a5ffb700 1 -- 172.21.15.26:0/1800591053 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80052a0 con 0x5568c3e9e550 2023-07-13T22:22:36.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.883+0000 7f96d9410b80 1 -- 172.21.15.26:0/1800591053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 msgr2=0x5568c3e9e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.883+0000 7f96d9410b80 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 0x5568c3e9e960 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f96b8007900 tx=0x7f96b80077d0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 0x7f96a80a3800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009040 tx=0x7f96b8005540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:36.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96d9410b80 1 -- 172.21.15.26:0/1800591053 shutdown_connections 2023-07-13T22:22:36.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96d9410b80 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96d9410b80 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9e550 0x5568c3e9e960 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96d9410b80 1 --2- 172.21.15.26:0/1800591053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3e9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.885+0000 7f96d9410b80 1 -- 172.21.15.26:0/1800591053 >> 172.21.15.26:0/1800591053 conn(0x5568c3e93e20 msgr2=0x5568c3e99250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:36.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.895+0000 7f96d9410b80 1 -- 172.21.15.26:0/1800591053 shutdown_connections 2023-07-13T22:22:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.896+0000 7f96d9410b80 1 -- 172.21.15.26:0/1800591053 wait complete. 2023-07-13T22:22:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.897+0000 7f96d9410b80 1 -- start start 2023-07-13T22:22:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.898+0000 7f96d9410b80 1 Processor -- start 2023-07-13T22:22:36.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9eea0 0x5568c3ea8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3d8fb40 con 0x5568c3e9eea0 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3d8f840 con 0x5568c3e9e550 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3c739a0 con 0x5568c3e9fcf0 2023-07-13T22:22:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.899+0000 7f96d9410b80 10 client.0 _ll_register_callbacks cb 0x5568c3d8d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:36.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.900+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.900+0000 7f96beffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60048/0 (socket says 172.21.15.26:60048) 2023-07-13T22:22:36.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.900+0000 7f96beffd700 1 -- 172.21.15.26:0/2646448273 learned_addr learned my addr 172.21.15.26:0/2646448273 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:36.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.900+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.901+0000 7f96beffd700 1 -- 172.21.15.26:0/2646448273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 msgr2=0x5568c3eaf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.901+0000 7f96beffd700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.901+0000 7f96beffd700 1 -- 172.21.15.26:0/2646448273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9eea0 msgr2=0x5568c3ea8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.901+0000 7f96beffd700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9eea0 0x5568c3ea8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.901+0000 7f96beffd700 1 -- 172.21.15.26:0/2646448273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c0117640 con 0x5568c3e9e550 2023-07-13T22:22:36.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.902+0000 7f96beffd700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f96b0011da0 tx=0x7f96b000f520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae69506e4eb5eec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:36.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.904+0000 7f969effd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:36.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.904+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0009e20 con 0x5568c3e9e550 2023-07-13T22:22:36.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.904+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96b000fb30 con 0x5568c3e9e550 2023-07-13T22:22:36.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.905+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568c3d49570 con 0x5568c3e9e550 2023-07-13T22:22:36.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b000fcf0 con 0x5568c3e9e550 2023-07-13T22:22:36.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f96d9410b80 10 client.26885 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:36.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5568c3c06ce0 con 0x5568c3e9e550 2023-07-13T22:22:36.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568c3d49570 con 0x5568c3e9e550 2023-07-13T22:22:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96b001b020 con 0x5568c3e9e550 2023-07-13T22:22:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f96d9410b80 10 client.26885 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.906+0000 7f96d9410b80 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.907+0000 7f96d9410b80 20 client.26885 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:36.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.907+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3eb3f90 0x5568c3eb63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.907+0000 7f96d9410b80 4 client.26885 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:36.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.907+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568c3de8f30 con 0x5568c3eb3f90 2023-07-13T22:22:36.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.907+0000 7f96be7fc700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3eb3f90 0x5568c3eb63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.910+0000 7f96be7fc700 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3eb3f90 0x5568c3eb63a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bcc7c64ccc5195e server_cookie=966db89921300d12 in_seq=0 out_seq=0 2023-07-13T22:22:36.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.911+0000 7f969effd700 10 client.26885 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:36.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.911+0000 7f969effd700 1 -- 172.21.15.26:0/2646448273 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5568c3de8f30 con 0x5568c3eb3f90 2023-07-13T22:22:36.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.911+0000 7f969effd700 10 client.26885 handle_command_reply: tid=0 2023-07-13T22:22:36.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.912+0000 7f96d9410b80 1 client.26885 shutdown 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.912+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 msgr2=0x5568c3ea8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.912+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f96b0011da0 tx=0x7f96b000f520 comp rx=0 tx=0).stop 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 shutdown_connections 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3eb3f90 0x5568c3eb63a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9e550 0x5568c3ea8200 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 --2- 172.21.15.26:0/2646448273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9eea0 0x5568c3ea8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 >> 172.21.15.26:0/2646448273 conn(0x5568c3e93e20 msgr2=0x5568c3e98930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:36.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f969effd700 0 client.26885 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:36.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 shutdown_connections 2023-07-13T22:22:36.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.913+0000 7f96d9410b80 1 -- 172.21.15.26:0/2646448273 wait complete. 2023-07-13T22:22:36.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.914+0000 7f96d9410b80 20 client.26885 trim_cache size 0 max 16384 2023-07-13T22:22:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.960+0000 7f96d9410b80 1 Processor -- start 2023-07-13T22:22:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.961+0000 7f96d9410b80 1 -- start start 2023-07-13T22:22:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.962+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.963+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9eea0 0x5568c3eaf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.963+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.963+0000 7f96d9410b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3ea7dd0 con 0x5568c3e9e550 2023-07-13T22:22:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.963+0000 7f96d9410b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3de8f30 con 0x5568c3e9fcf0 2023-07-13T22:22:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.963+0000 7f96d9410b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3d49570 con 0x5568c3e9eea0 2023-07-13T22:22:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.968+0000 7f96be7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9eea0 0x5568c3eaf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.969+0000 7f96bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.969+0000 7f96bf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60050/0 (socket says 172.21.15.26:60050) 2023-07-13T22:22:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.969+0000 7f96bf7fe700 1 -- 172.21.15.26:0/937593584 learned_addr learned my addr 172.21.15.26:0/937593584 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.970+0000 7f96bf7fe700 1 -- 172.21.15.26:0/937593584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9eea0 msgr2=0x5568c3eaf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.970+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9eea0 0x5568c3eaf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.970+0000 7f96bf7fe700 1 -- 172.21.15.26:0/937593584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 msgr2=0x5568c3ea2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.970+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.970+0000 7f96bf7fe700 1 -- 172.21.15.26:0/937593584 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568c3c442d0 con 0x5568c3e9fcf0 2023-07-13T22:22:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.980+0000 7f96bf7fe700 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96b80054c0 tx=0x7f96b8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900549bbd644e39e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.983+0000 7f96a67fc700 1 -- 172.21.15.26:0/937593584 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b800e040 con 0x5568c3e9fcf0 2023-07-13T22:22:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.983+0000 7f96a67fc700 1 -- 172.21.15.26:0/937593584 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8005a30 con 0x5568c3e9fcf0 2023-07-13T22:22:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.983+0000 7f96a67fc700 1 -- 172.21.15.26:0/937593584 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b8021af0 con 0x5568c3e9fcf0 2023-07-13T22:22:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.984+0000 7f96d9410b80 1 -- 172.21.15.26:0/937593584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 msgr2=0x5568c3ea3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.984+0000 7f96d9410b80 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f96b80054c0 tx=0x7f96b8005040 comp rx=0 tx=0).stop 2023-07-13T22:22:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.990+0000 7f96d9410b80 1 -- 172.21.15.26:0/937593584 shutdown_connections 2023-07-13T22:22:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.990+0000 7f96d9410b80 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.990+0000 7f96d9410b80 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9fcf0 0x5568c3ea3170 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.990+0000 7f96d9410b80 1 --2- 172.21.15.26:0/937593584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9eea0 0x5568c3eaf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:36.990+0000 7f96d9410b80 1 -- 172.21.15.26:0/937593584 >> 172.21.15.26:0/937593584 conn(0x5568c3e93e20 msgr2=0x5568c3eb5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.002+0000 7f96d9410b80 1 -- 172.21.15.26:0/937593584 shutdown_connections 2023-07-13T22:22:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.003+0000 7f96d9410b80 1 -- 172.21.15.26:0/937593584 wait complete. 2023-07-13T22:22:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.003+0000 7f96d9410b80 1 -- start start 2023-07-13T22:22:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.004+0000 7f96d9410b80 1 Processor -- start 2023-07-13T22:22:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.004+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.004+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.004+0000 7f96d9410b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96d9410b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3eaf790 con 0x5568c3e9e550 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96d9410b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3eaf900 con 0x5568c3e9eea0 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96d9410b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568c3eafac0 con 0x5568c3e9fcf0 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96d9410b80 10 client.0 _ll_register_callbacks cb 0x7f96b0011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96be7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96be7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60054/0 (socket says 172.21.15.26:60054) 2023-07-13T22:22:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.005+0000 7f96be7fc700 1 -- 172.21.15.26:0/699796312 learned_addr learned my addr 172.21.15.26:0/699796312 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.006+0000 7f96be7fc700 1 -- 172.21.15.26:0/699796312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 msgr2=0x5568c3eaf250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.006+0000 7f96be7fc700 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.006+0000 7f96be7fc700 1 -- 172.21.15.26:0/699796312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 msgr2=0x5568c3ea9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.006+0000 7f96be7fc700 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.006+0000 7f96be7fc700 1 -- 172.21.15.26:0/699796312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568c3c1de60 con 0x5568c3e9eea0 2023-07-13T22:22:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.017+0000 7f96be7fc700 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b40966e0 tx=0x7f96b4099680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb567827e691bfe server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.017+0000 7f969effd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.018+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b409dca0 con 0x5568c3e9eea0 2023-07-13T22:22:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.018+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f96b40a20c0 con 0x5568c3e9eea0 2023-07-13T22:22:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.018+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96b8021460 con 0x5568c3e9eea0 2023-07-13T22:22:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.023+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b40ca470 con 0x5568c3e9eea0 2023-07-13T22:22:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.028+0000 7f96d9410b80 10 client.26891 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.028+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5568c3c1c330 con 0x5568c3e9eea0 2023-07-13T22:22:37.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.047+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96b8021460 con 0x5568c3e9eea0 2023-07-13T22:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.047+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96b4099160 con 0x5568c3e9eea0 2023-07-13T22:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 10 client.26891 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:37.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 20 client.26891 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3f1a520 0x5568c3f1c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 4 client.26891 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.048+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5568c3dab0d0 con 0x5568c3f1a520 2023-07-13T22:22:37.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.049+0000 7f96beffd700 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3f1a520 0x5568c3f1c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.083+0000 7f96beffd700 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3f1a520 0x5568c3f1c930 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4278cd3d081f744 server_cookie=d494491854481520 in_seq=0 out_seq=0 2023-07-13T22:22:37.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.083+0000 7f969effd700 10 client.26891 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:37.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.095+0000 7f969effd700 1 -- 172.21.15.26:0/699796312 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5568c3dab0d0 con 0x5568c3f1a520 2023-07-13T22:22:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.095+0000 7f969effd700 10 client.26891 handle_command_reply: tid=0 2023-07-13T22:22:37.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.095+0000 7f96d9410b80 1 client.26891 shutdown 2023-07-13T22:22:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 msgr2=0x5568c3eaa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f96b40966e0 tx=0x7f96b4099680 comp rx=0 tx=0).stop 2023-07-13T22:22:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 shutdown_connections 2023-07-13T22:22:37.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5568c3e9e550 0x5568c3ea9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568c3f1a520 0x5568c3f1c930 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5568c3e9fcf0 0x5568c3eaf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 --2- 172.21.15.26:0/699796312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5568c3e9eea0 0x5568c3eaa270 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 >> 172.21.15.26:0/699796312 conn(0x5568c3e93e20 msgr2=0x5568c3ea2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.097+0000 7f969effd700 0 client.26891 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:37.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.114+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 shutdown_connections 2023-07-13T22:22:37.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.114+0000 7f96d9410b80 1 -- 172.21.15.26:0/699796312 wait complete. 2023-07-13T22:22:37.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.121+0000 7f96d9410b80 20 client.26891 trim_cache size 0 max 16384 2023-07-13T22:22:37.105 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:22:37.106 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:22:37.106 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "d3011db5-2f8d-4887-b003-7c8b35f50797", 2023-07-13T22:22:37.106 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:22:37.106 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:22:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 msgr2=0x7f96a80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 0x7f96a80a3800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009040 tx=0x7f96b8005540 comp rx=0 tx=0).stop 2023-07-13T22:22:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 msgr2=0x7f96c01a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 0x7f96c01a4ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005fd0 tx=0x7f96b400eca0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 shutdown_connections 2023-07-13T22:22:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f96a80a1350 0x7f96a80a3800 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96c010bd60 0x7f96c01abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96c0070ee0 0x7f96c01a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.127+0000 7f96c59ff700 1 --2- 172.21.15.26:0/57702592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96c0071830 0x7f96c01a4ce0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.128+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 >> 172.21.15.26:0/57702592 conn(0x7f96c006ca90 msgr2=0x7f96c006dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.141+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 shutdown_connections 2023-07-13T22:22:37.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.158+0000 7f96c59ff700 1 -- 172.21.15.26:0/57702592 wait complete. 2023-07-13T22:22:37.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:22:37.433 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:37 smithi026 ceph-mon[95647]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:22:37.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:37 smithi117 ceph-mon[129074]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:22:37.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:37 smithi121 ceph-mon[129187]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:22:37.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 -- 172.21.15.26:0/3514107785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 msgr2=0x7f63400712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 --2- 172.21.15.26:0/3514107785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f63400712f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6328009990 tx=0x7f6328009ca0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 -- 172.21.15.26:0/3514107785 shutdown_connections 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 --2- 172.21.15.26:0/3514107785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634010bd60 0x7f634010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 --2- 172.21.15.26:0/3514107785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f63400712f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 --2- 172.21.15.26:0/3514107785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6340071830 0x7f634010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.778+0000 7f63469ff700 1 -- 172.21.15.26:0/3514107785 >> 172.21.15.26:0/3514107785 conn(0x7f634006ca90 msgr2=0x7f634006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.779+0000 7f63469ff700 1 -- 172.21.15.26:0/3514107785 shutdown_connections 2023-07-13T22:22:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.779+0000 7f63469ff700 1 -- 172.21.15.26:0/3514107785 wait complete. 2023-07-13T22:22:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.780+0000 7f63469ff700 1 Processor -- start 2023-07-13T22:22:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.781+0000 7f63469ff700 1 -- start start 2023-07-13T22:22:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.781+0000 7f63469ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.781+0000 7f63469ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6340071830 0x7f6340113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.782+0000 7f633ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.782+0000 7f63469ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634010bd60 0x7f63401b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.782+0000 7f63469ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6340117080 con 0x7f6340071830 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.782+0000 7f63469ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401171f0 con 0x7f6340070ee0 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.782+0000 7f63469ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63401b4dd0 con 0x7f634010bd60 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.783+0000 7f633ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60072/0 (socket says 172.21.15.26:60072) 2023-07-13T22:22:37.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.783+0000 7f633ffff700 1 -- 172.21.15.26:0/3964649069 learned_addr learned my addr 172.21.15.26:0/3964649069 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.783+0000 7f6337fff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6340071830 0x7f6340113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 -- 172.21.15.26:0/3964649069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634010bd60 msgr2=0x7f63401b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634010bd60 0x7f63401b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 -- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6340071830 msgr2=0x7f6340113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6340071830 0x7f6340113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 -- 172.21.15.26:0/3964649069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280095f0 con 0x7f6340070ee0 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.784+0000 7f633ffff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6328005f50 tx=0x7f6328003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.785+0000 7f633dffb700 1 -- 172.21.15.26:0/3964649069 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632801d070 con 0x7f6340070ee0 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.785+0000 7f633dffb700 1 -- 172.21.15.26:0/3964649069 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632800f460 con 0x7f6340070ee0 2023-07-13T22:22:37.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.785+0000 7f633dffb700 1 -- 172.21.15.26:0/3964649069 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328003ea0 con 0x7f6340070ee0 2023-07-13T22:22:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.792+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63401b4f70 con 0x7f6340070ee0 2023-07-13T22:22:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.792+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63401b52e0 con 0x7f6340070ee0 2023-07-13T22:22:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.821+0000 7f635a3bab80 1 Processor -- start 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.822+0000 7f633dffb700 1 -- 172.21.15.26:0/3964649069 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f632800f890 con 0x7f6340070ee0 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.822+0000 7f635a3bab80 1 -- start start 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.825+0000 7f633dffb700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 0x7f632c0a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.825+0000 7f633dffb700 1 -- 172.21.15.26:0/3964649069 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63280cdb00 con 0x7f6340070ee0 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.826+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.826+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.826+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a8e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.827+0000 7f635a3bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0991cb0 con 0x55eac0a8c4d0 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.827+0000 7f635a3bab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac09e4a40 con 0x55eac0a8ce20 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.827+0000 7f635a3bab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac09dded0 con 0x55eac0a8dc70 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.831+0000 7f6337fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.832+0000 7f6337fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a8d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60086/0 (socket says 172.21.15.26:60086) 2023-07-13T22:22:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.832+0000 7f6337fff700 1 -- 172.21.15.26:0/172881056 learned_addr learned my addr 172.21.15.26:0/172881056 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.837+0000 7f633ffff700 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 -- 172.21.15.26:0/172881056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 msgr2=0x55eac0a8e0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:37.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a8e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 -- 172.21.15.26:0/172881056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 msgr2=0x55eac0a8d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a8d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 -- 172.21.15.26:0/172881056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac080bde0 con 0x55eac0a8c4d0 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.838+0000 7f633ffff700 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a8c8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f63280cb5b0 tx=0x7f632809ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d221c86c0c002 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.842+0000 7f6344f9c700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 0x7f632c0a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.849+0000 7f63367fc700 1 -- 172.21.15.26:0/172881056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328029340 con 0x55eac0a8c4d0 2023-07-13T22:22:37.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.849+0000 7f63367fc700 1 -- 172.21.15.26:0/172881056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632809af50 con 0x55eac0a8c4d0 2023-07-13T22:22:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.849+0000 7f63367fc700 1 -- 172.21.15.26:0/172881056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328099380 con 0x55eac0a8c4d0 2023-07-13T22:22:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.851+0000 7f635a3bab80 1 -- 172.21.15.26:0/172881056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 msgr2=0x55eac0a8c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.851+0000 7f635a3bab80 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a8c8e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f63280cb5b0 tx=0x7f632809ada0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 -- 172.21.15.26:0/172881056 shutdown_connections 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a8c8e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a8e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 --2- 172.21.15.26:0/172881056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a8d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 -- 172.21.15.26:0/172881056 >> 172.21.15.26:0/172881056 conn(0x55eac0a81da0 msgr2=0x55eac0a871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 -- 172.21.15.26:0/172881056 shutdown_connections 2023-07-13T22:22:37.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.859+0000 7f635a3bab80 1 -- 172.21.15.26:0/172881056 wait complete. 2023-07-13T22:22:37.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.860+0000 7f635a3bab80 1 -- start start 2023-07-13T22:22:37.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.861+0000 7f635a3bab80 1 Processor -- start 2023-07-13T22:22:37.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a95e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a96380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a9d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac097dac0 con 0x55eac0a8dc70 2023-07-13T22:22:37.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac097d7c0 con 0x55eac0a8ce20 2023-07-13T22:22:37.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0861920 con 0x55eac0a8c4d0 2023-07-13T22:22:37.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.862+0000 7f635a3bab80 10 client.0 _ll_register_callbacks cb 0x55eac097b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:37.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.868+0000 7f633ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a95e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.868+0000 7f6337fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a96380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.869+0000 7f633ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a95e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44384/0 (socket says 172.21.15.26:44384) 2023-07-13T22:22:37.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.869+0000 7f633ffff700 1 -- 172.21.15.26:0/3889336381 learned_addr learned my addr 172.21.15.26:0/3889336381 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:37.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.869+0000 7f6337fff700 1 -- 172.21.15.26:0/3889336381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 msgr2=0x55eac0a95e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.869+0000 7f6337fff700 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a95e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.869+0000 7f6337fff700 1 -- 172.21.15.26:0/3889336381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 msgr2=0x55eac0a9d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.870+0000 7f6337fff700 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a9d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.870+0000 7f6337fff700 1 -- 172.21.15.26:0/3889336381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63401b52e0 con 0x55eac0a8ce20 2023-07-13T22:22:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.878+0000 7f6337fff700 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a96380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6330005c50 tx=0x7f6330012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d334b9db0861547 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.878+0000 7f631f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:22:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.878+0000 7f6344f9c700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 0x7f632c0a3780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6338008f60 tx=0x7f6338008560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.879+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300108a0 con 0x55eac0a8ce20 2023-07-13T22:22:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.879+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6330010a00 con 0x55eac0a8ce20 2023-07-13T22:22:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.879+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eac09374f0 con 0x55eac0a8ce20 2023-07-13T22:22:37.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.893+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6330010bc0 con 0x55eac0a8ce20 2023-07-13T22:22:37.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.894+0000 7f635a3bab80 10 client.26903 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:37.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.894+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eac07f4c60 con 0x55eac0a8ce20 2023-07-13T22:22:37.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.896+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eac09374f0 con 0x55eac0a8ce20 2023-07-13T22:22:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.896+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f633001b440 con 0x55eac0a8ce20 2023-07-13T22:22:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.897+0000 7f635a3bab80 10 client.26903 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.897+0000 7f635a3bab80 10 client.26903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.898+0000 7f635a3bab80 20 client.26903 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.898+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0aa1bc0 0x55eac0aa3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.898+0000 7f635a3bab80 4 client.26903 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:22:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.898+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eac09d6eb0 con 0x55eac0aa1bc0 2023-07-13T22:22:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.898+0000 7f633ffff700 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0aa1bc0 0x55eac0aa3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.899+0000 7f633ffff700 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0aa1bc0 0x55eac0aa3fd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86aee0bdeb90869e server_cookie=7888de88e174816 in_seq=0 out_seq=0 2023-07-13T22:22:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.899+0000 7f631f7fe700 10 client.26903 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.900+0000 7f631f7fe700 1 -- 172.21.15.26:0/3889336381 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eac09d6eb0 con 0x55eac0aa1bc0 2023-07-13T22:22:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.900+0000 7f631f7fe700 10 client.26903 handle_command_reply: tid=0 2023-07-13T22:22:37.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.900+0000 7f635a3bab80 1 client.26903 shutdown 2023-07-13T22:22:37.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 msgr2=0x55eac0a96380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a96380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6330005c50 tx=0x7f6330012730 comp rx=0 tx=0).stop 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 shutdown_connections 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a9d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0aa1bc0 0x55eac0aa3fd0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a95e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 --2- 172.21.15.26:0/3889336381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a96380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 >> 172.21.15.26:0/3889336381 conn(0x55eac0a81da0 msgr2=0x55eac0a868b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f631f7fe700 0 client.26903 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 shutdown_connections 2023-07-13T22:22:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.901+0000 7f635a3bab80 1 -- 172.21.15.26:0/3889336381 wait complete. 2023-07-13T22:22:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.902+0000 7f635a3bab80 20 client.26903 trim_cache size 0 max 16384 2023-07-13T22:22:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 Processor -- start 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 -- start start 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a95e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a88680 con 0x55eac0a8dc70 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a9f0b0 con 0x55eac0a8ce20 2023-07-13T22:22:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.939+0000 7f635a3bab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a959b0 con 0x55eac0a8c4d0 2023-07-13T22:22:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.940+0000 7f6337fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.940+0000 7f6337fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60098/0 (socket says 172.21.15.26:60098) 2023-07-13T22:22:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.940+0000 7f6337fff700 1 -- 172.21.15.26:0/50580850 learned_addr learned my addr 172.21.15.26:0/50580850 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.940+0000 7f633ffff700 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.941+0000 7f6337fff700 1 -- 172.21.15.26:0/50580850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 msgr2=0x55eac0a940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.941+0000 7f6337fff700 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.941+0000 7f6337fff700 1 -- 172.21.15.26:0/50580850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 msgr2=0x55eac0a95e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.941+0000 7f6337fff700 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a95e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.941+0000 7f6337fff700 1 -- 172.21.15.26:0/50580850 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eac0832250 con 0x55eac0a8ce20 2023-07-13T22:22:37.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.943+0000 7f6337fff700 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f633000c010 tx=0x7f6330005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e28d09cda8d6771 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.944+0000 7f6336ffd700 1 -- 172.21.15.26:0/50580850 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330004020 con 0x55eac0a8ce20 2023-07-13T22:22:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.944+0000 7f6336ffd700 1 -- 172.21.15.26:0/50580850 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6330004710 con 0x55eac0a8ce20 2023-07-13T22:22:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.944+0000 7f6336ffd700 1 -- 172.21.15.26:0/50580850 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300052a0 con 0x55eac0a8ce20 2023-07-13T22:22:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 -- 172.21.15.26:0/50580850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 msgr2=0x55eac0a9d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f633000c010 tx=0x7f6330005ce0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 -- 172.21.15.26:0/50580850 shutdown_connections 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8dc70 0x55eac0a95e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8c4d0 0x55eac0a940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 --2- 172.21.15.26:0/50580850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9d380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.945+0000 7f635a3bab80 1 -- 172.21.15.26:0/50580850 >> 172.21.15.26:0/50580850 conn(0x55eac0a81da0 msgr2=0x55eac0a94b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.946+0000 7f635a3bab80 1 -- 172.21.15.26:0/50580850 shutdown_connections 2023-07-13T22:22:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.946+0000 7f635a3bab80 1 -- 172.21.15.26:0/50580850 wait complete. 2023-07-13T22:22:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.946+0000 7f635a3bab80 1 -- start start 2023-07-13T22:22:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 Processor -- start 2023-07-13T22:22:37.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a9c150 con 0x55eac0a8c4d0 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a9c2c0 con 0x55eac0a8ce20 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eac0a9c480 con 0x55eac0a8dc70 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.947+0000 7f635a3bab80 10 client.0 _ll_register_callbacks cb 0x7f633000b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.948+0000 7f6344f9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.948+0000 7f6344f9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44408/0 (socket says 172.21.15.26:44408) 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.948+0000 7f6344f9c700 1 -- 172.21.15.26:0/1466993407 learned_addr learned my addr 172.21.15.26:0/1466993407 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 -- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 msgr2=0x55eac0a9b6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9b6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 -- 172.21.15.26:0/1466993407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 msgr2=0x55eac0a9d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 -- 172.21.15.26:0/1466993407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eac07f4c60 con 0x55eac0a8dc70 2023-07-13T22:22:37.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.949+0000 7f6344f9c700 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6338017c10 tx=0x7f633800bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28b3a79032adccac server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:22:37.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.953+0000 7f631f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:22:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.953+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633801c070 con 0x55eac0a8dc70 2023-07-13T22:22:37.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.953+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6338015700 con 0x55eac0a8dc70 2023-07-13T22:22:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.959+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f632809af50 con 0x55eac0a8dc70 2023-07-13T22:22:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.959+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6338010070 con 0x55eac0a8dc70 2023-07-13T22:22:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.959+0000 7f635a3bab80 10 client.26902 fetch_fsmap learned FSMap version 29 2023-07-13T22:22:37.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.959+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55eac080a2b0 con 0x55eac0a8dc70 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.970+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f632809af50 con 0x55eac0a8dc70 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.970+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6338022a50 con 0x55eac0a8dc70 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 10 client.26902 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 10 client.26902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 20 client.26902 populate_metadata read hostname 'smithi026' 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0af7220 0x55eac0af9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:22:37.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 4 client.26902 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:22:37.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.971+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eac0999050 con 0x55eac0af7220 2023-07-13T22:22:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:37.982+0000 7f633ffff700 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0af7220 0x55eac0af9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:22:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.007+0000 7f633ffff700 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0af7220 0x55eac0af9630 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70efcbdddaa54443 server_cookie=26617c8efaebd91 in_seq=0 out_seq=0 2023-07-13T22:22:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.007+0000 7f631f7fe700 10 client.26902 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.008+0000 7f631f7fe700 1 -- 172.21.15.26:0/1466993407 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eac0999050 con 0x55eac0af7220 2023-07-13T22:22:37.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.008+0000 7f631f7fe700 10 client.26902 handle_command_reply: tid=0 2023-07-13T22:22:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.008+0000 7f635a3bab80 1 client.26902 shutdown 2023-07-13T22:22:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 msgr2=0x55eac0a9bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6338017c10 tx=0x7f633800bd90 comp rx=0 tx=0).stop 2023-07-13T22:22:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 shutdown_connections 2023-07-13T22:22:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55eac0a8c4d0 0x55eac0a9d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55eac0af7220 0x55eac0af9630 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.033+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eac0a8dc70 0x55eac0a9bc10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f635a3bab80 1 --2- 172.21.15.26:0/1466993407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55eac0a8ce20 0x55eac0a9b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 >> 172.21.15.26:0/1466993407 conn(0x55eac0a81da0 msgr2=0x55eac0a940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f631f7fe700 0 client.26902 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 shutdown_connections 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f635a3bab80 1 -- 172.21.15.26:0/1466993407 wait complete. 2023-07-13T22:22:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.034+0000 7f635a3bab80 20 client.26902 trim_cache size 0 max 16384 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout: "d3011db5-2f8d-4887-b003-7c8b35f50797": { 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:22:38.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "d3011db5-2f8d-4887-b003-7c8b35f50797", 2023-07-13T22:22:38.022 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:22:38.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:22:38.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:22:38.022 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:22:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.068+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 msgr2=0x7f632c0a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.068+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 0x7f632c0a3780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6338008f60 tx=0x7f6338008560 comp rx=0 tx=0).stop 2023-07-13T22:22:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.068+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 msgr2=0x7f6340113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:22:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.068+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6328005f50 tx=0x7f6328003cc0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 shutdown_connections 2023-07-13T22:22:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f634010bd60 0x7f63401b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6340070ee0 0x7f6340113a60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f632c0a12d0 0x7f632c0a3780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 --2- 172.21.15.26:0/3964649069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6340071830 0x7f6340113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:22:38.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.069+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 >> 172.21.15.26:0/3964649069 conn(0x7f634006ca90 msgr2=0x7f634006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:22:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.084+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 shutdown_connections 2023-07-13T22:22:38.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:22:38.084+0000 7f63469ff700 1 -- 172.21.15.26:0/3964649069 wait complete. 2023-07-13T22:22:38.108 INFO:tasks.cephfs.filesystem:scrub status for tag:d3011db5-2f8d-4887-b003-7c8b35f50797 - {'path': '/', 'tag': 'd3011db5-2f8d-4887-b003-7c8b35f50797', 'options': 'recursive,force'} 2023-07-13T22:22:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:22:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:22:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:22:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:38 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:22:38.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:22:38.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:22:38.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:38 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:22:38.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:22:38.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:22:38.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:38.989 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:38 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:22:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:39 smithi117 ceph-mon[129074]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 320 MiB/s wr, 107 op/s 2023-07-13T22:22:39.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:39 smithi026 ceph-mon[95647]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 320 MiB/s wr, 107 op/s 2023-07-13T22:22:39.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:39 smithi121 ceph-mon[129187]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 320 MiB/s wr, 107 op/s 2023-07-13T22:22:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:41 smithi117 ceph-mon[129074]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 99 op/s 2023-07-13T22:22:41.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:41 smithi026 ceph-mon[95647]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 99 op/s 2023-07-13T22:22:41.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:41 smithi121 ceph-mon[129187]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 304 MiB/s wr, 99 op/s 2023-07-13T22:22:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:43 smithi117 ceph-mon[129074]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 275 MiB/s wr, 93 op/s 2023-07-13T22:22:43.696 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:43 smithi026 ceph-mon[95647]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 275 MiB/s wr, 93 op/s 2023-07-13T22:22:43.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:43 smithi121 ceph-mon[129187]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 275 MiB/s wr, 93 op/s 2023-07-13T22:22:45.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:45 smithi117 ceph-mon[129074]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 104 op/s 2023-07-13T22:22:45.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:45 smithi026 ceph-mon[95647]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 104 op/s 2023-07-13T22:22:45.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:45 smithi121 ceph-mon[129187]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 104 op/s 2023-07-13T22:22:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:46 smithi117 ceph-mon[129074]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:22:46.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:46 smithi026 ceph-mon[95647]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:22:46.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:46 smithi121 ceph-mon[129187]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 274 MiB/s wr, 90 op/s 2023-07-13T22:22:47.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:47 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:22:47.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:47 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:22:47.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:47 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:22:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:48 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:22:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:48 smithi117 ceph-mon[129074]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 101 op/s 2023-07-13T22:22:48.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:48 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:22:48.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:48 smithi026 ceph-mon[95647]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 101 op/s 2023-07-13T22:22:48.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:48 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:22:48.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:48 smithi121 ceph-mon[129187]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 305 MiB/s wr, 101 op/s 2023-07-13T22:22:51.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:51 smithi117 ceph-mon[129074]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:22:51.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:51 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:22:51.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:51 smithi026 ceph-mon[95647]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:22:51.952 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:51 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:22:51.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:51 smithi121 ceph-mon[129187]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 288 MiB/s wr, 93 op/s 2023-07-13T22:22:51.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:51 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:22:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:52 smithi117 ceph-mon[129074]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 89 op/s 2023-07-13T22:22:52.959 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:52 smithi026 ceph-mon[95647]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 89 op/s 2023-07-13T22:22:52.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:52 smithi121 ceph-mon[129187]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 273 MiB/s wr, 89 op/s 2023-07-13T22:22:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:22:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:22:53.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:53 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:22:53.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:22:53.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:22:53.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:53 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:22:53.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:22:53.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:22:53.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:53 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:22:54.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:54 smithi121 ceph-mon[129187]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:22:55.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:54 smithi117 ceph-mon[129074]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:22:55.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:54 smithi026 ceph-mon[95647]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 104 op/s 2023-07-13T22:22:57.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:57 smithi117 ceph-mon[129074]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:22:57.702 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:57 smithi026 ceph-mon[95647]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:22:57.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:57 smithi121 ceph-mon[129187]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 286 MiB/s wr, 91 op/s 2023-07-13T22:22:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:22:59 smithi117 ceph-mon[129074]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 311 MiB/s wr, 103 op/s 2023-07-13T22:22:59.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:22:59 smithi026 ceph-mon[95647]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 311 MiB/s wr, 103 op/s 2023-07-13T22:22:59.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:22:59 smithi121 ceph-mon[129187]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 311 MiB/s wr, 103 op/s 2023-07-13T22:23:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:01 smithi117 ceph-mon[129074]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:01.701 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:01 smithi026 ceph-mon[95647]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:01.738 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:01 smithi121 ceph-mon[129187]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:03 smithi117 ceph-mon[129074]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 93 op/s 2023-07-13T22:23:03.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:03 smithi026 ceph-mon[95647]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 93 op/s 2023-07-13T22:23:03.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:03 smithi121 ceph-mon[129187]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 280 MiB/s wr, 93 op/s 2023-07-13T22:23:05.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:05 smithi117 ceph-mon[129074]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 320 MiB/s wr, 104 op/s 2023-07-13T22:23:05.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:05 smithi026 ceph-mon[95647]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 320 MiB/s wr, 104 op/s 2023-07-13T22:23:05.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:05 smithi121 ceph-mon[129187]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 320 MiB/s wr, 104 op/s 2023-07-13T22:23:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:07 smithi117 ceph-mon[129074]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 88 op/s 2023-07-13T22:23:07.955 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:07 smithi026 ceph-mon[95647]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 88 op/s 2023-07-13T22:23:07.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:07 smithi121 ceph-mon[129187]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 MiB/s wr, 88 op/s 2023-07-13T22:23:08.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:23:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.615+0000 7f288a1ff700 1 -- 172.21.15.26:0/2499471743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28841080d0 msgr2=0x7f288410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.615+0000 7f288a1ff700 1 --2- 172.21.15.26:0/2499471743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28841080d0 0x7f288410e970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f286c0098f0 tx=0x7f286c009c00 comp rx=0 tx=0).stop 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.615+0000 7f288a1ff700 1 -- 172.21.15.26:0/2499471743 shutdown_connections 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 --2- 172.21.15.26:0/2499471743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28841080d0 0x7f288410e970 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 --2- 172.21.15.26:0/2499471743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 0x7f2884071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 --2- 172.21.15.26:0/2499471743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884107720 0x7f2884107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 -- 172.21.15.26:0/2499471743 >> 172.21.15.26:0/2499471743 conn(0x7f288406cd80 msgr2=0x7f288406f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 -- 172.21.15.26:0/2499471743 shutdown_connections 2023-07-13T22:23:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.616+0000 7f288a1ff700 1 -- 172.21.15.26:0/2499471743 wait complete. 2023-07-13T22:23:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.617+0000 7f288a1ff700 1 Processor -- start 2023-07-13T22:23:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 -- start start 2023-07-13T22:23:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 0x7f2884117080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2884107720 0x7f28841175c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288411ec70 con 0x7f2884107720 2023-07-13T22:23:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288411ede0 con 0x7f28840716f0 2023-07-13T22:23:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.618+0000 7f288a1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288411ef50 con 0x7f28841080d0 2023-07-13T22:23:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49418/0 (socket says 172.21.15.26:49418) 2023-07-13T22:23:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 -- 172.21.15.26:0/3537871510 learned_addr learned my addr 172.21.15.26:0/3537871510 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f28837fe700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 0x7f2884117080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 -- 172.21.15.26:0/3537871510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 msgr2=0x7f2884117080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 0x7f2884117080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 -- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2884107720 msgr2=0x7f28841175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2884107720 0x7f28841175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.619+0000 7f2883fff700 1 -- 172.21.15.26:0/3537871510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c009520 con 0x7f28841080d0 2023-07-13T22:23:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.620+0000 7f2883fff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f286c00fa40 tx=0x7f286c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.620+0000 7f2880ff9700 1 -- 172.21.15.26:0/3537871510 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c01c070 con 0x7f28841080d0 2023-07-13T22:23:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.620+0000 7f2880ff9700 1 -- 172.21.15.26:0/3537871510 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c00ba60 con 0x7f28841080d0 2023-07-13T22:23:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.620+0000 7f2880ff9700 1 -- 172.21.15.26:0/3537871510 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c003be0 con 0x7f28841080d0 2023-07-13T22:23:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.621+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288411f1d0 con 0x7f28841080d0 2023-07-13T22:23:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.621+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28841a4390 con 0x7f28841080d0 2023-07-13T22:23:08.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.628+0000 7f2880ff9700 1 -- 172.21.15.26:0/3537871510 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f286c020030 con 0x7f28841080d0 2023-07-13T22:23:08.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.628+0000 7f289db66b80 1 Processor -- start 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.628+0000 7f289db66b80 1 -- start start 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f0286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f028c70 0x56023f0290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef2ccb0 con 0x56023f027e20 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef7fa40 con 0x56023f0274d0 2023-07-13T22:23:08.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.629+0000 7f289db66b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef78ed0 con 0x56023f028c70 2023-07-13T22:23:08.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48614/0 (socket says 172.21.15.26:48614) 2023-07-13T22:23:08.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 -- 172.21.15.26:0/3552503 learned_addr learned my addr 172.21.15.26:0/3552503 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f028c70 0x56023f0290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 -- 172.21.15.26:0/3552503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f028c70 msgr2=0x56023f0290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f028c70 0x56023f0290e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 -- 172.21.15.26:0/3552503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 msgr2=0x56023f0286e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f0286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.630+0000 7f2882ffd700 1 -- 172.21.15.26:0/3552503 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023eda6de0 con 0x56023f0274d0 2023-07-13T22:23:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.631+0000 7f2882ffd700 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2874013840 tx=0x7f2874009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b689f62ddcdd7bba server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.631+0000 7f2879ffb700 1 -- 172.21.15.26:0/3552503 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874004a30 con 0x56023f0274d0 2023-07-13T22:23:08.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.631+0000 7f2880ff9700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 0x7f28700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.631+0000 7f2879ffb700 1 -- 172.21.15.26:0/3552503 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2874019070 con 0x56023f0274d0 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.631+0000 7f2879ffb700 1 -- 172.21.15.26:0/3552503 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400f6e0 con 0x56023f0274d0 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.632+0000 7f2880ff9700 1 -- 172.21.15.26:0/3537871510 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f286c0cdf40 con 0x7f28841080d0 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.632+0000 7f28837fe700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 0x7f28700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f289db66b80 1 -- 172.21.15.26:0/3552503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 msgr2=0x56023f0278e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f289db66b80 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2874013840 tx=0x7f2874009770 comp rx=0 tx=0).stop 2023-07-13T22:23:08.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f28837fe700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 0x7f28700a3680 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f28841185d0 tx=0x7f287c0099e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f289db66b80 1 -- 172.21.15.26:0/3552503 shutdown_connections 2023-07-13T22:23:08.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f289db66b80 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f028c70 0x56023f0290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.633+0000 7f289db66b80 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f0274d0 0x56023f0278e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.634+0000 7f289db66b80 1 --2- 172.21.15.26:0/3552503 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f0286e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.634+0000 7f289db66b80 1 -- 172.21.15.26:0/3552503 >> 172.21.15.26:0/3552503 conn(0x56023f01cda0 msgr2=0x56023f0221d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.635+0000 7f289db66b80 1 -- 172.21.15.26:0/3552503 shutdown_connections 2023-07-13T22:23:08.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.636+0000 7f289db66b80 1 -- 172.21.15.26:0/3552503 wait complete. 2023-07-13T22:23:08.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.636+0000 7f289db66b80 1 -- start start 2023-07-13T22:23:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.637+0000 7f289db66b80 1 Processor -- start 2023-07-13T22:23:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f031130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f031670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef18ac0 con 0x56023f0274d0 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023ef187c0 con 0x56023f028c70 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023edfc920 con 0x56023f027e20 2023-07-13T22:23:08.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.638+0000 7f289db66b80 10 client.0 _ll_register_callbacks cb 0x56023ef16760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.639+0000 7f2883fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.639+0000 7f2883fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48624/0 (socket says 172.21.15.26:48624) 2023-07-13T22:23:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.639+0000 7f2883fff700 1 -- 172.21.15.26:0/850161325 learned_addr learned my addr 172.21.15.26:0/850161325 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.639+0000 7f28837fe700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f031130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.640+0000 7f2883fff700 1 -- 172.21.15.26:0/850161325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 msgr2=0x56023f031670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.640+0000 7f2883fff700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f031670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.640+0000 7f2883fff700 1 -- 172.21.15.26:0/850161325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 msgr2=0x56023f031130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.640+0000 7f2883fff700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f031130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.640+0000 7f2883fff700 1 -- 172.21.15.26:0/850161325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2874013520 con 0x56023f028c70 2023-07-13T22:23:08.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.641+0000 7f2883fff700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f286c097890 tx=0x7f286c09acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8f0bfeb47db63 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.650+0000 7f2862ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:08.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.650+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c09f3c0 con 0x56023f028c70 2023-07-13T22:23:08.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.650+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f286c0a3750 con 0x56023f028c70 2023-07-13T22:23:08.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.651+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56023eed24f0 con 0x56023f028c70 2023-07-13T22:23:08.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.652+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f286c003d40 con 0x56023f028c70 2023-07-13T22:23:08.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.652+0000 7f289db66b80 10 client.26918 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:08.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.652+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56023ed8fc60 con 0x56023f028c70 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.654+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56023eed24f0 con 0x56023f028c70 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.654+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f286c084be0 con 0x56023f028c70 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 10 client.26918 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 10 client.26918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 20 client.26918 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:08.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f03caa0 0x56023f03eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 4 client.26918 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:08.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.655+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56023ef71eb0 con 0x56023f03caa0 2023-07-13T22:23:08.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.658+0000 7f28837fe700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f03caa0 0x56023f03eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.660+0000 7f2862ffd700 10 client.26918 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:08.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.660+0000 7f28837fe700 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f03caa0 0x56023f03eeb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f18143c6f518298 server_cookie=430438688817130c in_seq=0 out_seq=0 2023-07-13T22:23:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.661+0000 7f2862ffd700 1 -- 172.21.15.26:0/850161325 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56023ef71eb0 con 0x56023f03caa0 2023-07-13T22:23:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.661+0000 7f2862ffd700 10 client.26918 handle_command_reply: tid=0 2023-07-13T22:23:08.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.662+0000 7f289db66b80 1 client.26918 shutdown 2023-07-13T22:23:08.676 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:08 smithi026 ceph-mon[95647]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 303 MiB/s wr, 101 op/s 2023-07-13T22:23:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.693+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 msgr2=0x56023f0386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.693+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f286c097890 tx=0x7f286c09acf0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 shutdown_connections 2023-07-13T22:23:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f031130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f03caa0 0x56023f03eeb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f0386b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 --2- 172.21.15.26:0/850161325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f031670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.696+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 >> 172.21.15.26:0/850161325 conn(0x56023f01cda0 msgr2=0x56023f0218b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.698+0000 7f2862ffd700 0 client.26918 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.698+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 shutdown_connections 2023-07-13T22:23:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.698+0000 7f289db66b80 1 -- 172.21.15.26:0/850161325 wait complete. 2023-07-13T22:23:08.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.699+0000 7f289db66b80 20 client.26918 trim_cache size 0 max 16384 2023-07-13T22:23:08.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 Processor -- start 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 -- start start 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 0x56023f03eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f038670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f02f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f01ca00 con 0x56023f027e20 2023-07-13T22:23:08.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f023cf0 con 0x56023f028c70 2023-07-13T22:23:08.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.760+0000 7f289db66b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f02ee10 con 0x56023f0274d0 2023-07-13T22:23:08.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.761+0000 7f28837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 0x56023f03eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f28837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 0x56023f03eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49442/0 (socket says 172.21.15.26:49442) 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f28837fe700 1 -- 172.21.15.26:0/153571217 learned_addr learned my addr 172.21.15.26:0/153571217 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f02f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2882ffd700 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f038670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 -- 172.21.15.26:0/153571217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 msgr2=0x56023f03eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 0x56023f03eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 -- 172.21.15.26:0/153571217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 msgr2=0x56023f038670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f038670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.762+0000 7f2883fff700 1 -- 172.21.15.26:0/153571217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023edcd250 con 0x56023f028c70 2023-07-13T22:23:08.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.763+0000 7f2883fff700 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f02f060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f286c0098c0 tx=0x7f286c086180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f753be85d8b77491 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.764+0000 7f287a7fc700 1 -- 172.21.15.26:0/153571217 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c0a3750 con 0x56023f028c70 2023-07-13T22:23:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.764+0000 7f287a7fc700 1 -- 172.21.15.26:0/153571217 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286c09f3c0 con 0x56023f028c70 2023-07-13T22:23:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.764+0000 7f287a7fc700 1 -- 172.21.15.26:0/153571217 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286c09b710 con 0x56023f028c70 2023-07-13T22:23:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 -- 172.21.15.26:0/153571217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 msgr2=0x56023f02f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f02f060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f286c0098c0 tx=0x7f286c086180 comp rx=0 tx=0).stop 2023-07-13T22:23:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 -- 172.21.15.26:0/153571217 shutdown_connections 2023-07-13T22:23:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f02f060 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f0274d0 0x56023f03eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 --2- 172.21.15.26:0/153571217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f027e20 0x56023f038670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.765+0000 7f289db66b80 1 -- 172.21.15.26:0/153571217 >> 172.21.15.26:0/153571217 conn(0x56023f01cda0 msgr2=0x56023f02fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.766+0000 7f289db66b80 1 -- 172.21.15.26:0/153571217 shutdown_connections 2023-07-13T22:23:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.766+0000 7f289db66b80 1 -- 172.21.15.26:0/153571217 wait complete. 2023-07-13T22:23:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.766+0000 7f289db66b80 1 -- start start 2023-07-13T22:23:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 Processor -- start 2023-07-13T22:23:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f034800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f037b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f0380c0 con 0x56023f0274d0 2023-07-13T22:23:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f038230 con 0x56023f028c70 2023-07-13T22:23:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023f0383f0 con 0x56023f027e20 2023-07-13T22:23:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.767+0000 7f289db66b80 10 client.0 _ll_register_callbacks cb 0x7f286c0977a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.768+0000 7f2882ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.768+0000 7f2882ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49452/0 (socket says 172.21.15.26:49452) 2023-07-13T22:23:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.768+0000 7f2882ffd700 1 -- 172.21.15.26:0/264444914 learned_addr learned my addr 172.21.15.26:0/264444914 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f28837fe700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f034800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2883fff700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2882ffd700 1 -- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 msgr2=0x56023f037b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2882ffd700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f037b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2882ffd700 1 -- 172.21.15.26:0/264444914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 msgr2=0x56023f034800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2882ffd700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f034800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2882ffd700 1 -- 172.21.15.26:0/264444914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56023ed8fc60 con 0x56023f027e20 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.769+0000 7f2883fff700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f037b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.770+0000 7f2882ffd700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f28740092f0 tx=0x7f2874014e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc64e73c3781ed7d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.770+0000 7f2862ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.771+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874025070 con 0x56023f027e20 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.771+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2874004ac0 con 0x56023f027e20 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.771+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2874019070 con 0x56023f027e20 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.771+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2874004dd0 con 0x56023f027e20 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 10 client.26917 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56023eda52b0 con 0x56023f027e20 2023-07-13T22:23:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2874019070 con 0x56023f027e20 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f287400f6b0 con 0x56023f027e20 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 10 client.26917 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 10 client.26917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 20 client.26917 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f091910 0x56023f093d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 4 client.26917 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:23:08.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.772+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56023ef34050 con 0x56023f091910 2023-07-13T22:23:08.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.773+0000 7f28837fe700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f091910 0x56023f093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:08.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.774+0000 7f28837fe700 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f091910 0x56023f093d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ccf912b8b7d655 server_cookie=9b403519bbbe124f in_seq=0 out_seq=0 2023-07-13T22:23:08.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.774+0000 7f2862ffd700 10 client.26917 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:08.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.774+0000 7f2862ffd700 1 -- 172.21.15.26:0/264444914 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56023ef34050 con 0x56023f091910 2023-07-13T22:23:08.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.774+0000 7f2862ffd700 10 client.26917 handle_command_reply: tid=0 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.774+0000 7f289db66b80 1 client.26917 shutdown 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 msgr2=0x56023f034d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f28740092f0 tx=0x7f2874014e40 comp rx=0 tx=0).stop 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 shutdown_connections 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56023f0274d0 0x56023f034800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56023f091910 0x56023f093d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56023f028c70 0x56023f037b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 --2- 172.21.15.26:0/264444914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56023f027e20 0x56023f034d40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 >> 172.21.15.26:0/264444914 conn(0x56023f01cda0 msgr2=0x56023f03eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f2862ffd700 0 client.26917 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 shutdown_connections 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.775+0000 7f289db66b80 1 -- 172.21.15.26:0/264444914 wait complete. 2023-07-13T22:23:08.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.776+0000 7f289db66b80 20 client.26917 trim_cache size 0 max 16384 2023-07-13T22:23:08.763 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:08.763 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:23:08.763 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:23:08.763 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 msgr2=0x7f28700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 0x7f28700a3680 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f28841185d0 tx=0x7f287c0099e0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 msgr2=0x7f288411e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f286c00fa40 tx=0x7f286c00b840 comp rx=0 tx=0).stop 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 shutdown_connections 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f28841080d0 0x7f288411e670 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f28840716f0 0x7f2884117080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f28700a11d0 0x7f28700a3680 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 --2- 172.21.15.26:0/3537871510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2884107720 0x7f28841175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:08.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 >> 172.21.15.26:0/3537871510 conn(0x7f288406cd80 msgr2=0x7f288406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:08.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.777+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 shutdown_connections 2023-07-13T22:23:08.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:08.778+0000 7f288a1ff700 1 -- 172.21.15.26:0/3537871510 wait complete. 2023-07-13T22:23:08.802 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:23:08.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:23:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:08 smithi117 ceph-mon[129074]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 303 MiB/s wr, 101 op/s 2023-07-13T22:23:08.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:08 smithi121 ceph-mon[129187]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 303 MiB/s wr, 101 op/s 2023-07-13T22:23:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.438+0000 7fa5243ff700 1 -- 172.21.15.26:0/2075205037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c100ca0 msgr2=0x7fa51c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.438+0000 7fa5243ff700 1 --2- 172.21.15.26:0/2075205037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c100ca0 0x7fa51c103120 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa514009f90 tx=0x7fa51400b330 comp rx=0 tx=0).stop 2023-07-13T22:23:09.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.447+0000 7fa5243ff700 1 -- 172.21.15.26:0/2075205037 shutdown_connections 2023-07-13T22:23:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.448+0000 7fa5243ff700 1 --2- 172.21.15.26:0/2075205037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c103660 0x7fa51c103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.448+0000 7fa5243ff700 1 --2- 172.21.15.26:0/2075205037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c0fe340 0x7fa51c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.448+0000 7fa5243ff700 1 --2- 172.21.15.26:0/2075205037 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c100ca0 0x7fa51c103120 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.448+0000 7fa5243ff700 1 -- 172.21.15.26:0/2075205037 >> 172.21.15.26:0/2075205037 conn(0x7fa51c0f9fd0 msgr2=0x7fa51c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.458+0000 7fa5243ff700 1 -- 172.21.15.26:0/2075205037 shutdown_connections 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.461+0000 7fa5243ff700 1 -- 172.21.15.26:0/2075205037 wait complete. 2023-07-13T22:23:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.462+0000 7fa5243ff700 1 Processor -- start 2023-07-13T22:23:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 -- start start 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0fe340 0x7fa51c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c103660 0x7fa51c0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10bc20 con 0x7fa51c103660 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10bd90 con 0x7fa51c100ca0 2023-07-13T22:23:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.463+0000 7fa5243ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c10bf50 con 0x7fa51c0fe340 2023-07-13T22:23:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.464+0000 7fa52199a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.464+0000 7fa52199a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48662/0 (socket says 172.21.15.26:48662) 2023-07-13T22:23:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.464+0000 7fa52199a700 1 -- 172.21.15.26:0/101119919 learned_addr learned my addr 172.21.15.26:0/101119919 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.465+0000 7fa52219b700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0fe340 0x7fa51c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.474+0000 7fa52199a700 1 -- 172.21.15.26:0/101119919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0fe340 msgr2=0x7fa51c070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.474+0000 7fa52199a700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0fe340 0x7fa51c070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.474+0000 7fa52199a700 1 -- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c103660 msgr2=0x7fa51c0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.474+0000 7fa52199a700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c103660 0x7fa51c0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:09.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.474+0000 7fa52199a700 1 -- 172.21.15.26:0/101119919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514009c40 con 0x7fa51c100ca0 2023-07-13T22:23:09.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.484+0000 7fa52199a700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa51400b6c0 tx=0x7fa514004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:09.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.492+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51400e040 con 0x7fa51c100ca0 2023-07-13T22:23:09.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.492+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51400be00 con 0x7fa51c100ca0 2023-07-13T22:23:09.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.492+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51401b680 con 0x7fa51c100ca0 2023-07-13T22:23:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.492+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c10c1d0 con 0x7fa51c100ca0 2023-07-13T22:23:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.493+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c10c630 con 0x7fa51c100ca0 2023-07-13T22:23:09.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.495+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa500005320 con 0x7fa51c100ca0 2023-07-13T22:23:09.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.499+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa51401b7e0 con 0x7fa51c100ca0 2023-07-13T22:23:09.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.502+0000 7fa5137fe700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 0x7fa5080a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.502+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa5140cdee0 con 0x7fa51c100ca0 2023-07-13T22:23:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.503+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa514097750 con 0x7fa51c100ca0 2023-07-13T22:23:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.504+0000 7fa52219b700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 0x7fa5080a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:09.507+0000 7fa52219b700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 0x7fa5080a3780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa518009c60 tx=0x7fa518009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.184+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa500005190 con 0x7fa51c100ca0 2023-07-13T22:23:10.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.196+0000 7fa5137fe700 1 -- 172.21.15.26:0/101119919 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fa51409ad70 con 0x7fa51c100ca0 2023-07-13T22:23:10.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:23:10.184 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:23:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.201+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 msgr2=0x7fa5080a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.201+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 0x7fa5080a3780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa518009c60 tx=0x7fa518009260 comp rx=0 tx=0).stop 2023-07-13T22:23:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.201+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 msgr2=0x7fa51c071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.201+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa51400b6c0 tx=0x7fa514004c80 comp rx=0 tx=0).stop 2023-07-13T22:23:10.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 shutdown_connections 2023-07-13T22:23:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa5080a12d0 0x7fa5080a3780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c103660 0x7fa51c0715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa51c0fe340 0x7fa51c070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 --2- 172.21.15.26:0/101119919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa51c100ca0 0x7fa51c071080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.221+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 >> 172.21.15.26:0/101119919 conn(0x7fa51c0f9fd0 msgr2=0x7fa51c0fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.231+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 shutdown_connections 2023-07-13T22:23:10.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.231+0000 7fa5243ff700 1 -- 172.21.15.26:0/101119919 wait complete. 2023-07-13T22:23:10.250 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:23:10.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:23:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:10 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/101119919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:10.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:10 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/101119919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.750+0000 7ff64edff700 1 -- 172.21.15.26:0/1514137769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648100560 msgr2=0x7ff648102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.750+0000 7ff64edff700 1 --2- 172.21.15.26:0/1514137769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648100560 0x7ff648102940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009990 tx=0x7ff63c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:10 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/101119919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.757+0000 7ff64edff700 1 -- 172.21.15.26:0/1514137769 shutdown_connections 2023-07-13T22:23:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.757+0000 7ff64edff700 1 --2- 172.21.15.26:0/1514137769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6481057a0 0x7ff648107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.757+0000 7ff64edff700 1 --2- 172.21.15.26:0/1514137769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff648100560 0x7ff648102940 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.757+0000 7ff64edff700 1 --2- 172.21.15.26:0/1514137769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 0x7ff648105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.757+0000 7ff64edff700 1 -- 172.21.15.26:0/1514137769 >> 172.21.15.26:0/1514137769 conn(0x7ff6480fa000 msgr2=0x7ff6480fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.758+0000 7ff64edff700 1 -- 172.21.15.26:0/1514137769 shutdown_connections 2023-07-13T22:23:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.758+0000 7ff64edff700 1 -- 172.21.15.26:0/1514137769 wait complete. 2023-07-13T22:23:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.759+0000 7ff64edff700 1 Processor -- start 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 -- start start 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 0x7ff64806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6481057a0 0x7ff6481ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480722d0 con 0x7ff648102e80 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648072410 con 0x7ff648100560 2023-07-13T22:23:10.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.760+0000 7ff64edff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480725a0 con 0x7ff6481057a0 2023-07-13T22:23:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.761+0000 7ff64cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.761+0000 7ff64cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48692/0 (socket says 172.21.15.26:48692) 2023-07-13T22:23:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.761+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3277270138 learned_addr learned my addr 172.21.15.26:0/3277270138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.762+0000 7ff64d39c700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6481057a0 0x7ff6481ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.762+0000 7ff647fff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 0x7ff64806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.763+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3277270138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6481057a0 msgr2=0x7ff6481ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.763+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6481057a0 0x7ff6481ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.763+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 msgr2=0x7ff64806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.763+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 0x7ff64806b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.763+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3277270138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0095f0 con 0x7ff648100560 2023-07-13T22:23:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.764+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0095c0 tx=0x7ff63c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.764+0000 7ff645ffb700 1 -- 172.21.15.26:0/3277270138 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c01d070 con 0x7ff648100560 2023-07-13T22:23:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.764+0000 7ff645ffb700 1 -- 172.21.15.26:0/3277270138 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63c005580 con 0x7ff648100560 2023-07-13T22:23:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.764+0000 7ff645ffb700 1 -- 172.21.15.26:0/3277270138 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00fdf0 con 0x7ff648100560 2023-07-13T22:23:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.765+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64810dd20 con 0x7ff648100560 2023-07-13T22:23:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.766+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64810e0f0 con 0x7ff648100560 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 Processor -- start 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 -- start start 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f9839528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f9839536e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983857cb0 con 0x55f9839524d0 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9838aaa40 con 0x55f983952e20 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.773+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9838a3ed0 con 0x55f983953c70 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.774+0000 7ff647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.774+0000 7ff647fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49516/0 (socket says 172.21.15.26:49516) 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.774+0000 7ff647fff700 1 -- 172.21.15.26:0/3978722806 learned_addr learned my addr 172.21.15.26:0/3978722806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff64d39c700 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f9839528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff647fff700 1 -- 172.21.15.26:0/3978722806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 msgr2=0x55f9839536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff647fff700 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f9839536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff647fff700 1 -- 172.21.15.26:0/3978722806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 msgr2=0x55f9839528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff647fff700 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f9839528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.776+0000 7ff647fff700 1 -- 172.21.15.26:0/3978722806 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9836d1de0 con 0x55f983953c70 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.777+0000 7ff647fff700 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff63400bd70 tx=0x7ff6340106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71f15adb716c544 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.778+0000 7ff632ffd700 1 -- 172.21.15.26:0/3978722806 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634010e10 con 0x55f983953c70 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.778+0000 7ff632ffd700 1 -- 172.21.15.26:0/3978722806 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff634015070 con 0x55f983953c70 2023-07-13T22:23:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.778+0000 7ff632ffd700 1 -- 172.21.15.26:0/3978722806 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634011440 con 0x55f983953c70 2023-07-13T22:23:10.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.779+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3978722806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 msgr2=0x55f9839540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.779+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff63400bd70 tx=0x7ff6340106b0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.780+0000 7ff645ffb700 1 -- 172.21.15.26:0/3277270138 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff63c00f4f0 con 0x7ff648100560 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.783+0000 7ff645ffb700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 0x7ff6380a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.783+0000 7ff645ffb700 1 -- 172.21.15.26:0/3277270138 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff63c0cfb90 con 0x7ff648100560 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.784+0000 7ff64d39c700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 0x7ff6380a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3978722806 shutdown_connections 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f9839528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f9839540e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3978722806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f9839536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3978722806 >> 172.21.15.26:0/3978722806 conn(0x55f983947da0 msgr2=0x55f98394d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3978722806 shutdown_connections 2023-07-13T22:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.785+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3978722806 wait complete. 2023-07-13T22:23:10.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.786+0000 7ff6627e0b80 1 -- start start 2023-07-13T22:23:10.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.786+0000 7ff6627e0b80 1 Processor -- start 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983952e20 0x55f98395c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f983953c70 0x55f983963780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983843ac0 con 0x55f983953c70 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9838437c0 con 0x55f9839524d0 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.787+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983727920 con 0x55f983952e20 2023-07-13T22:23:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.788+0000 7ff6627e0b80 10 client.0 _ll_register_callbacks cb 0x55f983841760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.789+0000 7ff64cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.789+0000 7ff64cb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48714/0 (socket says 172.21.15.26:48714) 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.789+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3618709271 learned_addr learned my addr 172.21.15.26:0/3618709271 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.789+0000 7ff64d39c700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 0x7ff6380a3700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff64000a720 tx=0x7ff64000c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff647fff700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983952e20 0x55f98395c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3618709271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983952e20 msgr2=0x55f98395c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983952e20 0x55f98395c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3618709271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f983953c70 msgr2=0x55f983963780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f983953c70 0x55f983963780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.790+0000 7ff64cb9b700 1 -- 172.21.15.26:0/3618709271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63400b9a0 con 0x55f9839524d0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.792+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0994c0 tx=0x7ff63c09d500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bc8cc7468b0cb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.792+0000 7ff62bfff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.792+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c097dd0 con 0x55f9839524d0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.792+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff63c09cfc0 con 0x55f9839524d0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.793+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9837fd4f0 con 0x55f9839524d0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.793+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63c09c920 con 0x55f9839524d0 2023-07-13T22:23:10.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.793+0000 7ff6627e0b80 10 client.26933 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.793+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f9836bac60 con 0x55f9839524d0 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9837fd4f0 con 0x55f9839524d0 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff63c098db0 con 0x55f9839524d0 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 10 client.26933 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 10 client.26933 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 20 client.26933 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f983967ab0 0x55f983969ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 4 client.26933 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.794+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f98389ceb0 con 0x55f983967ab0 2023-07-13T22:23:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.796+0000 7ff647fff700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f983967ab0 0x55f983969ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.798+0000 7ff647fff700 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f983967ab0 0x55f983969ec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc1645d3042e9a3c server_cookie=66ae78e5f64e560e in_seq=0 out_seq=0 2023-07-13T22:23:10.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.798+0000 7ff62bfff700 10 client.26933 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.798+0000 7ff62bfff700 1 -- 172.21.15.26:0/3618709271 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f98389ceb0 con 0x55f983967ab0 2023-07-13T22:23:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.799+0000 7ff62bfff700 10 client.26933 handle_command_reply: tid=0 2023-07-13T22:23:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.799+0000 7ff6627e0b80 1 client.26933 shutdown 2023-07-13T22:23:10.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.799+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 msgr2=0x55f98395c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.799+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0994c0 tx=0x7ff63c09d500 comp rx=0 tx=0).stop 2023-07-13T22:23:10.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 shutdown_connections 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f983967ab0 0x55f983969ec0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f983953c70 0x55f983963780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9839524d0 0x55f98395c200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/3618709271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983952e20 0x55f98395c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 >> 172.21.15.26:0/3618709271 conn(0x55f983947da0 msgr2=0x55f98394c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.816+0000 7ff62bfff700 0 client.26933 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.841+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 shutdown_connections 2023-07-13T22:23:10.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.841+0000 7ff6627e0b80 1 -- 172.21.15.26:0/3618709271 wait complete. 2023-07-13T22:23:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.843+0000 7ff6627e0b80 20 client.26933 trim_cache size 0 max 16384 2023-07-13T22:23:10.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.892+0000 7ff6627e0b80 1 Processor -- start 2023-07-13T22:23:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.921+0000 7ff6627e0b80 1 -- start start 2023-07-13T22:23:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f983964070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f983963740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983959e00 con 0x55f9839524d0 2023-07-13T22:23:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983967830 con 0x55f983952e20 2023-07-13T22:23:10.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.922+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f98394e270 con 0x55f983953c70 2023-07-13T22:23:10.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.933+0000 7ff64d39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.938+0000 7ff64d39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49528/0 (socket says 172.21.15.26:49528) 2023-07-13T22:23:10.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.938+0000 7ff64d39c700 1 -- 172.21.15.26:0/1286914443 learned_addr learned my addr 172.21.15.26:0/1286914443 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.952+0000 7ff647fff700 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f983963740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.953+0000 7ff64d39c700 1 -- 172.21.15.26:0/1286914443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 msgr2=0x55f983963740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.953+0000 7ff64d39c700 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f983963740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.953+0000 7ff64d39c700 1 -- 172.21.15.26:0/1286914443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 msgr2=0x55f983964070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.953+0000 7ff64d39c700 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f983964070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.953+0000 7ff64d39c700 1 -- 172.21.15.26:0/1286914443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9836f8250 con 0x55f983953c70 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.954+0000 7ff64d39c700 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff640018590 tx=0x7ff640004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1d8ff45f388b67 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.960+0000 7ff6337fe700 1 -- 172.21.15.26:0/1286914443 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000f040 con 0x55f983953c70 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.960+0000 7ff6337fe700 1 -- 172.21.15.26:0/1286914443 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff64001c070 con 0x55f983953c70 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.960+0000 7ff6337fe700 1 -- 172.21.15.26:0/1286914443 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640016a70 con 0x55f983953c70 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.961+0000 7ff6627e0b80 1 -- 172.21.15.26:0/1286914443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 msgr2=0x55f98395a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:10.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.961+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff640018590 tx=0x7ff640004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.981+0000 7ff6627e0b80 1 -- 172.21.15.26:0/1286914443 shutdown_connections 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.981+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f983964070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.981+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f98395a050 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.981+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/1286914443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f983963740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.981+0000 7ff6627e0b80 1 -- 172.21.15.26:0/1286914443 >> 172.21.15.26:0/1286914443 conn(0x55f983947da0 msgr2=0x55f98395aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.992+0000 7ff6627e0b80 1 -- 172.21.15.26:0/1286914443 shutdown_connections 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.992+0000 7ff6627e0b80 1 -- 172.21.15.26:0/1286914443 wait complete. 2023-07-13T22:23:10.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.992+0000 7ff6627e0b80 1 -- start start 2023-07-13T22:23:10.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.993+0000 7ff6627e0b80 1 Processor -- start 2023-07-13T22:23:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f98395f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f983962d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f983963280 con 0x55f9839524d0 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9839633f0 con 0x55f983952e20 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.994+0000 7ff6627e0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9839635b0 con 0x55f983953c70 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.995+0000 7ff6627e0b80 10 client.0 _ll_register_callbacks cb 0x55f983841760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.999+0000 7ff647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.999+0000 7ff647fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48734/0 (socket says 172.21.15.26:48734) 2023-07-13T22:23:10.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.999+0000 7ff647fff700 1 -- 172.21.15.26:0/2695646339 learned_addr learned my addr 172.21.15.26:0/2695646339 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:10.999+0000 7ff64d39c700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f983962d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.000+0000 7ff647fff700 1 -- 172.21.15.26:0/2695646339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 msgr2=0x55f983962d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.000+0000 7ff647fff700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f983962d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.000+0000 7ff647fff700 1 -- 172.21.15.26:0/2695646339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 msgr2=0x55f98395f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.000+0000 7ff647fff700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f98395f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.000+0000 7ff647fff700 1 -- 172.21.15.26:0/2695646339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64000a380 con 0x55f983952e20 2023-07-13T22:23:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.010+0000 7ff647fff700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff63400bd70 tx=0x7ff634005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62eac452ba562f0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.010+0000 7ff62bfff700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.021+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63401f040 con 0x55f983952e20 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.021+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff634017070 con 0x55f983952e20 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.021+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff634015070 con 0x55f983952e20 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.022+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff634029610 con 0x55f983952e20 2023-07-13T22:23:11.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.022+0000 7ff6627e0b80 10 client.26939 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.022+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f9836d02b0 con 0x55f983952e20 2023-07-13T22:23:11.032 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.023+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff634015070 con 0x55f983952e20 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.023+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff6340103d0 con 0x55f983952e20 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 10 client.26939 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 10 client.26939 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 20 client.26939 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:11.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f9839bcb60 0x55f9839bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 4 client.26939 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f98385f050 con 0x55f9839bcb60 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.024+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f9839bcb60 0x55f9839bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.028+0000 7ff64cb9b700 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f9839bcb60 0x55f9839bef70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a401fb026c0749ff server_cookie=6eea65d981e17763 in_seq=0 out_seq=0 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.028+0000 7ff62bfff700 10 client.26939 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff62bfff700 1 -- 172.21.15.26:0/2695646339 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f98385f050 con 0x55f9839bcb60 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff62bfff700 10 client.26939 handle_command_reply: tid=0 2023-07-13T22:23:11.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff6627e0b80 1 client.26939 shutdown 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 msgr2=0x55f98395fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff63400bd70 tx=0x7ff634005ce0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 shutdown_connections 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.029+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55f9839bcb60 0x55f9839bef70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9839524d0 0x55f98395f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f983953c70 0x55f983962d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 --2- 172.21.15.26:0/2695646339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f983952e20 0x55f98395fd20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 >> 172.21.15.26:0/2695646339 conn(0x55f983947da0 msgr2=0x55f98395a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff62bfff700 0 client.26939 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 shutdown_connections 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 1 -- 172.21.15.26:0/2695646339 wait complete. 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.030+0000 7ff6627e0b80 20 client.26939 trim_cache size 0 max 16384 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.042+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 msgr2=0x7ff6380a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.042+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 0x7ff6380a3700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff64000a720 tx=0x7ff64000c040 comp rx=0 tx=0).stop 2023-07-13T22:23:11.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.042+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 msgr2=0x7ff64806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.042+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0095c0 tx=0x7ff63c003b90 comp rx=0 tx=0).stop 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 shutdown_connections 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff6481057a0 0x7ff6481ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff648100560 0x7ff64806ac90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff6380a1250 0x7ff6380a3700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 --2- 172.21.15.26:0/3277270138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff648102e80 0x7ff64806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 >> 172.21.15.26:0/3277270138 conn(0x7ff6480fa000 msgr2=0x7ff6481011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 shutdown_connections 2023-07-13T22:23:11.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:11.043+0000 7ff64edff700 1 -- 172.21.15.26:0/3277270138 wait complete. 2023-07-13T22:23:11.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:23:11.383 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:11 smithi026 ceph-mon[95647]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:23:11.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:11 smithi117 ceph-mon[129074]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:23:11.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:11 smithi121 ceph-mon[129187]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 291 MiB/s wr, 94 op/s 2023-07-13T22:23:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.001+0000 7f6d201ff700 1 -- 172.21.15.26:0/744694924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18070fb0 msgr2=0x7f6d180713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.001+0000 7f6d201ff700 1 --2- 172.21.15.26:0/744694924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18070fb0 0x7f6d180713c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009990 tx=0x7f6d14009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 -- 172.21.15.26:0/744694924 shutdown_connections 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 --2- 172.21.15.26:0/744694924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d1810bff0 0x7f6d1810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 --2- 172.21.15.26:0/744694924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18070fb0 0x7f6d180713c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 --2- 172.21.15.26:0/744694924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d18109630 0x7f6d1810bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 -- 172.21.15.26:0/744694924 >> 172.21.15.26:0/744694924 conn(0x7f6d1806ca30 msgr2=0x7f6d1806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 -- 172.21.15.26:0/744694924 shutdown_connections 2023-07-13T22:23:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.010+0000 7f6d201ff700 1 -- 172.21.15.26:0/744694924 wait complete. 2023-07-13T22:23:12.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.012+0000 7f6d201ff700 1 Processor -- start 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.012+0000 7f6d201ff700 1 -- start start 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d18070fb0 0x7f6d181a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1810bff0 0x7f6d181abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1810ea20 con 0x7f6d1810bff0 2023-07-13T22:23:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1810eb90 con 0x7f6d18070fb0 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.013+0000 7f6d201ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1810ed50 con 0x7f6d18109630 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.014+0000 7f6d1d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.014+0000 7f6d1d79a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49554/0 (socket says 172.21.15.26:49554) 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.014+0000 7f6d1d79a700 1 -- 172.21.15.26:0/2497859779 learned_addr learned my addr 172.21.15.26:0/2497859779 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.014+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d18070fb0 0x7f6d181a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.015+0000 7f6d1d79a700 1 -- 172.21.15.26:0/2497859779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d18070fb0 msgr2=0x7f6d181a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.015+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d18070fb0 0x7f6d181a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.015+0000 7f6d1d79a700 1 -- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1810bff0 msgr2=0x7f6d181abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.015+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1810bff0 0x7f6d181abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.015+0000 7f6d1d79a700 1 -- 172.21.15.26:0/2497859779 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140095f0 con 0x7f6d18109630 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.016+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b530 tx=0x7f6d0800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.017+0000 7f6d0effd700 1 -- 172.21.15.26:0/2497859779 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08033640 con 0x7f6d18109630 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.017+0000 7f6d0effd700 1 -- 172.21.15.26:0/2497859779 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d08033c60 con 0x7f6d18109630 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.017+0000 7f6d0effd700 1 -- 172.21.15.26:0/2497859779 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08032df0 con 0x7f6d18109630 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.017+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1810efd0 con 0x7f6d18109630 2023-07-13T22:23:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.018+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1810f510 con 0x7f6d18109630 2023-07-13T22:23:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.027+0000 7f6d33afab80 1 Processor -- start 2023-07-13T22:23:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.027+0000 7f6d33afab80 1 -- start start 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702de20 0x5565c702e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702ec70 0x5565c702f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6f32cb0 con 0x5565c702de20 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6f85a40 con 0x5565c702ec70 2023-07-13T22:23:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.028+0000 7f6d33afab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6f7eed0 con 0x5565c702d4d0 2023-07-13T22:23:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.030+0000 7f6d0effd700 1 -- 172.21.15.26:0/2497859779 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6d08032490 con 0x7f6d18109630 2023-07-13T22:23:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.032+0000 7f6d0effd700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 0x7f6d040a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.032+0000 7f6d0effd700 1 -- 172.21.15.26:0/2497859779 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d080edf30 con 0x7f6d18109630 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.035+0000 7f6d1e79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.035+0000 7f6d1e79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49556/0 (socket says 172.21.15.26:49556) 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.035+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2618357318 learned_addr learned my addr 172.21.15.26:0/2618357318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.035+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702ec70 0x5565c702f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.036+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2618357318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702ec70 msgr2=0x5565c702f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.036+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702ec70 0x5565c702f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.036+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2618357318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702de20 msgr2=0x5565c702e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.036+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702de20 0x5565c702e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.036+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2618357318 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c6dacde0 con 0x5565c702d4d0 2023-07-13T22:23:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.037+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000a650 tx=0x7f6d1000a960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59830d35efa5d606 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.037+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 0x7f6d040a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.040+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 0x7f6d040a3800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6d1001b000 tx=0x7f6d10014df0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.049+0000 7f6d03fff700 1 -- 172.21.15.26:0/2618357318 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1001b030 con 0x5565c702d4d0 2023-07-13T22:23:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.049+0000 7f6d03fff700 1 -- 172.21.15.26:0/2618357318 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d10008ab0 con 0x5565c702d4d0 2023-07-13T22:23:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.049+0000 7f6d03fff700 1 -- 172.21.15.26:0/2618357318 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10014bb0 con 0x5565c702d4d0 2023-07-13T22:23:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.051+0000 7f6d33afab80 1 -- 172.21.15.26:0/2618357318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 msgr2=0x5565c702d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.051+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000a650 tx=0x7f6d1000a960 comp rx=0 tx=0).stop 2023-07-13T22:23:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 -- 172.21.15.26:0/2618357318 shutdown_connections 2023-07-13T22:23:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702ec70 0x5565c702f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c702d8e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2618357318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702de20 0x5565c702e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 -- 172.21.15.26:0/2618357318 >> 172.21.15.26:0/2618357318 conn(0x5565c7022da0 msgr2=0x5565c70281d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 -- 172.21.15.26:0/2618357318 shutdown_connections 2023-07-13T22:23:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.052+0000 7f6d33afab80 1 -- 172.21.15.26:0/2618357318 wait complete. 2023-07-13T22:23:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.053+0000 7f6d33afab80 1 -- start start 2023-07-13T22:23:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.053+0000 7f6d33afab80 1 Processor -- start 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c70371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 0x5565c7037720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6f1eac0 con 0x5565c702ec70 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6f1e7c0 con 0x5565c702de20 2023-07-13T22:23:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c6e02920 con 0x5565c702d4d0 2023-07-13T22:23:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.054+0000 7f6d33afab80 10 client.0 _ll_register_callbacks cb 0x5565c6f1c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.056+0000 7f6d1d79a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 0x5565c7037720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.056+0000 7f6d1df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c70371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.057+0000 7f6d1df9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c70371e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49564/0 (socket says 172.21.15.26:49564) 2023-07-13T22:23:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.057+0000 7f6d1df9b700 1 -- 172.21.15.26:0/1813057091 learned_addr learned my addr 172.21.15.26:0/1813057091 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.058+0000 7f6d1d79a700 1 -- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 msgr2=0x5565c70371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.058+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c70371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.058+0000 7f6d1d79a700 1 -- 172.21.15.26:0/1813057091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 msgr2=0x5565c703e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.058+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.058+0000 7f6d1d79a700 1 -- 172.21.15.26:0/1813057091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d10007e10 con 0x5565c702de20 2023-07-13T22:23:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.072+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 0x5565c7037720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b500 tx=0x7f6d08033dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2bc0883e2b9cd47 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.085+0000 7f6d017fa700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:12.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.085+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ebea0 con 0x5565c702de20 2023-07-13T22:23:12.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.085+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d080bf320 con 0x5565c702de20 2023-07-13T22:23:12.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.086+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565c6ed84f0 con 0x5565c702de20 2023-07-13T22:23:12.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.087+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d080c3740 con 0x5565c702de20 2023-07-13T22:23:12.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.087+0000 7f6d33afab80 10 client.26948 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:12.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.087+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565c6d95c60 con 0x5565c702de20 2023-07-13T22:23:12.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565c6ed84f0 con 0x5565c702de20 2023-07-13T22:23:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d080c32b0 con 0x5565c702de20 2023-07-13T22:23:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 10 client.26948 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:12.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 10 client.26948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:23:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 20 client.26948 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7042fc0 0x5565c70453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 4 client.26948 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.098+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565c6f77eb0 con 0x5565c7042fc0 2023-07-13T22:23:12.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.102+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7042fc0 0x5565c70453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.134+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7042fc0 0x5565c70453d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e1e295323de0d2f server_cookie=aa9b4916c74c6031 in_seq=0 out_seq=0 2023-07-13T22:23:12.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.148+0000 7f6d017fa700 10 client.26948 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.149+0000 7f6d017fa700 1 -- 172.21.15.26:0/1813057091 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565c6f77eb0 con 0x5565c7042fc0 2023-07-13T22:23:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.149+0000 7f6d017fa700 10 client.26948 handle_command_reply: tid=0 2023-07-13T22:23:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.149+0000 7f6d33afab80 1 client.26948 shutdown 2023-07-13T22:23:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.152+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 msgr2=0x5565c7037720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.152+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 0x5565c7037720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b500 tx=0x7f6d08033dd0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 shutdown_connections 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7042fc0 0x5565c70453d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c702d4d0 0x5565c70371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 --2- 172.21.15.26:0/1813057091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702de20 0x5565c7037720 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b500 tx=0x7f6d08033dd0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.153+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 >> 172.21.15.26:0/1813057091 conn(0x5565c7022da0 msgr2=0x5565c70278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.160+0000 7f6d017fa700 0 client.26948 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:12.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.185+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 shutdown_connections 2023-07-13T22:23:12.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.187+0000 7f6d33afab80 1 -- 172.21.15.26:0/1813057091 wait complete. 2023-07-13T22:23:12.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.187+0000 7f6d33afab80 20 client.26948 trim_cache size 0 max 16384 2023-07-13T22:23:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 Processor -- start 2023-07-13T22:23:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 -- start start 2023-07-13T22:23:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c7031bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c7036690 con 0x5565c702ec70 2023-07-13T22:23:12.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c7036d50 con 0x5565c702d4d0 2023-07-13T22:23:12.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.216+0000 7f6d33afab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c7029ca0 con 0x5565c703f470 2023-07-13T22:23:12.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.218+0000 7f6d1df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.218+0000 7f6d1e79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c7031bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.218+0000 7f6d1df9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:48784/0 (socket says 172.21.15.26:48784) 2023-07-13T22:23:12.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.218+0000 7f6d1df9b700 1 -- 172.21.15.26:0/4108658563 learned_addr learned my addr 172.21.15.26:0/4108658563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.221+0000 7f6d1d79a700 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 -- 172.21.15.26:0/4108658563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 msgr2=0x5565c7031bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c7031bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 -- 172.21.15.26:0/4108658563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 msgr2=0x5565c703e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 -- 172.21.15.26:0/4108658563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565c6dd3250 con 0x5565c702d4d0 2023-07-13T22:23:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.228+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d140098e0 tx=0x7f6d14019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f71d650d7cfca1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.234+0000 7f6d0cff9700 1 -- 172.21.15.26:0/4108658563 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1401d040 con 0x5565c702d4d0 2023-07-13T22:23:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.234+0000 7f6d0cff9700 1 -- 172.21.15.26:0/4108658563 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d14029070 con 0x5565c702d4d0 2023-07-13T22:23:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.234+0000 7f6d0cff9700 1 -- 172.21.15.26:0/4108658563 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14025050 con 0x5565c702d4d0 2023-07-13T22:23:12.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.241+0000 7f6d33afab80 1 -- 172.21.15.26:0/4108658563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 msgr2=0x5565c703ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.241+0000 7f6d33afab80 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d140098e0 tx=0x7f6d14019040 comp rx=0 tx=0).stop 2023-07-13T22:23:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 -- 172.21.15.26:0/4108658563 shutdown_connections 2023-07-13T22:23:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c7031bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703ef30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 --2- 172.21.15.26:0/4108658563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 -- 172.21.15.26:0/4108658563 >> 172.21.15.26:0/4108658563 conn(0x5565c7022da0 msgr2=0x5565c7044fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 -- 172.21.15.26:0/4108658563 shutdown_connections 2023-07-13T22:23:12.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.242+0000 7f6d33afab80 1 -- 172.21.15.26:0/4108658563 wait complete. 2023-07-13T22:23:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.243+0000 7f6d33afab80 1 -- start start 2023-07-13T22:23:12.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.244+0000 7f6d33afab80 1 Processor -- start 2023-07-13T22:23:12.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c703b8f0 con 0x5565c702ec70 2023-07-13T22:23:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c703ba60 con 0x5565c702d4d0 2023-07-13T22:23:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565c7096e70 con 0x5565c703f470 2023-07-13T22:23:12.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.245+0000 7f6d33afab80 10 client.0 _ll_register_callbacks cb 0x7f6d080bae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.246+0000 7f6d1e79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.246+0000 7f6d1e79c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:49570/0 (socket says 172.21.15.26:49570) 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.246+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2899086538 learned_addr learned my addr 172.21.15.26:0/2899086538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2899086538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 msgr2=0x5565c703e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2899086538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 msgr2=0x5565c703ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 -- 172.21.15.26:0/2899086538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d18117750 con 0x5565c703f470 2023-07-13T22:23:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.247+0000 7f6d1e79c700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6d10016de0 tx=0x7f6d10013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32cd118a9f41a8f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.249+0000 7f6d017fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.249+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10007590 con 0x5565c703f470 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.249+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6d1001c070 con 0x5565c703f470 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.249+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d10008ab0 con 0x5565c703f470 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.250+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d10003c00 con 0x5565c703f470 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.250+0000 7f6d33afab80 10 client.26959 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.250+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5565c6dab2b0 con 0x5565c703f470 2023-07-13T22:23:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.250+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d10008ab0 con 0x5565c703f470 2023-07-13T22:23:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.251+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6d100143a0 con 0x5565c703f470 2023-07-13T22:23:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.251+0000 7f6d33afab80 10 client.26959 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.251+0000 7f6d33afab80 10 client.26959 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:23:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.251+0000 7f6d33afab80 20 client.26959 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.252+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7098720 0x5565c709ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.252+0000 7f6d33afab80 4 client.26959 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.252+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5565c6f3a050 con 0x5565c7098720 2023-07-13T22:23:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.252+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7098720 0x5565c709ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d1df9b700 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7098720 0x5565c709ab30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f73746c13f334827 server_cookie=a5998caa71609fd7 in_seq=0 out_seq=0 2023-07-13T22:23:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d017fa700 10 client.26959 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d017fa700 1 -- 172.21.15.26:0/2899086538 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5565c6f3a050 con 0x5565c7098720 2023-07-13T22:23:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d017fa700 10 client.26959 handle_command_reply: tid=0 2023-07-13T22:23:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d33afab80 1 client.26959 shutdown 2023-07-13T22:23:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 msgr2=0x5565c703b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.253+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6d10016de0 tx=0x7f6d10013000 comp rx=0 tx=0).stop 2023-07-13T22:23:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 shutdown_connections 2023-07-13T22:23:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5565c7098720 0x5565c709ab30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565c703f470 0x5565c703b3b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565c702d4d0 0x5565c703e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 --2- 172.21.15.26:0/2899086538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5565c702ec70 0x5565c703ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 >> 172.21.15.26:0/2899086538 conn(0x5565c7022da0 msgr2=0x5565c7031bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d017fa700 0 client.26959 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 shutdown_connections 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 1 -- 172.21.15.26:0/2899086538 wait complete. 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.254+0000 7f6d33afab80 20 client.26959 trim_cache size 0 max 16384 2023-07-13T22:23:12.241 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.255+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 msgr2=0x7f6d040a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.255+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 0x7f6d040a3800 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6d1001b000 tx=0x7f6d10014df0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.255+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 msgr2=0x7f6d181a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.255+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b530 tx=0x7f6d0800b840 comp rx=0 tx=0).stop 2023-07-13T22:23:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 shutdown_connections 2023-07-13T22:23:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6d040a1350 0x7f6d040a3800 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1810bff0 0x7f6d181abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d18070fb0 0x7f6d181a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 --2- 172.21.15.26:0/2497859779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d18109630 0x7f6d181a4c70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.256+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 >> 172.21.15.26:0/2497859779 conn(0x7f6d1806ca30 msgr2=0x7f6d1810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.257+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 shutdown_connections 2023-07-13T22:23:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.257+0000 7f6d201ff700 1 -- 172.21.15.26:0/2497859779 wait complete. 2023-07-13T22:23:12.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:23:12.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.855+0000 7f018d3ff700 1 -- 172.21.15.26:0/2289150346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188070ee0 msgr2=0x7f01880712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.855+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2289150346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188070ee0 0x7f01880712f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0178009990 tx=0x7f0178009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.856+0000 7f018d3ff700 1 -- 172.21.15.26:0/2289150346 shutdown_connections 2023-07-13T22:23:12.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.856+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2289150346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f018810bd60 0x7f018810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.856+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2289150346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188070ee0 0x7f01880712f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.856+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2289150346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0188071830 0x7f018810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.856+0000 7f018d3ff700 1 -- 172.21.15.26:0/2289150346 >> 172.21.15.26:0/2289150346 conn(0x7f018806ca90 msgr2=0x7f018806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.858+0000 7f018d3ff700 1 -- 172.21.15.26:0/2289150346 shutdown_connections 2023-07-13T22:23:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.866+0000 7f018d3ff700 1 -- 172.21.15.26:0/2289150346 wait complete. 2023-07-13T22:23:12.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.869+0000 7f018d3ff700 1 Processor -- start 2023-07-13T22:23:12.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 -- start start 2023-07-13T22:23:12.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0188070ee0 0x7f01881a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 0x7f01881a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881ac280 con 0x7f018810bd60 2023-07-13T22:23:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881ac3c0 con 0x7f0188071830 2023-07-13T22:23:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.870+0000 7f018d3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01881ac550 con 0x7f0188070ee0 2023-07-13T22:23:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.872+0000 7f01877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.872+0000 7f01877fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49152/0 (socket says 172.21.15.26:49152) 2023-07-13T22:23:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.872+0000 7f01877fe700 1 -- 172.21.15.26:0/2867439395 learned_addr learned my addr 172.21.15.26:0/2867439395 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.872+0000 7f01867fc700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 0x7f01881a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f0186ffd700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0188070ee0 0x7f01881a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01877fe700 1 -- 172.21.15.26:0/2867439395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0188070ee0 msgr2=0x7f01881a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01877fe700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0188070ee0 0x7f01881a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01877fe700 1 -- 172.21.15.26:0/2867439395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 msgr2=0x7f01881a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01877fe700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 0x7f01881a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01877fe700 1 -- 172.21.15.26:0/2867439395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780095f0 con 0x7f018810bd60 2023-07-13T22:23:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.873+0000 7f01867fc700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 0x7f01881a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:23:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.874+0000 7f01877fe700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f018000ada0 tx=0x7f018000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.874+0000 7f016ffff700 1 -- 172.21.15.26:0/2867439395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018000f070 con 0x7f018810bd60 2023-07-13T22:23:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.874+0000 7f016ffff700 1 -- 172.21.15.26:0/2867439395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0180007b30 con 0x7f018810bd60 2023-07-13T22:23:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.874+0000 7f016ffff700 1 -- 172.21.15.26:0/2867439395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800053e0 con 0x7f018810bd60 2023-07-13T22:23:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.874+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01881ac830 con 0x7f018810bd60 2023-07-13T22:23:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.875+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01881acd70 con 0x7f018810bd60 2023-07-13T22:23:12.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.878+0000 7f016ffff700 1 -- 172.21.15.26:0/2867439395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f018000c980 con 0x7f018810bd60 2023-07-13T22:23:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.879+0000 7f01a0df5b80 1 Processor -- start 2023-07-13T22:23:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 -- start start 2023-07-13T22:23:12.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe2088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe20a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe10dcb0 con 0x562afe2084d0 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe160a40 con 0x562afe208e20 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.880+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe159ed0 con 0x562afe209c70 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59856/0 (socket says 172.21.15.26:59856) 2023-07-13T22:23:12.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 -- 172.21.15.26:0/1484027131 learned_addr learned my addr 172.21.15.26:0/1484027131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f016ffff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 0x7f01700a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f01867fc700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe20a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f016ffff700 1 -- 172.21.15.26:0/2867439395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f01800cbe40 con 0x7f018810bd60 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f01867fc700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe2088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 -- 172.21.15.26:0/1484027131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 msgr2=0x562afe20a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe20a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 -- 172.21.15.26:0/1484027131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 msgr2=0x562afe2088e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe2088e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f0186ffd700 1 -- 172.21.15.26:0/1484027131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562afdf87de0 con 0x562afe208e20 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f01867fc700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe2088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.881+0000 7f01877fe700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 0x7f01700a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.882+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0178015000 tx=0x7f0178009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8b9707faa62090 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.882+0000 7f016d7fa700 1 -- 172.21.15.26:0/1484027131 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800b9d0 con 0x562afe208e20 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.882+0000 7f01877fe700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 0x7f01700a3780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f018000a180 tx=0x7f0180004cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.882+0000 7f016d7fa700 1 -- 172.21.15.26:0/1484027131 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017801e040 con 0x562afe208e20 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.882+0000 7f016d7fa700 1 -- 172.21.15.26:0/1484027131 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800f460 con 0x562afe208e20 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1484027131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 msgr2=0x562afe2096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0178015000 tx=0x7f0178009f20 comp rx=0 tx=0).stop 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1484027131 shutdown_connections 2023-07-13T22:23:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe2088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe20a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1484027131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe2096e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1484027131 >> 172.21.15.26:0/1484027131 conn(0x562afe1fdda0 msgr2=0x562afe2031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1484027131 shutdown_connections 2023-07-13T22:23:12.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.883+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1484027131 wait complete. 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.884+0000 7f01a0df5b80 1 -- start start 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.884+0000 7f01a0df5b80 1 Processor -- start 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.884+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.884+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe212680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe2196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe0f9ac0 con 0x562afe2084d0 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe0f97c0 con 0x562afe208e20 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afdfdd920 con 0x562afe209c70 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01a0df5b80 10 client.0 _ll_register_callbacks cb 0x562afe0f7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50318/0 (socket says 172.21.15.26:50318) 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 -- 172.21.15.26:0/2049161416 learned_addr learned my addr 172.21.15.26:0/2049161416 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f0186ffd700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe212680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe2196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 -- 172.21.15.26:0/2049161416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 msgr2=0x562afe2196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe2196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 -- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 msgr2=0x562afe212680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe212680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.885+0000 7f01867fc700 1 -- 172.21.15.26:0/2049161416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0188117640 con 0x562afe2084d0 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f01867fc700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f017c0093b0 tx=0x7f017c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272f6282931becb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f016aff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c014430 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562afe0b34f0 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f017c014590 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017c014750 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f01a0df5b80 10 client.16452 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.886+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562afdf70c60 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562afe0b34f0 con 0x562afe2084d0 2023-07-13T22:23:12.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f017c017810 con 0x562afe2084d0 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 10 client.16452 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 10 client.16452 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 20 client.16452 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe21de40 0x562afe220250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 4 client.16452 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:12.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562afe152eb0 con 0x562afe21de40 2023-07-13T22:23:12.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.887+0000 7f0186ffd700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe21de40 0x562afe220250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.895+0000 7f0186ffd700 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe21de40 0x562afe220250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4f08aeac4a0dda9 server_cookie=8332298797c2a9c9 in_seq=0 out_seq=0 2023-07-13T22:23:12.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.896+0000 7f016aff5700 10 client.16452 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:12.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.906+0000 7f016aff5700 1 -- 172.21.15.26:0/2049161416 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562afe152eb0 con 0x562afe21de40 2023-07-13T22:23:12.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.906+0000 7f016aff5700 10 client.16452 handle_command_reply: tid=0 2023-07-13T22:23:12.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.906+0000 7f01a0df5b80 1 client.16452 shutdown 2023-07-13T22:23:12.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.939+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 msgr2=0x562afe212140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.939+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f017c0093b0 tx=0x7f017c009190 comp rx=0 tx=0).stop 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 shutdown_connections 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe2084d0 0x562afe212140 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe21de40 0x562afe220250 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe2196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/2049161416 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe208e20 0x562afe212680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 >> 172.21.15.26:0/2049161416 conn(0x562afe1fdda0 msgr2=0x562afe2028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.950+0000 7f016aff5700 0 client.16452 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:12.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.961+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 shutdown_connections 2023-07-13T22:23:12.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.961+0000 7f01a0df5b80 1 -- 172.21.15.26:0/2049161416 wait complete. 2023-07-13T22:23:12.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.962+0000 7f01a0df5b80 20 client.16452 trim_cache size 0 max 16384 2023-07-13T22:23:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.994+0000 7f01a0df5b80 1 Processor -- start 2023-07-13T22:23:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 -- start start 2023-07-13T22:23:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe2084d0 0x562afe210040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe219680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe1fd970 con 0x562afe208e20 2023-07-13T22:23:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe202fc0 con 0x562afe2084d0 2023-07-13T22:23:12.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.995+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe20fdf0 con 0x562afe209c70 2023-07-13T22:23:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.996+0000 7f01867fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.996+0000 7f01867fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe2084d0 0x562afe210040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.996+0000 7f01867fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50506/0 (socket says 172.21.15.26:50506) 2023-07-13T22:23:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.996+0000 7f01867fc700 1 -- 172.21.15.26:0/1405854410 learned_addr learned my addr 172.21.15.26:0/1405854410 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 -- 172.21.15.26:0/1405854410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe2084d0 msgr2=0x562afe210040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe2084d0 0x562afe210040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 -- 172.21.15.26:0/1405854410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 msgr2=0x562afe219680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe219680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 -- 172.21.15.26:0/1405854410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562afdfae250 con 0x562afe209c70 2023-07-13T22:23:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.997+0000 7f01867fc700 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f017c017e10 tx=0x7f017c004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b5d1f9b304231f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.998+0000 7f016dffb700 1 -- 172.21.15.26:0/1405854410 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c009b10 con 0x562afe209c70 2023-07-13T22:23:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.998+0000 7f016dffb700 1 -- 172.21.15.26:0/1405854410 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017c01d070 con 0x562afe209c70 2023-07-13T22:23:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.998+0000 7f016dffb700 1 -- 172.21.15.26:0/1405854410 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c0149f0 con 0x562afe209c70 2023-07-13T22:23:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1405854410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 msgr2=0x562afe21fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f017c017e10 tx=0x7f017c004860 comp rx=0 tx=0).stop 2023-07-13T22:23:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1405854410 shutdown_connections 2023-07-13T22:23:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe209c70 0x562afe21fe50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe2084d0 0x562afe210040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1405854410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe219680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:12.999+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1405854410 >> 172.21.15.26:0/1405854410 conn(0x562afe1fdda0 msgr2=0x562afe2028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.000+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1405854410 shutdown_connections 2023-07-13T22:23:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.000+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1405854410 wait complete. 2023-07-13T22:23:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.000+0000 7f01a0df5b80 1 -- start start 2023-07-13T22:23:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.001+0000 7f01a0df5b80 1 Processor -- start 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe2084d0 0x562afe213a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe209c70 0x562afe218fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe219500 con 0x562afe208e20 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe219670 con 0x562afe209c70 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562afe219830 con 0x562afe2084d0 2023-07-13T22:23:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f01a0df5b80 10 client.0 _ll_register_callbacks cb 0x7f017c0198e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:12.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f0186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f0186ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50328/0 (socket says 172.21.15.26:50328) 2023-07-13T22:23:12.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.002+0000 7f0186ffd700 1 -- 172.21.15.26:0/1910455431 learned_addr learned my addr 172.21.15.26:0/1910455431 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:12.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.003+0000 7f0186ffd700 1 -- 172.21.15.26:0/1910455431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe2084d0 msgr2=0x562afe213a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.003+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe2084d0 0x562afe213a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.003+0000 7f0186ffd700 1 -- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe209c70 msgr2=0x562afe218fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:12.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.003+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe209c70 0x562afe218fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:12.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.003+0000 7f0186ffd700 1 -- 172.21.15.26:0/1910455431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562afdf70c60 con 0x562afe208e20 2023-07-13T22:23:12.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.004+0000 7f0186ffd700 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f017800c010 tx=0x7f01780057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd7bad405352e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:12.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.004+0000 7f016aff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:12.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.005+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800baf0 con 0x562afe208e20 2023-07-13T22:23:12.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.005+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f01780052d0 con 0x562afe208e20 2023-07-13T22:23:12.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.005+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f017801e040 con 0x562afe208e20 2023-07-13T22:23:12.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.006+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017802ea60 con 0x562afe208e20 2023-07-13T22:23:12.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.006+0000 7f01a0df5b80 10 client.16455 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:12.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.006+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562afdf862b0 con 0x562afe208e20 2023-07-13T22:23:12.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.009+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f017801e040 con 0x562afe208e20 2023-07-13T22:23:12.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.009+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f017802a040 con 0x562afe208e20 2023-07-13T22:23:12.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:12.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:23:12.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 20 client.16455 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:12.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe283da0 0x562afe2861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:12.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 4 client.16455 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:12.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562afe115050 con 0x562afe283da0 2023-07-13T22:23:12.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.011+0000 7f01867fc700 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe283da0 0x562afe2861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:12.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.013+0000 7f01867fc700 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe283da0 0x562afe2861b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d874a09f3feb7b20 server_cookie=9eed7cd8778b4ea5 in_seq=0 out_seq=0 2023-07-13T22:23:12.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.014+0000 7f016aff5700 10 client.16455 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:12.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.014+0000 7f016aff5700 1 -- 172.21.15.26:0/1910455431 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562afe115050 con 0x562afe283da0 2023-07-13T22:23:12.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.014+0000 7f016aff5700 10 client.16455 handle_command_reply: tid=0 2023-07-13T22:23:12.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.014+0000 7f01a0df5b80 1 client.16455 shutdown 2023-07-13T22:23:13.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 msgr2=0x562afe213f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f017800c010 tx=0x7f01780057e0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 shutdown_connections 2023-07-13T22:23:13.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x562afe283da0 0x562afe2861b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562afe209c70 0x562afe218fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562afe2084d0 0x562afe213a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 --2- 172.21.15.26:0/1910455431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562afe208e20 0x562afe213f90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.025+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 >> 172.21.15.26:0/1910455431 conn(0x562afe1fdda0 msgr2=0x562afe21fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.026+0000 7f016aff5700 0 client.16455 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.026+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 shutdown_connections 2023-07-13T22:23:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.026+0000 7f01a0df5b80 1 -- 172.21.15.26:0/1910455431 wait complete. 2023-07-13T22:23:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.027+0000 7f01a0df5b80 20 client.16455 trim_cache size 0 max 16384 2023-07-13T22:23:13.012 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.038+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 msgr2=0x7f01700a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.038+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 0x7f01700a3780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f018000a180 tx=0x7f0180004cc0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.038+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 msgr2=0x7f01881abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.038+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f018000ada0 tx=0x7f018000c330 comp rx=0 tx=0).stop 2023-07-13T22:23:13.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 shutdown_connections 2023-07-13T22:23:13.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f01700a12d0 0x7f01700a3780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f018810bd60 0x7f01881abcb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0188070ee0 0x7f01881a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 --2- 172.21.15.26:0/2867439395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0188071830 0x7f01881a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 >> 172.21.15.26:0/2867439395 conn(0x7f018806ca90 msgr2=0x7f018806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 shutdown_connections 2023-07-13T22:23:13.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.039+0000 7f018d3ff700 1 -- 172.21.15.26:0/2867439395 wait complete. 2023-07-13T22:23:13.085 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:23:13.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.681+0000 7f2c76bff700 1 -- 172.21.15.26:0/3610237430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 msgr2=0x7f2c7010a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.681+0000 7f2c76bff700 1 --2- 172.21.15.26:0/3610237430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 0x7f2c7010a150 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b0a0 tx=0x7f2c6800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.683+0000 7f2c76bff700 1 -- 172.21.15.26:0/3610237430 shutdown_connections 2023-07-13T22:23:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.683+0000 7f2c76bff700 1 --2- 172.21.15.26:0/3610237430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 0x7f2c7010a150 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.683+0000 7f2c76bff700 1 --2- 172.21.15.26:0/3610237430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c70101d10 0x7f2c70102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.683+0000 7f2c76bff700 1 --2- 172.21.15.26:0/3610237430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70102f00 0x7f2c70103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.683+0000 7f2c76bff700 1 -- 172.21.15.26:0/3610237430 >> 172.21.15.26:0/3610237430 conn(0x7f2c700fd350 msgr2=0x7f2c700ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.691+0000 7f2c76bff700 1 -- 172.21.15.26:0/3610237430 shutdown_connections 2023-07-13T22:23:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.691+0000 7f2c76bff700 1 -- 172.21.15.26:0/3610237430 wait complete. 2023-07-13T22:23:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.693+0000 7f2c76bff700 1 Processor -- start 2023-07-13T22:23:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.693+0000 7f2c76bff700 1 -- start start 2023-07-13T22:23:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c70102f00 0x7f2c7006c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 0x7f2c70071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70071920 con 0x7f2c70103890 2023-07-13T22:23:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70071a90 con 0x7f2c70102f00 2023-07-13T22:23:13.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c76bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70071c50 con 0x7f2c70101d10 2023-07-13T22:23:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50530/0 (socket says 172.21.15.26:50530) 2023-07-13T22:23:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.694+0000 7f2c7499b700 1 -- 172.21.15.26:0/968249137 learned_addr learned my addr 172.21.15.26:0/968249137 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.695+0000 7f2c7499b700 1 -- 172.21.15.26:0/968249137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c70102f00 msgr2=0x7f2c7006c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.695+0000 7f2c7499b700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c70102f00 0x7f2c7006c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.695+0000 7f2c7499b700 1 -- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 msgr2=0x7f2c70071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.695+0000 7f2c7499b700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 0x7f2c70071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.695+0000 7f2c7499b700 1 -- 172.21.15.26:0/968249137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68009d00 con 0x7f2c70101d10 2023-07-13T22:23:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.696+0000 7f2c7499b700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c970 tx=0x7f2c6400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.697+0000 7f2c6dffb700 1 -- 172.21.15.26:0/968249137 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64004220 con 0x7f2c70101d10 2023-07-13T22:23:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.697+0000 7f2c6dffb700 1 -- 172.21.15.26:0/968249137 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c64007d60 con 0x7f2c70101d10 2023-07-13T22:23:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.697+0000 7f2c6dffb700 1 -- 172.21.15.26:0/968249137 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64003c90 con 0x7f2c70101d10 2023-07-13T22:23:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.699+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c70071ed0 con 0x7f2c70101d10 2023-07-13T22:23:13.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.699+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c70072360 con 0x7f2c70101d10 2023-07-13T22:23:13.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.711+0000 7f2c8a5aab80 1 Processor -- start 2023-07-13T22:23:13.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 -- start start 2023-07-13T22:23:13.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b504e20 0x555e0b5056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b5060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b409cb0 con 0x555e0b5044d0 2023-07-13T22:23:13.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b45ca40 con 0x555e0b505c70 2023-07-13T22:23:13.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.712+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b455ed0 con 0x555e0b504e20 2023-07-13T22:23:13.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50364/0 (socket says 172.21.15.26:50364) 2023-07-13T22:23:13.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 -- 172.21.15.26:0/3795523766 learned_addr learned my addr 172.21.15.26:0/3795523766 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:13.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 -- 172.21.15.26:0/3795523766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b504e20 msgr2=0x555e0b5056e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:13.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b504e20 0x555e0b5056e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 -- 172.21.15.26:0/3795523766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 msgr2=0x555e0b5060e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:13.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b5060e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.714+0000 7f2c7519c700 1 -- 172.21.15.26:0/3795523766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0b283de0 con 0x555e0b5044d0 2023-07-13T22:23:13.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.715+0000 7f2c7519c700 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c68008d70 tx=0x7f2c68008da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b4a95782e42684 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.717+0000 7f2c5affd700 1 -- 172.21.15.26:0/3795523766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800e040 con 0x555e0b5044d0 2023-07-13T22:23:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.717+0000 7f2c5affd700 1 -- 172.21.15.26:0/3795523766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c680093b0 con 0x555e0b5044d0 2023-07-13T22:23:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.717+0000 7f2c5affd700 1 -- 172.21.15.26:0/3795523766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801b790 con 0x555e0b5044d0 2023-07-13T22:23:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.718+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/3795523766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 msgr2=0x555e0b5048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.718+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2c68008d70 tx=0x7f2c68008da0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.719+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/3795523766 shutdown_connections 2023-07-13T22:23:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.719+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b5048e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.719+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b5060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.719+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/3795523766 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b504e20 0x555e0b5056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.719+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/3795523766 >> 172.21.15.26:0/3795523766 conn(0x555e0b4f9da0 msgr2=0x555e0b4ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.720+0000 7f2c6dffb700 1 -- 172.21.15.26:0/968249137 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c64004380 con 0x7f2c70101d10 2023-07-13T22:23:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.721+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/3795523766 shutdown_connections 2023-07-13T22:23:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.721+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/3795523766 wait complete. 2023-07-13T22:23:13.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.722+0000 7f2c8a5aab80 1 -- start start 2023-07-13T22:23:13.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.723+0000 7f2c8a5aab80 1 Processor -- start 2023-07-13T22:23:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.723+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b51f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.723+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b51f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.723+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b526c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.723+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b3f5ac0 con 0x555e0b5044d0 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b3f57c0 con 0x555e0b505c70 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b2d9920 con 0x555e0b5248a0 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c8a5aab80 10 client.0 _ll_register_callbacks cb 0x555e0b3f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b51f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b51f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59904/0 (socket says 172.21.15.26:59904) 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c7499b700 1 -- 172.21.15.26:0/2281687039 learned_addr learned my addr 172.21.15.26:0/2281687039 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.724+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b526c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.725+0000 7f2c6ffff700 1 -- 172.21.15.26:0/2281687039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 msgr2=0x555e0b51f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.725+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b51f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.725+0000 7f2c6ffff700 1 -- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 msgr2=0x555e0b51f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.725+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b51f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.725+0000 7f2c6ffff700 1 -- 172.21.15.26:0/2281687039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7010aa80 con 0x555e0b5248a0 2023-07-13T22:23:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.726+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b526c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c320 tx=0x7f2c5c00c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de90d862f49caa67 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.726+0000 7f2c6dffb700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 0x7f2c600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.726+0000 7f2c6dffb700 1 -- 172.21.15.26:0/968249137 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c640cc4f0 con 0x7f2c70101d10 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.727+0000 7f2c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.727+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00f650 con 0x555e0b5248a0 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.727+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c5c00f7b0 con 0x555e0b5248a0 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.727+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e0b3af4f0 con 0x555e0b5248a0 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.728+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c5c00f970 con 0x555e0b5248a0 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.728+0000 7f2c8a5aab80 10 client.26977 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.728+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e0b26cc60 con 0x555e0b5248a0 2023-07-13T22:23:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.729+0000 7f2c7519c700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 0x7f2c600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.729+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e0b3af4f0 con 0x555e0b5248a0 2023-07-13T22:23:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.729+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c5c016020 con 0x555e0b5248a0 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 10 client.26977 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 10 client.26977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 20 client.26977 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b52b470 0x555e0b52d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 4 client.26977 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.730+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e0b44eeb0 con 0x555e0b52b470 2023-07-13T22:23:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.731+0000 7f2c7499b700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b52b470 0x555e0b52d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.732+0000 7f2c7519c700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 0x7f2c600a35e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800d010 tx=0x7f2c68009960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.733+0000 7f2c7499b700 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b52b470 0x555e0b52d880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4783cbfb1d7bb782 server_cookie=3260e994278af7c1 in_seq=0 out_seq=0 2023-07-13T22:23:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.733+0000 7f2c4ffff700 10 client.26977 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.739+0000 7f2c4ffff700 1 -- 172.21.15.26:0/2281687039 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e0b44eeb0 con 0x555e0b52b470 2023-07-13T22:23:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.739+0000 7f2c4ffff700 10 client.26977 handle_command_reply: tid=0 2023-07-13T22:23:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.739+0000 7f2c8a5aab80 1 client.26977 shutdown 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 msgr2=0x555e0b526c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b526c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00c320 tx=0x7f2c5c00c630 comp rx=0 tx=0).stop 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 shutdown_connections 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b52b470 0x555e0b52d880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b51f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b526c80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/2281687039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b505c70 0x555e0b51f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 >> 172.21.15.26:0/2281687039 conn(0x555e0b4f9da0 msgr2=0x555e0b4fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c4ffff700 0 client.26977 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:13.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.740+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 shutdown_connections 2023-07-13T22:23:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.741+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/2281687039 wait complete. 2023-07-13T22:23:13.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.741+0000 7f2c8a5aab80 20 client.26977 trim_cache size 0 max 16384 2023-07-13T22:23:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 Processor -- start 2023-07-13T22:23:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 -- start start 2023-07-13T22:23:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 0x555e0b50c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b51de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b500c50 con 0x555e0b505c70 2023-07-13T22:23:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b50bf40 con 0x555e0b5044d0 2023-07-13T22:23:13.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.768+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b51ff70 con 0x555e0b5248a0 2023-07-13T22:23:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.769+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.769+0000 7f2c7499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59914/0 (socket says 172.21.15.26:59914) 2023-07-13T22:23:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.769+0000 7f2c7499b700 1 -- 172.21.15.26:0/1590460119 learned_addr learned my addr 172.21.15.26:0/1590460119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.769+0000 7f2c7519c700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b51de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 0x555e0b50c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 -- 172.21.15.26:0/1590460119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 msgr2=0x555e0b51de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b51de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 -- 172.21.15.26:0/1590460119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 msgr2=0x555e0b50c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 0x555e0b50c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 -- 172.21.15.26:0/1590460119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0b2aa250 con 0x555e0b5044d0 2023-07-13T22:23:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 0x555e0b50c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:13.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.770+0000 7f2c7499b700 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c64096380 tx=0x7f2c640cff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4378aa46439543 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.771+0000 7f2c5b7fe700 1 -- 172.21.15.26:0/1590460119 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640996e0 con 0x555e0b5044d0 2023-07-13T22:23:13.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.771+0000 7f2c5b7fe700 1 -- 172.21.15.26:0/1590460119 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c64099450 con 0x555e0b5044d0 2023-07-13T22:23:13.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.771+0000 7f2c5b7fe700 1 -- 172.21.15.26:0/1590460119 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640a1df0 con 0x555e0b5044d0 2023-07-13T22:23:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.772+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/1590460119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 msgr2=0x555e0b53b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.772+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c64096380 tx=0x7f2c640cff90 comp rx=0 tx=0).stop 2023-07-13T22:23:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/1590460119 shutdown_connections 2023-07-13T22:23:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b5248a0 0x555e0b51de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5044d0 0x555e0b53b100 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/1590460119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b505c70 0x555e0b50c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/1590460119 >> 172.21.15.26:0/1590460119 conn(0x555e0b4f9da0 msgr2=0x555e0b505480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/1590460119 shutdown_connections 2023-07-13T22:23:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.773+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/1590460119 wait complete. 2023-07-13T22:23:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.774+0000 7f2c8a5aab80 1 -- start start 2023-07-13T22:23:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.774+0000 7f2c8a5aab80 1 Processor -- start 2023-07-13T22:23:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b579c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5248a0 0x555e0b57e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b57ee90 con 0x555e0b5044d0 2023-07-13T22:23:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b57f000 con 0x555e0b5248a0 2023-07-13T22:23:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0b57f1c0 con 0x555e0b505c70 2023-07-13T22:23:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.775+0000 7f2c8a5aab80 10 client.0 _ll_register_callbacks cb 0x7f2c5c00a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.776+0000 7f2c6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.776+0000 7f2c6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50542/0 (socket says 172.21.15.26:50542) 2023-07-13T22:23:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.776+0000 7f2c6ffff700 1 -- 172.21.15.26:0/524364627 learned_addr learned my addr 172.21.15.26:0/524364627 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.777+0000 7f2c7499b700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b579c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.778+0000 7f2c6ffff700 1 -- 172.21.15.26:0/524364627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5248a0 msgr2=0x555e0b57e750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.778+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5248a0 0x555e0b57e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.778+0000 7f2c6ffff700 1 -- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 msgr2=0x555e0b579c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.778+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b579c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.778+0000 7f2c6ffff700 1 -- 172.21.15.26:0/524364627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c70072360 con 0x555e0b505c70 2023-07-13T22:23:13.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.779+0000 7f2c6ffff700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000c00 tx=0x7f2c5c019ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c9d3bc8bb0c88b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.785+0000 7f2c4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.785+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c028070 con 0x555e0b505c70 2023-07-13T22:23:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.785+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2c5c00a300 con 0x555e0b505c70 2023-07-13T22:23:13.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.785+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c680093b0 con 0x555e0b505c70 2023-07-13T22:23:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.796+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c5c022630 con 0x555e0b505c70 2023-07-13T22:23:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.796+0000 7f2c8a5aab80 10 client.26983 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:13.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.796+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e0b2822b0 con 0x555e0b505c70 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c680093b0 con 0x555e0b505c70 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2c5c016050 con 0x555e0b505c70 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 10 client.26983 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 10 client.26983 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 20 client.26983 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:13.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b580ae0 0x555e0b582ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 4 client.26983 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:13.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.800+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555e0b411050 con 0x555e0b580ae0 2023-07-13T22:23:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.806+0000 7f2c7499b700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b580ae0 0x555e0b582ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.838+0000 7f2c7499b700 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b580ae0 0x555e0b582ef0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=878a6a33025d2251 server_cookie=408c2e126419d002 in_seq=0 out_seq=0 2023-07-13T22:23:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.838+0000 7f2c4ffff700 10 client.26983 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.846+0000 7f2c4ffff700 1 -- 172.21.15.26:0/524364627 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555e0b411050 con 0x555e0b580ae0 2023-07-13T22:23:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.846+0000 7f2c4ffff700 10 client.26983 handle_command_reply: tid=0 2023-07-13T22:23:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.847+0000 7f2c8a5aab80 1 client.26983 shutdown 2023-07-13T22:23:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 msgr2=0x555e0b57c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000c00 tx=0x7f2c5c019ee0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 shutdown_connections 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x555e0b580ae0 0x555e0b582ef0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0b5044d0 0x555e0b579c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0b5248a0 0x555e0b57e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 --2- 172.21.15.26:0/524364627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0b505c70 0x555e0b57c1c0 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c000c00 tx=0x7f2c5c019ee0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.878+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 >> 172.21.15.26:0/524364627 conn(0x555e0b4f9da0 msgr2=0x555e0b4fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.879+0000 7f2c4ffff700 0 client.26983 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.886+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 shutdown_connections 2023-07-13T22:23:13.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.887+0000 7f2c8a5aab80 1 -- 172.21.15.26:0/524364627 wait complete. 2023-07-13T22:23:13.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.889+0000 7f2c8a5aab80 20 client.26983 trim_cache size 0 max 16384 2023-07-13T22:23:13.873 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.899+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 msgr2=0x7f2c600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.899+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 0x7f2c600a35e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800d010 tx=0x7f2c68009960 comp rx=0 tx=0).stop 2023-07-13T22:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.899+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 msgr2=0x7f2c7006bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:13.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.899+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400c970 tx=0x7f2c6400cd30 comp rx=0 tx=0).stop 2023-07-13T22:23:13.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 shutdown_connections 2023-07-13T22:23:13.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2c600a1130 0x7f2c600a35e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c70103890 0x7f2c70071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c70101d10 0x7f2c7006bdb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 --2- 172.21.15.26:0/968249137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c70102f00 0x7f2c7006c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:13.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 >> 172.21.15.26:0/968249137 conn(0x7f2c700fd350 msgr2=0x7f2c70078080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:13.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 shutdown_connections 2023-07-13T22:23:13.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:13.900+0000 7f2c76bff700 1 -- 172.21.15.26:0/968249137 wait complete. 2023-07-13T22:23:13.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:13 smithi117 ceph-mon[129074]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 265 MiB/s wr, 86 op/s 2023-07-13T22:23:13.940 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:13 smithi026 ceph-mon[95647]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 265 MiB/s wr, 86 op/s 2023-07-13T22:23:13.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:23:13.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:13 smithi121 ceph-mon[129187]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 265 MiB/s wr, 86 op/s 2023-07-13T22:23:14.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 -- 172.21.15.26:0/1755068978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 msgr2=0x7f2778103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 --2- 172.21.15.26:0/1755068978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f2778103ae0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f277000b0a0 tx=0x7f277000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 -- 172.21.15.26:0/1755068978 shutdown_connections 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 --2- 172.21.15.26:0/1755068978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f2778103ae0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 --2- 172.21.15.26:0/1755068978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f2778100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 --2- 172.21.15.26:0/1755068978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f2778103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 -- 172.21.15.26:0/1755068978 >> 172.21.15.26:0/1755068978 conn(0x7f27780fa000 msgr2=0x7f27780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.310+0000 7f277ddff700 1 -- 172.21.15.26:0/1755068978 shutdown_connections 2023-07-13T22:23:14.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.311+0000 7f277ddff700 1 -- 172.21.15.26:0/1755068978 wait complete. 2023-07-13T22:23:14.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.312+0000 7f277ddff700 1 Processor -- start 2023-07-13T22:23:14.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.312+0000 7f277ddff700 1 -- start start 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f277806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f277806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f27781ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277806f970 con 0x7f2778100cb0 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277806fae0 con 0x7f2778103670 2023-07-13T22:23:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.313+0000 7f277ddff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781ac590 con 0x7f27780fe350 2023-07-13T22:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f277806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f2777fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f27781ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f277806f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50408/0 (socket says 172.21.15.26:50408) 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f2776ffd700 1 -- 172.21.15.26:0/86209579 learned_addr learned my addr 172.21.15.26:0/86209579 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f277806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 -- 172.21.15.26:0/86209579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 msgr2=0x7f27781ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f27781ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 -- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 msgr2=0x7f277806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f277806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.314+0000 7f27777fe700 1 -- 172.21.15.26:0/86209579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009d00 con 0x7f27780fe350 2023-07-13T22:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f27777fe700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f277806ee60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f2774ff9700 1 -- 172.21.15.26:0/86209579 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800eee0 con 0x7f27780fe350 2023-07-13T22:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f2774ff9700 1 -- 172.21.15.26:0/86209579 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276800cbd0 con 0x7f27780fe350 2023-07-13T22:23:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27781ac840 con 0x7f27780fe350 2023-07-13T22:23:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f2774ff9700 1 -- 172.21.15.26:0/86209579 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768018680 con 0x7f27780fe350 2023-07-13T22:23:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.315+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27781acda0 con 0x7f27780fe350 2023-07-13T22:23:14.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.321+0000 7f2774ff9700 1 -- 172.21.15.26:0/86209579 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f276800cd40 con 0x7f27780fe350 2023-07-13T22:23:14.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 Processor -- start 2023-07-13T22:23:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 -- start start 2023-07-13T22:23:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x5600604338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x5600604346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060338cb0 con 0x5600604334d0 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006038ba40 con 0x560060433e20 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.323+0000 7f279174eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060384ed0 con 0x560060434c70 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.324+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.324+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50570/0 (socket says 172.21.15.26:50570) 2023-07-13T22:23:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.324+0000 7f2776ffd700 1 -- 172.21.15.26:0/105930328 learned_addr learned my addr 172.21.15.26:0/105930328 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.324+0000 7f27777fe700 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x5600604346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2776ffd700 1 -- 172.21.15.26:0/105930328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 msgr2=0x5600604346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2776ffd700 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x5600604346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2776ffd700 1 -- 172.21.15.26:0/105930328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 msgr2=0x5600604338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2776ffd700 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x5600604338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2776ffd700 1 -- 172.21.15.26:0/105930328 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600601b2de0 con 0x560060434c70 2023-07-13T22:23:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.325+0000 7f2774ff9700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 0x7f27600a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.326+0000 7f2774ff9700 1 -- 172.21.15.26:0/86209579 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2768014070 con 0x7f27780fe350 2023-07-13T22:23:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.327+0000 7f2776ffd700 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f276c009e50 tx=0x7f276c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3da6800007073d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.327+0000 7f2777fff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 0x7f27600a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.327+0000 7f275dffb700 1 -- 172.21.15.26:0/105930328 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c004a30 con 0x560060434c70 2023-07-13T22:23:14.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.327+0000 7f275dffb700 1 -- 172.21.15.26:0/105930328 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c0054e0 con 0x560060434c70 2023-07-13T22:23:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.327+0000 7f275dffb700 1 -- 172.21.15.26:0/105930328 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c019930 con 0x560060434c70 2023-07-13T22:23:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.328+0000 7f279174eb80 1 -- 172.21.15.26:0/105930328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 msgr2=0x5600604350e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.328+0000 7f279174eb80 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f276c009e50 tx=0x7f276c011720 comp rx=0 tx=0).stop 2023-07-13T22:23:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 -- 172.21.15.26:0/105930328 shutdown_connections 2023-07-13T22:23:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x5600604338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x5600604350e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 --2- 172.21.15.26:0/105930328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x5600604346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 -- 172.21.15.26:0/105930328 >> 172.21.15.26:0/105930328 conn(0x560060428da0 msgr2=0x56006042e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.338+0000 7f279174eb80 1 -- 172.21.15.26:0/105930328 shutdown_connections 2023-07-13T22:23:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.343+0000 7f279174eb80 1 -- 172.21.15.26:0/105930328 wait complete. 2023-07-13T22:23:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.344+0000 7f279174eb80 1 -- start start 2023-07-13T22:23:14.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.345+0000 7f279174eb80 1 Processor -- start 2023-07-13T22:23:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.345+0000 7f279174eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006043d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.345+0000 7f279174eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x56006043d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.345+0000 7f279174eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.346+0000 7f279174eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060324ac0 con 0x5600604334d0 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.346+0000 7f279174eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600603247c0 con 0x560060433e20 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.346+0000 7f279174eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060208920 con 0x560060434c70 2023-07-13T22:23:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.346+0000 7f279174eb80 10 client.0 _ll_register_callbacks cb 0x560060322760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.350+0000 7f2777fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.350+0000 7f2777fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50572/0 (socket says 172.21.15.26:50572) 2023-07-13T22:23:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.350+0000 7f2777fff700 1 -- 172.21.15.26:0/2758626822 learned_addr learned my addr 172.21.15.26:0/2758626822 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 -- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 msgr2=0x56006043d700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x56006043d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 -- 172.21.15.26:0/2758626822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 msgr2=0x56006043d1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006043d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 -- 172.21.15.26:0/2758626822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c009a80 con 0x560060434c70 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2770015000 tx=0x7f277000b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f25c46df47048d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.351+0000 7f2777fff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 0x7f27600a3700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f27700062a0 tx=0x7f27700061f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.352+0000 7f2756ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:14.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.352+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000be90 con 0x560060434c70 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.352+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2770005250 con 0x560060434c70 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.352+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600602de4f0 con 0x560060434c70 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.353+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2770009b80 con 0x560060434c70 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.353+0000 7f279174eb80 10 client.26995 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:14.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.353+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56006019bc60 con 0x560060434c70 2023-07-13T22:23:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.354+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600602de4f0 con 0x560060434c70 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.354+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f277001ba40 con 0x560060434c70 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 10 client.26995 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 10 client.26995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 20 client.26995 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560060459f90 0x56006045c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 4 client.26995 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.355+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56006037deb0 con 0x560060459f90 2023-07-13T22:23:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.364+0000 7f27777fe700 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560060459f90 0x56006045c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.393+0000 7f27777fe700 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560060459f90 0x56006045c440 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0b9e93f181b5b44 server_cookie=3a824748942373c5 in_seq=0 out_seq=0 2023-07-13T22:23:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.394+0000 7f2756ffd700 10 client.26995 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.408+0000 7f2756ffd700 1 -- 172.21.15.26:0/2758626822 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56006037deb0 con 0x560060459f90 2023-07-13T22:23:14.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.408+0000 7f2756ffd700 10 client.26995 handle_command_reply: tid=0 2023-07-13T22:23:14.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.408+0000 7f279174eb80 1 client.26995 shutdown 2023-07-13T22:23:14.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.438+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 msgr2=0x560060444740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.438+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2770015000 tx=0x7f277000b3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 shutdown_connections 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006043d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560060459f90 0x56006045c440 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060444740 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 --2- 172.21.15.26:0/2758626822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x56006043d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.441+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 >> 172.21.15.26:0/2758626822 conn(0x560060428da0 msgr2=0x56006042d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.450+0000 7f2756ffd700 0 client.26995 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:14.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.452+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 shutdown_connections 2023-07-13T22:23:14.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.457+0000 7f279174eb80 1 -- 172.21.15.26:0/2758626822 wait complete. 2023-07-13T22:23:14.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.457+0000 7f279174eb80 20 client.26995 trim_cache size 0 max 16384 2023-07-13T22:23:14.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.486+0000 7f279174eb80 1 Processor -- start 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.486+0000 7f279174eb80 1 -- start start 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006045bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060437bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006043cd90 con 0x5600604334d0 2023-07-13T22:23:14.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600604341f0 con 0x560060433e20 2023-07-13T22:23:14.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f279174eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006042f0d0 con 0x560060434c70 2023-07-13T22:23:14.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.487+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59954/0 (socket says 172.21.15.26:59954) 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 -- 172.21.15.26:0/304856637 learned_addr learned my addr 172.21.15.26:0/304856637 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2777fff700 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060437bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 -- 172.21.15.26:0/304856637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 msgr2=0x560060437bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060437bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 -- 172.21.15.26:0/304856637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 msgr2=0x56006045bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006045bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 -- 172.21.15.26:0/304856637 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56006042a5e0 con 0x560060433e20 2023-07-13T22:23:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.488+0000 7f2776ffd700 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c009d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6b443eb0ee02ee server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.489+0000 7f275e7fc700 1 -- 172.21.15.26:0/304856637 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0052c0 con 0x560060433e20 2023-07-13T22:23:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.489+0000 7f275e7fc700 1 -- 172.21.15.26:0/304856637 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c0056d0 con 0x560060433e20 2023-07-13T22:23:14.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.489+0000 7f275e7fc700 1 -- 172.21.15.26:0/304856637 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c022ab0 con 0x560060433e20 2023-07-13T22:23:14.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 -- 172.21.15.26:0/304856637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 msgr2=0x560060444700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c009d50 comp rx=0 tx=0).stop 2023-07-13T22:23:14.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 -- 172.21.15.26:0/304856637 shutdown_connections 2023-07-13T22:23:14.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x56006045bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060434c70 0x560060437bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 --2- 172.21.15.26:0/304856637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060433e20 0x560060444700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.490+0000 7f279174eb80 1 -- 172.21.15.26:0/304856637 >> 172.21.15.26:0/304856637 conn(0x560060428da0 msgr2=0x56006043b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.491+0000 7f279174eb80 1 -- 172.21.15.26:0/304856637 shutdown_connections 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.491+0000 7f279174eb80 1 -- 172.21.15.26:0/304856637 wait complete. 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.491+0000 7f279174eb80 1 -- start start 2023-07-13T22:23:14.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 Processor -- start 2023-07-13T22:23:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x560060440360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060434c70 0x5600604438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060443e00 con 0x5600604334d0 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060443f70 con 0x560060434c70 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.492+0000 7f279174eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560060444130 con 0x560060433e20 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f279174eb80 10 client.0 _ll_register_callbacks cb 0x7f27700077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50578/0 (socket says 172.21.15.26:50578) 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 -- 172.21.15.26:0/4058845794 learned_addr learned my addr 172.21.15.26:0/4058845794 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 -- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060434c70 msgr2=0x5600604438c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060434c70 0x5600604438c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 -- 172.21.15.26:0/4058845794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 msgr2=0x560060440360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x560060440360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.493+0000 7f2776ffd700 1 -- 172.21.15.26:0/4058845794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600601b2de0 con 0x560060433e20 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.494+0000 7f2776ffd700 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f276c023e60 tx=0x7f276c023e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e50ab154adefcf server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:14.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.494+0000 7f2756ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:14.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.494+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c003880 con 0x560060433e20 2023-07-13T22:23:14.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.494+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f276c005300 con 0x560060433e20 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.494+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f276c0054e0 con 0x560060433e20 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f276c0039e0 con 0x560060433e20 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f279174eb80 10 client.27001 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5600601d9250 con 0x560060433e20 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276c0054e0 con 0x560060433e20 2023-07-13T22:23:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f276c02bc10 con 0x560060433e20 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f279174eb80 10 client.27001 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f279174eb80 10 client.27001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.495+0000 7f279174eb80 20 client.27001 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.496+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5600604ae840 0x5600604b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.496+0000 7f279174eb80 4 client.27001 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.496+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560060340050 con 0x5600604ae840 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.496+0000 7f27777fe700 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5600604ae840 0x5600604b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:14.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.496+0000 7f27777fe700 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5600604ae840 0x5600604b0c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea6cf8c7870bb75 server_cookie=5213fa0672b8f42b in_seq=0 out_seq=0 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f2756ffd700 10 client.27001 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f2756ffd700 1 -- 172.21.15.26:0/4058845794 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560060340050 con 0x5600604ae840 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f2756ffd700 10 client.27001 handle_command_reply: tid=0 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f279174eb80 1 client.27001 shutdown 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 msgr2=0x5600604408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.497+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f276c023e60 tx=0x7f276c023e90 comp rx=0 tx=0).stop 2023-07-13T22:23:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 shutdown_connections 2023-07-13T22:23:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5600604334d0 0x560060440360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5600604ae840 0x5600604b0c50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560060434c70 0x5600604438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 --2- 172.21.15.26:0/4058845794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560060433e20 0x5600604408a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f2756ffd700 0 client.27001 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 >> 172.21.15.26:0/4058845794 conn(0x560060428da0 msgr2=0x56006045bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 shutdown_connections 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.498+0000 7f279174eb80 1 -- 172.21.15.26:0/4058845794 wait complete. 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.500+0000 7f279174eb80 20 client.27001 trim_cache size 0 max 16384 2023-07-13T22:23:14.485 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.503+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 msgr2=0x7f27600a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.503+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 0x7f27600a3700 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f27700062a0 tx=0x7f27700061f0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.503+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 msgr2=0x7f277806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.503+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f277806ee60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9f0 tx=0x7f276800ed00 comp rx=0 tx=0).stop 2023-07-13T22:23:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 shutdown_connections 2023-07-13T22:23:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2778103670 0x7f27781ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27780fe350 0x7f277806ee60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f27600a1250 0x7f27600a3700 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 --2- 172.21.15.26:0/86209579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2778100cb0 0x7f277806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.504+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 >> 172.21.15.26:0/86209579 conn(0x7f27780fa000 msgr2=0x7f27780feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.505+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 shutdown_connections 2023-07-13T22:23:14.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:14.505+0000 7f277ddff700 1 -- 172.21.15.26:0/86209579 wait complete. 2023-07-13T22:23:15.528 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:23:15.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:23:15.871 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:15 smithi026 ceph-mon[95647]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 98 op/s 2023-07-13T22:23:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:15 smithi117 ceph-mon[129074]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 98 op/s 2023-07-13T22:23:15.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:15 smithi121 ceph-mon[129187]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 306 MiB/s wr, 98 op/s 2023-07-13T22:23:16.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 -- 172.21.15.26:0/1082551471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 msgr2=0x7fd9900719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 --2- 172.21.15.26:0/1082551471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 0x7fd9900719d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd984009990 tx=0x7fd984009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 -- 172.21.15.26:0/1082551471 shutdown_connections 2023-07-13T22:23:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 --2- 172.21.15.26:0/1082551471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010d080 0x7fd99010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 --2- 172.21.15.26:0/1082551471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 0x7fd9900719d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 --2- 172.21.15.26:0/1082551471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99010a760 0x7fd99010cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 -- 172.21.15.26:0/1082551471 >> 172.21.15.26:0/1082551471 conn(0x7fd99006cd80 msgr2=0x7fd99006f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.283+0000 7fd9965ff700 1 -- 172.21.15.26:0/1082551471 shutdown_connections 2023-07-13T22:23:16.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.284+0000 7fd9965ff700 1 -- 172.21.15.26:0/1082551471 wait complete. 2023-07-13T22:23:16.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.285+0000 7fd9965ff700 1 Processor -- start 2023-07-13T22:23:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 -- start start 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 0x7fd99019bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99010d080 0x7fd9901b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99019f520 con 0x7fd99010d080 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99019f690 con 0x7fd99010a760 2023-07-13T22:23:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.287+0000 7fd9965ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901b4e70 con 0x7fd9900715c0 2023-07-13T22:23:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:59986/0 (socket says 172.21.15.26:59986) 2023-07-13T22:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 -- 172.21.15.26:0/3229880344 learned_addr learned my addr 172.21.15.26:0/3229880344 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 -- 172.21.15.26:0/3229880344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 msgr2=0x7fd99019bf00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 0x7fd99019bf00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 -- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99010d080 msgr2=0x7fd9901b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99010d080 0x7fd9901b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.288+0000 7fd98f7fe700 1 -- 172.21.15.26:0/3229880344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9840095f0 con 0x7fd99010a760 2023-07-13T22:23:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.289+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b9c0 tx=0x7fd98800bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.289+0000 7fd98d7fa700 1 -- 172.21.15.26:0/3229880344 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98800c9d0 con 0x7fd99010a760 2023-07-13T22:23:16.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.292+0000 7fd98d7fa700 1 -- 172.21.15.26:0/3229880344 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd988014440 con 0x7fd99010a760 2023-07-13T22:23:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.292+0000 7fd98d7fa700 1 -- 172.21.15.26:0/3229880344 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880125a0 con 0x7fd99010a760 2023-07-13T22:23:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.293+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9901b5070 con 0x7fd99010a760 2023-07-13T22:23:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.293+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9901b55b0 con 0x7fd99010a760 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.307+0000 7fd98d7fa700 1 -- 172.21.15.26:0/3229880344 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd988012700 con 0x7fd99010a760 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.309+0000 7fd9a9f03b80 1 Processor -- start 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.309+0000 7fd9a9f03b80 1 -- start start 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df263960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df265160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df168d30 con 0x5592df263ea0 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df1bbac0 con 0x5592df264cf0 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.310+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df1b4f50 con 0x5592df263550 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.311+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.311+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50452/0 (socket says 172.21.15.26:50452) 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.311+0000 7fd98ffff700 1 -- 172.21.15.26:0/399836779 learned_addr learned my addr 172.21.15.26:0/399836779 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.312+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df265160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.312+0000 7fd98ffff700 1 -- 172.21.15.26:0/399836779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 msgr2=0x5592df263960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.311+0000 7fd994b9c700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df263960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.316+0000 7fd98ffff700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df263960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.316+0000 7fd98ffff700 1 -- 172.21.15.26:0/399836779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 msgr2=0x5592df265160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.316+0000 7fd98ffff700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df265160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.316+0000 7fd98ffff700 1 -- 172.21.15.26:0/399836779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592defe2e60 con 0x5592df263ea0 2023-07-13T22:23:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.317+0000 7fd98d7fa700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 0x7fd9780a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.317+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df265160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.317+0000 7fd994b9c700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df263960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.318+0000 7fd98ffff700 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd984005950 tx=0x7fd984005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baede71dc60f097 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.319+0000 7fd9767fc700 1 -- 172.21.15.26:0/399836779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98403f070 con 0x5592df263ea0 2023-07-13T22:23:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.319+0000 7fd9767fc700 1 -- 172.21.15.26:0/399836779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98400be00 con 0x5592df263ea0 2023-07-13T22:23:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.319+0000 7fd9767fc700 1 -- 172.21.15.26:0/399836779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984005430 con 0x5592df263ea0 2023-07-13T22:23:16.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.320+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/399836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 msgr2=0x5592df264760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.320+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd984005950 tx=0x7fd984005dc0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.317+0000 7fd98d7fa700 1 -- 172.21.15.26:0/3229880344 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd988006e40 con 0x7fd99010a760 2023-07-13T22:23:16.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd994b9c700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 0x7fd9780a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/399836779 shutdown_connections 2023-07-13T22:23:16.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df265160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df263960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/399836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df264760 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.326+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/399836779 >> 172.21.15.26:0/399836779 conn(0x5592df258e20 msgr2=0x5592df25e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.327+0000 7fd994b9c700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 0x7fd9780a3730 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd980005bf0 tx=0x7fd980005b00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.332+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/399836779 shutdown_connections 2023-07-13T22:23:16.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.332+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/399836779 wait complete. 2023-07-13T22:23:16.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.333+0000 7fd9a9f03b80 1 -- start start 2023-07-13T22:23:16.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.333+0000 7fd9a9f03b80 1 Processor -- start 2023-07-13T22:23:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df26d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df274780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df154b40 con 0x5592df263ea0 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df154840 con 0x5592df264cf0 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df0389a0 con 0x5592df263550 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.334+0000 7fd9a9f03b80 10 client.0 _ll_register_callbacks cb 0x5592df1527e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.335+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.335+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50608/0 (socket says 172.21.15.26:50608) 2023-07-13T22:23:16.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.335+0000 7fd98ffff700 1 -- 172.21.15.26:0/904673435 learned_addr learned my addr 172.21.15.26:0/904673435 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.338+0000 7fd98ffff700 1 -- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 msgr2=0x5592df274780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.338+0000 7fd98ffff700 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df274780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.338+0000 7fd98ffff700 1 -- 172.21.15.26:0/904673435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 msgr2=0x5592df26d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.338+0000 7fd98ffff700 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df26d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.338+0000 7fd98ffff700 1 -- 172.21.15.26:0/904673435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99010ff20 con 0x5592df263550 2023-07-13T22:23:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.339+0000 7fd98ffff700 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd984009fd0 tx=0x7fd98400bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ada14dbbbe552c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.339+0000 7fd96f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:16.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.340+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98403f070 con 0x5592df263550 2023-07-13T22:23:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.340+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd984005540 con 0x5592df263550 2023-07-13T22:23:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.340+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592df10e570 con 0x5592df263550 2023-07-13T22:23:16.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.341+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd984005700 con 0x5592df263550 2023-07-13T22:23:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.341+0000 7fd9a9f03b80 10 client.27007 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.341+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5592defcbce0 con 0x5592df263550 2023-07-13T22:23:16.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.341+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd984051030 con 0x5592df263550 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd98404b060 con 0x5592df263550 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 10 client.27007 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 10 client.27007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 20 client.27007 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df278a20 0x5592df27ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 4 client.27007 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592df1adf30 con 0x5592df278a20 2023-07-13T22:23:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.344+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df278a20 0x5592df27ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.345+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df278a20 0x5592df27ae30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca423e35dcfefd91 server_cookie=a904ea1b5f3a7903 in_seq=0 out_seq=0 2023-07-13T22:23:16.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.345+0000 7fd96f7fe700 10 client.27007 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.345+0000 7fd96f7fe700 1 -- 172.21.15.26:0/904673435 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592df1adf30 con 0x5592df278a20 2023-07-13T22:23:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.345+0000 7fd96f7fe700 10 client.27007 handle_command_reply: tid=0 2023-07-13T22:23:16.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.345+0000 7fd9a9f03b80 1 client.27007 shutdown 2023-07-13T22:23:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 msgr2=0x5592df26d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd984009fd0 tx=0x7fd98400bab0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 shutdown_connections 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df278a20 0x5592df27ae30 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df274780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df26d200 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/904673435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df26d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 >> 172.21.15.26:0/904673435 conn(0x5592df258e20 msgr2=0x5592df25d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.347+0000 7fd96f7fe700 0 client.27007 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.348+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 shutdown_connections 2023-07-13T22:23:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.348+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/904673435 wait complete. 2023-07-13T22:23:16.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.348+0000 7fd9a9f03b80 20 client.27007 trim_cache size 0 max 16384 2023-07-13T22:23:16.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.385+0000 7fd9a9f03b80 1 Processor -- start 2023-07-13T22:23:16.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.385+0000 7fd9a9f03b80 1 -- start start 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df263ea0 0x5592df2680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 0x5592df2746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df25fce0 con 0x5592df263550 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df2589f0 con 0x5592df263ea0 2023-07-13T22:23:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df25fa80 con 0x5592df264cf0 2023-07-13T22:23:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd98ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50472/0 (socket says 172.21.15.26:50472) 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.386+0000 7fd98ffff700 1 -- 172.21.15.26:0/1767919799 learned_addr learned my addr 172.21.15.26:0/1767919799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd994b9c700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 0x5592df2746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df263ea0 0x5592df2680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98ffff700 1 -- 172.21.15.26:0/1767919799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 msgr2=0x5592df2746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 0x5592df2746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98ffff700 1 -- 172.21.15.26:0/1767919799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df263ea0 msgr2=0x5592df2680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df263ea0 0x5592df2680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd98ffff700 1 -- 172.21.15.26:0/1767919799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592df25a6a0 con 0x5592df263550 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.387+0000 7fd994b9c700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 0x5592df2746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.388+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb61bb1fcd0496b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.388+0000 7fd976ffd700 1 -- 172.21.15.26:0/1767919799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98403f070 con 0x5592df263550 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.388+0000 7fd976ffd700 1 -- 172.21.15.26:0/1767919799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd984051040 con 0x5592df263550 2023-07-13T22:23:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.388+0000 7fd976ffd700 1 -- 172.21.15.26:0/1767919799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984056850 con 0x5592df263550 2023-07-13T22:23:16.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1767919799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 msgr2=0x5592df267bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984005080 comp rx=0 tx=0).stop 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1767919799 shutdown_connections 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263550 0x5592df267bb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df264cf0 0x5592df2746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1767919799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df263ea0 0x5592df2680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.389+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1767919799 >> 172.21.15.26:0/1767919799 conn(0x5592df258e20 msgr2=0x5592df26b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.390+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1767919799 shutdown_connections 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.390+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1767919799 wait complete. 2023-07-13T22:23:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.390+0000 7fd9a9f03b80 1 -- start start 2023-07-13T22:23:16.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 Processor -- start 2023-07-13T22:23:16.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df2745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df271060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df2715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df271b30 con 0x5592df263ea0 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df271ca0 con 0x5592df264cf0 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592df2cc920 con 0x5592df263550 2023-07-13T22:23:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.391+0000 7fd9a9f03b80 10 client.0 _ll_register_callbacks cb 0x7fd984009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.392+0000 7fd994b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df2715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.392+0000 7fd994b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df2715a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60020/0 (socket says 172.21.15.26:60020) 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.392+0000 7fd994b9c700 1 -- 172.21.15.26:0/1680019763 learned_addr learned my addr 172.21.15.26:0/1680019763 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.392+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df271060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df2745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98f7fe700 1 -- 172.21.15.26:0/1680019763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 msgr2=0x5592df2745c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df2745c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98f7fe700 1 -- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 msgr2=0x5592df2715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df2715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.393+0000 7fd98f7fe700 1 -- 172.21.15.26:0/1680019763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592defcbce0 con 0x5592df263ea0 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd98f7fe700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df271060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9880cab30 tx=0x7fd988004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb0b55ecc1bcfb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd96f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98809e4a0 con 0x5592df263ea0 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd9880a28c0 con 0x5592df263ea0 2023-07-13T22:23:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd98400be00 con 0x5592df263ea0 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd988098810 con 0x5592df263ea0 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd9a9f03b80 10 client.16479 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.394+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5592df0092d0 con 0x5592df263ea0 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd98400be00 con 0x5592df263ea0 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd9880a2430 con 0x5592df263ea0 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 20 client.16479 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df2ce3b0 0x5592df2d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 4 client.16479 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.395+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5592df1700d0 con 0x5592df2ce3b0 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.396+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df2ce3b0 0x5592df2d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.396+0000 7fd98ffff700 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df2ce3b0 0x5592df2d07c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1a8ea60cade417 server_cookie=dfef2b0b9e2d0ba9 in_seq=0 out_seq=0 2023-07-13T22:23:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.396+0000 7fd96f7fe700 10 client.16479 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.397+0000 7fd96f7fe700 1 -- 172.21.15.26:0/1680019763 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592df1700d0 con 0x5592df2ce3b0 2023-07-13T22:23:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd96f7fe700 10 client.16479 handle_command_reply: tid=0 2023-07-13T22:23:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 client.16479 shutdown 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 msgr2=0x5592df271060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df271060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9880cab30 tx=0x7fd988004be0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 shutdown_connections 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5592df2ce3b0 0x5592df2d07c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5592df264cf0 0x5592df2715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592df263550 0x5592df2745c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 --2- 172.21.15.26:0/1680019763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592df263ea0 0x5592df271060 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd9880cab30 tx=0x7fd988004be0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 >> 172.21.15.26:0/1680019763 conn(0x5592df258e20 msgr2=0x5592df267bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.398+0000 7fd96f7fe700 0 client.16479 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.399+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 shutdown_connections 2023-07-13T22:23:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.399+0000 7fd9a9f03b80 1 -- 172.21.15.26:0/1680019763 wait complete. 2023-07-13T22:23:16.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.413+0000 7fd9a9f03b80 20 client.16479 trim_cache size 0 max 16384 2023-07-13T22:23:16.397 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:16.397 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:23:16.397 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "a1d75256-e675-4d04-98e1-f3f265dea29c", 2023-07-13T22:23:16.397 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:23:16.397 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:16.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 msgr2=0x7fd9780a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 0x7fd9780a3730 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd980005bf0 tx=0x7fd980005b00 comp rx=0 tx=0).stop 2023-07-13T22:23:16.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 msgr2=0x7fd99019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:16.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd98800b9c0 tx=0x7fd98800bd80 comp rx=0 tx=0).stop 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 shutdown_connections 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd9780a1280 0x7fd9780a3730 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd99010d080 0x7fd9901b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9900715c0 0x7fd99019bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 --2- 172.21.15.26:0/3229880344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99010a760 0x7fd99019c440 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.414+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 >> 172.21.15.26:0/3229880344 conn(0x7fd99006cd80 msgr2=0x7fd99010b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.419+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 shutdown_connections 2023-07-13T22:23:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:16.419+0000 7fd9965ff700 1 -- 172.21.15.26:0/3229880344 wait complete. 2023-07-13T22:23:16.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:23:17.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.192+0000 7fdeb9fff700 1 -- 172.21.15.26:0/3251987360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4070ee0 msgr2=0x7fdeb40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.192+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/3251987360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4070ee0 0x7fdeb40712f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009990 tx=0x7fdeac009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.194+0000 7fdeb9fff700 1 -- 172.21.15.26:0/3251987360 shutdown_connections 2023-07-13T22:23:17.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.194+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/3251987360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.194+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/3251987360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4070ee0 0x7fdeb40712f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.194+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/3251987360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4071830 0x7fdeb410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.194+0000 7fdeb9fff700 1 -- 172.21.15.26:0/3251987360 >> 172.21.15.26:0/3251987360 conn(0x7fdeb406ca90 msgr2=0x7fdeb406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.197+0000 7fdeb9fff700 1 -- 172.21.15.26:0/3251987360 shutdown_connections 2023-07-13T22:23:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.197+0000 7fdeb9fff700 1 -- 172.21.15.26:0/3251987360 wait complete. 2023-07-13T22:23:17.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.201+0000 7fdeb9fff700 1 Processor -- start 2023-07-13T22:23:17.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.204+0000 7fdeb9fff700 1 -- start start 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4070ee0 0x7fdeb4113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 0x7fdeb4113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb41b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4117030 con 0x7fdeb410bd60 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb41171a0 con 0x7fdeb4071830 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.206+0000 7fdeb9fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb41b4e50 con 0x7fdeb4070ee0 2023-07-13T22:23:17.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.208+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb41b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.208+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb41b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50488/0 (socket says 172.21.15.26:50488) 2023-07-13T22:23:17.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.208+0000 7fdeb3fff700 1 -- 172.21.15.26:0/1369532487 learned_addr learned my addr 172.21.15.26:0/1369532487 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.211+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 0x7fdeb4113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.211+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4070ee0 0x7fdeb4113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.222+0000 7fdeb2ffd700 1 -- 172.21.15.26:0/1369532487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4070ee0 msgr2=0x7fdeb4113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.222+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4070ee0 0x7fdeb4113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.222+0000 7fdeb2ffd700 1 -- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 msgr2=0x7fdeb41b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.222+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb41b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.222+0000 7fdeb2ffd700 1 -- 172.21.15.26:0/1369532487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0095f0 con 0x7fdeb4071830 2023-07-13T22:23:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.223+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 0x7fdeb4113f50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9c0 tx=0x7fdea800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.226+0000 7fdeb0ff9700 1 -- 172.21.15.26:0/1369532487 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8004220 con 0x7fdeb4071830 2023-07-13T22:23:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.226+0000 7fdeb0ff9700 1 -- 172.21.15.26:0/1369532487 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea8004380 con 0x7fdeb4071830 2023-07-13T22:23:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.226+0000 7fdeb0ff9700 1 -- 172.21.15.26:0/1369532487 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea8003b40 con 0x7fdeb4071830 2023-07-13T22:23:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.226+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb41b4ff0 con 0x7fdeb4071830 2023-07-13T22:23:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.226+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb41b54a0 con 0x7fdeb4071830 2023-07-13T22:23:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.234+0000 7fdecd982b80 1 Processor -- start 2023-07-13T22:23:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.245+0000 7fdecd982b80 1 -- start start 2023-07-13T22:23:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f44042cb0 con 0x564f4413de20 2023-07-13T22:23:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f44095a40 con 0x564f4413d4d0 2023-07-13T22:23:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdecd982b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4408eed0 con 0x564f4413ec70 2023-07-13T22:23:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.246+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.247+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60042/0 (socket says 172.21.15.26:60042) 2023-07-13T22:23:17.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.247+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3584497939 learned_addr learned my addr 172.21.15.26:0/3584497939 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:17.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.251+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3584497939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 msgr2=0x564f4413f0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:17.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.251+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4413f0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.251+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3584497939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 msgr2=0x564f4413e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.251+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.251+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3584497939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f43ebcde0 con 0x564f4413d4d0 2023-07-13T22:23:17.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.260+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e1c0 tx=0x7fdea4009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92676a013d25caa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.263+0000 7fdeb0ff9700 1 -- 172.21.15.26:0/1369532487 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdea8003ca0 con 0x7fdeb4071830 2023-07-13T22:23:17.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.266+0000 7fdeb0ff9700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 0x7fde9c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.266+0000 7fdeb0ff9700 1 -- 172.21.15.26:0/1369532487 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdea8014070 con 0x7fdeb4071830 2023-07-13T22:23:17.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.271+0000 7fde99ffb700 1 -- 172.21.15.26:0/3584497939 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4014480 con 0x564f4413d4d0 2023-07-13T22:23:17.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.271+0000 7fde99ffb700 1 -- 172.21.15.26:0/3584497939 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea400f040 con 0x564f4413d4d0 2023-07-13T22:23:17.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.271+0000 7fde99ffb700 1 -- 172.21.15.26:0/3584497939 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4013940 con 0x564f4413d4d0 2023-07-13T22:23:17.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.279+0000 7fdecd982b80 1 -- 172.21.15.26:0/3584497939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 msgr2=0x564f4413d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.279+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdea400e1c0 tx=0x7fdea4009710 comp rx=0 tx=0).stop 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.284+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 0x7fde9c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.291+0000 7fdecd982b80 1 -- 172.21.15.26:0/3584497939 shutdown_connections 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.291+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.291+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4413d8e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.291+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3584497939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4413e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.291+0000 7fdecd982b80 1 -- 172.21.15.26:0/3584497939 >> 172.21.15.26:0/3584497939 conn(0x564f44132da0 msgr2=0x564f441381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.339+0000 7fdecd982b80 1 -- 172.21.15.26:0/3584497939 shutdown_connections 2023-07-13T22:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.339+0000 7fdecd982b80 1 -- 172.21.15.26:0/3584497939 wait complete. 2023-07-13T22:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.340+0000 7fdecd982b80 1 -- start start 2023-07-13T22:23:17.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.340+0000 7fdecd982b80 1 Processor -- start 2023-07-13T22:23:17.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 0x564f441471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 0x564f44147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413ec70 0x564f4414e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4402eac0 con 0x564f4413ec70 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4402e7c0 con 0x564f4413de20 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f43f12920 con 0x564f4413d4d0 2023-07-13T22:23:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.341+0000 7fdecd982b80 10 client.0 _ll_register_callbacks cb 0x564f4402c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.343+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 0x564f44147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.343+0000 7fdeb2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 0x564f44147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60046/0 (socket says 172.21.15.26:60046) 2023-07-13T22:23:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.343+0000 7fdeb2ffd700 1 -- 172.21.15.26:0/82170660 learned_addr learned my addr 172.21.15.26:0/82170660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.343+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413ec70 0x564f4414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.353+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 0x564f441471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.354+0000 7fdeb37fe700 1 -- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 msgr2=0x564f44147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.354+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 0x564f44147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.354+0000 7fdeb37fe700 1 -- 172.21.15.26:0/82170660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413ec70 msgr2=0x564f4414e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.354+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413ec70 0x564f4414e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.354+0000 7fdeb37fe700 1 -- 172.21.15.26:0/82170660 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea4009d20 con 0x564f4413d4d0 2023-07-13T22:23:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.359+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 0x564f441471f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdeac006010 tx=0x7fdeac0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cb2996bd0234e5e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.359+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 0x7fde9c0a3800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b2e0 tx=0x7fdeac009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.359+0000 7fde92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.374+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00f660 con 0x564f4413d4d0 2023-07-13T22:23:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.374+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdeac00f7c0 con 0x564f4413d4d0 2023-07-13T22:23:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.374+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f43fe84f0 con 0x564f4413d4d0 2023-07-13T22:23:17.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.382+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac00f980 con 0x564f4413d4d0 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.386+0000 7fdecd982b80 10 client.27022 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.386+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f43ea5c60 con 0x564f4413d4d0 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.387+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f43fe84f0 con 0x564f4413d4d0 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.387+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdeac036070 con 0x564f4413d4d0 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:17.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 20 client.27022 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f44152f70 0x564f44155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 4 client.27022 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.388+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f44087eb0 con 0x564f44152f70 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.389+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f44152f70 0x564f44155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.392+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f44152f70 0x564f44155380 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8261451d767e644f server_cookie=d506a404f9dd6372 in_seq=0 out_seq=0 2023-07-13T22:23:17.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.392+0000 7fde92ffd700 10 client.27022 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.394+0000 7fde92ffd700 1 -- 172.21.15.26:0/82170660 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f44087eb0 con 0x564f44152f70 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.394+0000 7fde92ffd700 10 client.27022 handle_command_reply: tid=0 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.394+0000 7fdecd982b80 1 client.27022 shutdown 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 msgr2=0x564f441471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 0x564f441471f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdeac006010 tx=0x7fdeac0047e0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 shutdown_connections 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413ec70 0x564f4414e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f44152f70 0x564f44155380 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413d4d0 0x564f441471f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 --2- 172.21.15.26:0/82170660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413de20 0x564f44147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 >> 172.21.15.26:0/82170660 conn(0x564f44132da0 msgr2=0x564f441378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.395+0000 7fde92ffd700 0 client.27022 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.396+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 shutdown_connections 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.396+0000 7fdecd982b80 1 -- 172.21.15.26:0/82170660 wait complete. 2023-07-13T22:23:17.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.396+0000 7fdecd982b80 20 client.27022 trim_cache size 0 max 16384 2023-07-13T22:23:17.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.431+0000 7fdecd982b80 1 Processor -- start 2023-07-13T22:23:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.431+0000 7fdecd982b80 1 -- start start 2023-07-13T22:23:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f441450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f44148220 con 0x564f4413de20 2023-07-13T22:23:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f44139d00 con 0x564f4413d4d0 2023-07-13T22:23:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.432+0000 7fdecd982b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f44132970 con 0x564f4413ec70 2023-07-13T22:23:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.433+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.433+0000 7fdeb3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50656/0 (socket says 172.21.15.26:50656) 2023-07-13T22:23:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.433+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3201139203 learned_addr learned my addr 172.21.15.26:0/3201139203 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.433+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f441450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.434+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3201139203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 msgr2=0x564f441450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.434+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f441450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.434+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3201139203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 msgr2=0x564f4414e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.434+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.434+0000 7fdeb3fff700 1 -- 172.21.15.26:0/3201139203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f43ee3250 con 0x564f4413ec70 2023-07-13T22:23:17.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.435+0000 7fdeb3fff700 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000f80 tx=0x7fdea4013400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93f8c28511aac8a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.445+0000 7fde9a7fc700 1 -- 172.21.15.26:0/3201139203 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4022070 con 0x564f4413ec70 2023-07-13T22:23:17.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.445+0000 7fde9a7fc700 1 -- 172.21.15.26:0/3201139203 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea400ed90 con 0x564f4413ec70 2023-07-13T22:23:17.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.445+0000 7fde9a7fc700 1 -- 172.21.15.26:0/3201139203 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4013af0 con 0x564f4413ec70 2023-07-13T22:23:17.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.447+0000 7fdecd982b80 1 -- 172.21.15.26:0/3201139203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 msgr2=0x564f44141c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.447+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdea4000f80 tx=0x7fdea4013400 comp rx=0 tx=0).stop 2023-07-13T22:23:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.448+0000 7fdecd982b80 1 -- 172.21.15.26:0/3201139203 shutdown_connections 2023-07-13T22:23:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.448+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f44141c10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.448+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f441450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.448+0000 7fdecd982b80 1 --2- 172.21.15.26:0/3201139203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.448+0000 7fdecd982b80 1 -- 172.21.15.26:0/3201139203 >> 172.21.15.26:0/3201139203 conn(0x564f44132da0 msgr2=0x564f4414f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.449+0000 7fdecd982b80 1 -- 172.21.15.26:0/3201139203 shutdown_connections 2023-07-13T22:23:17.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.449+0000 7fdecd982b80 1 -- 172.21.15.26:0/3201139203 wait complete. 2023-07-13T22:23:17.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.450+0000 7fdecd982b80 1 -- start start 2023-07-13T22:23:17.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.461+0000 7fdecd982b80 1 Processor -- start 2023-07-13T22:23:17.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4414d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4414d7a0 con 0x564f4413de20 2023-07-13T22:23:17.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f4414d910 con 0x564f4413d4d0 2023-07-13T22:23:17.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f441a6e20 con 0x564f4413ec70 2023-07-13T22:23:17.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.462+0000 7fdecd982b80 10 client.0 _ll_register_callbacks cb 0x7fdeac005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:17.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.464+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.465+0000 7fdeb37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:60060/0 (socket says 172.21.15.26:60060) 2023-07-13T22:23:17.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.465+0000 7fdeb37fe700 1 -- 172.21.15.26:0/1637659447 learned_addr learned my addr 172.21.15.26:0/1637659447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:17.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.466+0000 7fdeb37fe700 1 -- 172.21.15.26:0/1637659447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 msgr2=0x564f4414d100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:17.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.466+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4414d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.466+0000 7fdeb37fe700 1 -- 172.21.15.26:0/1637659447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 msgr2=0x564f4414cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:17.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.466+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.466+0000 7fdeb37fe700 1 -- 172.21.15.26:0/1637659447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f43ebcde0 con 0x564f4413d4d0 2023-07-13T22:23:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.467+0000 7fdeb37fe700 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005a60 tx=0x7fdeac038c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bbd1527c67e00db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.467+0000 7fde92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.468+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac02e840 con 0x564f4413d4d0 2023-07-13T22:23:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.468+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdeac0243f0 con 0x564f4413d4d0 2023-07-13T22:23:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.468+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdea400f040 con 0x564f4413d4d0 2023-07-13T22:23:17.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.469+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac038e30 con 0x564f4413d4d0 2023-07-13T22:23:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.471+0000 7fdecd982b80 10 client.27020 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:17.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.471+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f43ebb2b0 con 0x564f4413d4d0 2023-07-13T22:23:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdea400f040 con 0x564f4413d4d0 2023-07-13T22:23:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdeac004500 con 0x564f4413d4d0 2023-07-13T22:23:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 10 client.27020 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 10 client.27020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 20 client.27020 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:17.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f441a81b0 0x564f441aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:17.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 4 client.27020 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:23:17.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.472+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f4404a050 con 0x564f441a81b0 2023-07-13T22:23:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.478+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f441a81b0 0x564f441aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.481+0000 7fde92ffd700 10 client.27020 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.481+0000 7fdeb2ffd700 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f441a81b0 0x564f441aa5c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc00cfec62a5819c server_cookie=ecf46ea41349ac99 in_seq=0 out_seq=0 2023-07-13T22:23:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fde92ffd700 1 -- 172.21.15.26:0/1637659447 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f4404a050 con 0x564f441a81b0 2023-07-13T22:23:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fde92ffd700 10 client.27020 handle_command_reply: tid=0 2023-07-13T22:23:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 client.27020 shutdown 2023-07-13T22:23:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 msgr2=0x564f4414e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005a60 tx=0x7fdeac038c10 comp rx=0 tx=0).stop 2023-07-13T22:23:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 shutdown_connections 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x564f441a81b0 0x564f441aa5c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f4413ec70 0x564f4414d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f4413d4d0 0x564f4414e5b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 --2- 172.21.15.26:0/1637659447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f4413de20 0x564f4414cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fde92ffd700 0 client.27020 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.482+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 >> 172.21.15.26:0/1637659447 conn(0x564f44132da0 msgr2=0x564f441450b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.483+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 shutdown_connections 2023-07-13T22:23:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.483+0000 7fdecd982b80 1 -- 172.21.15.26:0/1637659447 wait complete. 2023-07-13T22:23:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.483+0000 7fdecd982b80 20 client.27020 trim_cache size 0 max 16384 2023-07-13T22:23:17.468 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "a1d75256-e675-4d04-98e1-f3f265dea29c": { 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "a1d75256-e675-4d04-98e1-f3f265dea29c", 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:23:17.469 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:23:17.470 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:23:17.470 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.484+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 msgr2=0x7fde9c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.484+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 0x7fde9c0a3800 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b2e0 tx=0x7fdeac009510 comp rx=0 tx=0).stop 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.484+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 msgr2=0x7fdeb4113f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.484+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 0x7fdeb4113f50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdea800e9c0 tx=0x7fdea800ecd0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 shutdown_connections 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fde9c0a1350 0x7fde9c0a3800 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdeb410bd60 0x7fdeb41b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdeb4070ee0 0x7fdeb4113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 --2- 172.21.15.26:0/1369532487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdeb4071830 0x7fdeb4113f50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 >> 172.21.15.26:0/1369532487 conn(0x7fdeb406ca90 msgr2=0x7fdeb406dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 shutdown_connections 2023-07-13T22:23:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:17.485+0000 7fdeb9fff700 1 -- 172.21.15.26:0/1369532487 wait complete. 2023-07-13T22:23:17.504 INFO:tasks.cephfs.filesystem:scrub status for tag:a1d75256-e675-4d04-98e1-f3f265dea29c - {'path': '/', 'tag': 'a1d75256-e675-4d04-98e1-f3f265dea29c', 'options': 'recursive,force'} 2023-07-13T22:23:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:23:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:23:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:23:17.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:23:17.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:17 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:23:17.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:17 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 87 op/s 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:17.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:17 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:18.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:18 smithi121 ceph-mon[129187]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 302 MiB/s wr, 100 op/s 2023-07-13T22:23:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:18 smithi117 ceph-mon[129074]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 302 MiB/s wr, 100 op/s 2023-07-13T22:23:19.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:18 smithi026 ceph-mon[95647]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 302 MiB/s wr, 100 op/s 2023-07-13T22:23:21.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:21 smithi117 ceph-mon[129074]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-13T22:23:21.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:21 smithi026 ceph-mon[95647]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-13T22:23:21.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:21 smithi121 ceph-mon[129187]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 286 MiB/s wr, 92 op/s 2023-07-13T22:23:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:22 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:23:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:22 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:23:22.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:22 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:23:22.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:22 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:23:22.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:22 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:23:22.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:22 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:23:23.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:23 smithi117 ceph-mon[129074]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 267 MiB/s wr, 87 op/s 2023-07-13T22:23:23.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:23 smithi026 ceph-mon[95647]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 267 MiB/s wr, 87 op/s 2023-07-13T22:23:23.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:23 smithi121 ceph-mon[129187]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 267 MiB/s wr, 87 op/s 2023-07-13T22:23:25.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:25 smithi117 ceph-mon[129074]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 322 MiB/s wr, 104 op/s 2023-07-13T22:23:25.956 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:25 smithi026 ceph-mon[95647]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 322 MiB/s wr, 104 op/s 2023-07-13T22:23:25.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:25 smithi121 ceph-mon[129187]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 322 MiB/s wr, 104 op/s 2023-07-13T22:23:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:27 smithi117 ceph-mon[129074]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:23:27.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:27 smithi026 ceph-mon[95647]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:23:27.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:27 smithi121 ceph-mon[129187]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 289 MiB/s wr, 94 op/s 2023-07-13T22:23:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:29 smithi117 ceph-mon[129074]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 321 MiB/s wr, 107 op/s 2023-07-13T22:23:29.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:29 smithi026 ceph-mon[95647]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 321 MiB/s wr, 107 op/s 2023-07-13T22:23:29.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:29 smithi121 ceph-mon[129187]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 321 MiB/s wr, 107 op/s 2023-07-13T22:23:31.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:31 smithi117 ceph-mon[129074]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 97 op/s 2023-07-13T22:23:31.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:31 smithi026 ceph-mon[95647]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 97 op/s 2023-07-13T22:23:31.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:31 smithi121 ceph-mon[129187]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 302 MiB/s wr, 97 op/s 2023-07-13T22:23:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:33 smithi117 ceph-mon[129074]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 96 op/s 2023-07-13T22:23:33.695 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:33 smithi026 ceph-mon[95647]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 96 op/s 2023-07-13T22:23:33.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:33 smithi121 ceph-mon[129187]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 293 MiB/s wr, 96 op/s 2023-07-13T22:23:35.686 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:35 smithi117 ceph-mon[129074]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 336 MiB/s wr, 108 op/s 2023-07-13T22:23:35.697 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:35 smithi026 ceph-mon[95647]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 336 MiB/s wr, 108 op/s 2023-07-13T22:23:35.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:35 smithi121 ceph-mon[129187]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 336 MiB/s wr, 108 op/s 2023-07-13T22:23:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:37 smithi117 ceph-mon[129074]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 287 MiB/s wr, 91 op/s 2023-07-13T22:23:37.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:37 smithi026 ceph-mon[95647]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 287 MiB/s wr, 91 op/s 2023-07-13T22:23:37.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:37 smithi121 ceph-mon[129187]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 287 MiB/s wr, 91 op/s 2023-07-13T22:23:39.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:39 smithi117 ceph-mon[129074]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 308 MiB/s wr, 99 op/s 2023-07-13T22:23:39.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:39 smithi026 ceph-mon[95647]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 308 MiB/s wr, 99 op/s 2023-07-13T22:23:39.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:39 smithi121 ceph-mon[129187]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 308 MiB/s wr, 99 op/s 2023-07-13T22:23:41.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:41 smithi117 ceph-mon[129074]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:23:41.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:41 smithi026 ceph-mon[95647]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:23:41.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:41 smithi121 ceph-mon[129187]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 301 MiB/s wr, 94 op/s 2023-07-13T22:23:43.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:43 smithi121 ceph-mon[129187]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 272 MiB/s wr, 88 op/s 2023-07-13T22:23:44.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:43 smithi117 ceph-mon[129074]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 272 MiB/s wr, 88 op/s 2023-07-13T22:23:44.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:43 smithi026 ceph-mon[95647]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 272 MiB/s wr, 88 op/s 2023-07-13T22:23:45.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:44 smithi117 ceph-mon[129074]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 99 op/s 2023-07-13T22:23:45.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:44 smithi026 ceph-mon[95647]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 99 op/s 2023-07-13T22:23:45.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:44 smithi121 ceph-mon[129187]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 312 MiB/s wr, 99 op/s 2023-07-13T22:23:47.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:23:47.686 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:47 smithi117 ceph-mon[129074]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 272 MiB/s wr, 86 op/s 2023-07-13T22:23:47.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:47 smithi026 ceph-mon[95647]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 272 MiB/s wr, 86 op/s 2023-07-13T22:23:47.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:47 smithi121 ceph-mon[129187]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 272 MiB/s wr, 86 op/s 2023-07-13T22:23:47.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 -- 172.21.15.26:0/526858752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24109d50 msgr2=0x7f0e2410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:47.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 --2- 172.21.15.26:0/526858752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24109d50 0x7f0e2410a1d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009990 tx=0x7f0e18009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:47.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 -- 172.21.15.26:0/526858752 shutdown_connections 2023-07-13T22:23:47.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 --2- 172.21.15.26:0/526858752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24070ee0 0x7f0e240712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:47.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 --2- 172.21.15.26:0/526858752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24109d50 0x7f0e2410a1d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:47.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 --2- 172.21.15.26:0/526858752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24071830 0x7f0e241097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:47.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 -- 172.21.15.26:0/526858752 >> 172.21.15.26:0/526858752 conn(0x7f0e2406ca90 msgr2=0x7f0e2406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.005+0000 7f0e291ff700 1 -- 172.21.15.26:0/526858752 shutdown_connections 2023-07-13T22:23:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.008+0000 7f0e291ff700 1 -- 172.21.15.26:0/526858752 wait complete. 2023-07-13T22:23:47.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.009+0000 7f0e291ff700 1 Processor -- start 2023-07-13T22:23:47.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 -- start start 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 0x7f0e24117130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 0x7f0e24117670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24109d50 0x7f0e2411c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2411cca0 con 0x7f0e24071830 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2411ce10 con 0x7f0e24070ee0 2023-07-13T22:23:47.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.010+0000 7f0e291ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2411cfd0 con 0x7f0e24109d50 2023-07-13T22:23:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.011+0000 7f0e22d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 0x7f0e24117130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.012+0000 7f0e22d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 0x7f0e24117130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43630/0 (socket says 172.21.15.26:43630) 2023-07-13T22:23:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.012+0000 7f0e22d9d700 1 -- 172.21.15.26:0/1074128136 learned_addr learned my addr 172.21.15.26:0/1074128136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:47.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.012+0000 7f0e2359e700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24109d50 0x7f0e2411c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.013+0000 7f0e2259c700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 0x7f0e24117670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 -- 172.21.15.26:0/1074128136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24109d50 msgr2=0x7f0e2411c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24109d50 0x7f0e2411c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 -- 172.21.15.26:0/1074128136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 msgr2=0x7f0e24117130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 0x7f0e24117130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 -- 172.21.15.26:0/1074128136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180095f0 con 0x7f0e24071830 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.014+0000 7f0e2259c700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 0x7f0e24117670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400d740 tx=0x7f0e1400da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.017+0000 7f0e0bfff700 1 -- 172.21.15.26:0/1074128136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004220 con 0x7f0e24071830 2023-07-13T22:23:48.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.017+0000 7f0e0bfff700 1 -- 172.21.15.26:0/1074128136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e14004380 con 0x7f0e24071830 2023-07-13T22:23:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.017+0000 7f0e0bfff700 1 -- 172.21.15.26:0/1074128136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003c70 con 0x7f0e24071830 2023-07-13T22:23:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.017+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2411d250 con 0x7f0e24071830 2023-07-13T22:23:48.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.017+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241b5490 con 0x7f0e24071830 2023-07-13T22:23:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.032+0000 7f0e3cb3fb80 1 Processor -- start 2023-07-13T22:23:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 -- start start 2023-07-13T22:23:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b00824d0 0x55a6b00828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b00836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aff87cb0 con 0x55a6b0082e20 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6affdaa40 con 0x55a6b0083c70 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.033+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6affd3ed0 con 0x55a6b00824d0 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.034+0000 7f0e0bfff700 1 -- 172.21.15.26:0/1074128136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0e1400f690 con 0x7f0e24071830 2023-07-13T22:23:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.035+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.035+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43640/0 (socket says 172.21.15.26:43640) 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.035+0000 7f0e2259c700 1 -- 172.21.15.26:0/141383633 learned_addr learned my addr 172.21.15.26:0/141383633 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.035+0000 7f0e2359e700 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b00824d0 0x55a6b00828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.037+0000 7f0e0bfff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 0x7f0e0c0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.037+0000 7f0e0bfff700 1 -- 172.21.15.26:0/1074128136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e140cbff0 con 0x7f0e24071830 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.038+0000 7f0e2259c700 1 -- 172.21.15.26:0/141383633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b00824d0 msgr2=0x55a6b00828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.038+0000 7f0e2259c700 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b00824d0 0x55a6b00828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.038+0000 7f0e2259c700 1 -- 172.21.15.26:0/141383633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 msgr2=0x55a6b00836e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.038+0000 7f0e2259c700 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b00836e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.039+0000 7f0e2259c700 1 -- 172.21.15.26:0/141383633 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6afe01de0 con 0x55a6b0083c70 2023-07-13T22:23:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.039+0000 7f0e2259c700 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0e14095e20 tx=0x7f0e14003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81bbc60b16a130cc server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.042+0000 7f0e2359e700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 0x7f0e0c0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.046+0000 7f0e097fa700 1 -- 172.21.15.26:0/141383633 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14098d10 con 0x55a6b0083c70 2023-07-13T22:23:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.046+0000 7f0e097fa700 1 -- 172.21.15.26:0/141383633 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e140044f0 con 0x55a6b0083c70 2023-07-13T22:23:48.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.046+0000 7f0e097fa700 1 -- 172.21.15.26:0/141383633 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14097750 con 0x55a6b0083c70 2023-07-13T22:23:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.047+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/141383633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 msgr2=0x55a6b00840e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.047+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0e14095e20 tx=0x7f0e14003710 comp rx=0 tx=0).stop 2023-07-13T22:23:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.049+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/141383633 shutdown_connections 2023-07-13T22:23:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.049+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0083c70 0x55a6b00840e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.049+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b00824d0 0x55a6b00828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.049+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/141383633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b00836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.049+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/141383633 >> 172.21.15.26:0/141383633 conn(0x55a6b0077da0 msgr2=0x55a6b007d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.051+0000 7f0e2359e700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 0x7f0e0c0a37d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006010 tx=0x7f0e18009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.052+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/141383633 shutdown_connections 2023-07-13T22:23:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.052+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/141383633 wait complete. 2023-07-13T22:23:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.053+0000 7f0e3cb3fb80 1 -- start start 2023-07-13T22:23:48.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.053+0000 7f0e3cb3fb80 1 Processor -- start 2023-07-13T22:23:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 0x55a6b008c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0093730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aff73ac0 con 0x55a6b0082e20 2023-07-13T22:23:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6aff737c0 con 0x55a6b00824d0 2023-07-13T22:23:48.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6afe57920 con 0x55a6b0083c70 2023-07-13T22:23:48.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.054+0000 7f0e3cb3fb80 10 client.0 _ll_register_callbacks cb 0x55a6aff71760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.059+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.059+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60442/0 (socket says 172.21.15.26:60442) 2023-07-13T22:23:48.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.059+0000 7f0e2259c700 1 -- 172.21.15.26:0/2168993095 learned_addr learned my addr 172.21.15.26:0/2168993095 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 0x55a6b008c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e2259c700 1 -- 172.21.15.26:0/2168993095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 msgr2=0x55a6b0093730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:48.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e2259c700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0093730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e2259c700 1 -- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 msgr2=0x55a6b008c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e2259c700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 0x55a6b008c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e2259c700 1 -- 172.21.15.26:0/2168993095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241b5490 con 0x55a6b0082e20 2023-07-13T22:23:48.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.062+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 0x55a6b008c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.063+0000 7f0e2259c700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cc2e0 tx=0x7f0e140976a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44fd200f78bfd6f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.063+0000 7f0e06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.063+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140cf0d0 con 0x55a6b0082e20 2023-07-13T22:23:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.063+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e140cf230 con 0x55a6b0082e20 2023-07-13T22:23:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.063+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6aff2d4f0 con 0x55a6b0082e20 2023-07-13T22:23:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.064+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e140988a0 con 0x55a6b0082e20 2023-07-13T22:23:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.064+0000 7f0e3cb3fb80 10 client.16491 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:48.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.064+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a6afdeac60 con 0x55a6b0082e20 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e140b5020 con 0x55a6b0082e20 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e140a1370 con 0x55a6b0082e20 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 10 client.16491 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 20 client.16491 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00979f0 0x55a6b0099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 4 client.16491 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6affcceb0 con 0x55a6b00979f0 2023-07-13T22:23:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.065+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00979f0 0x55a6b0099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.066+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00979f0 0x55a6b0099dd0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=780a71798b17bcff server_cookie=f3c18816fe98e7d1 in_seq=0 out_seq=0 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.066+0000 7f0e06ff5700 10 client.16491 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.067+0000 7f0e06ff5700 1 -- 172.21.15.26:0/2168993095 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6affcceb0 con 0x55a6b00979f0 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.067+0000 7f0e06ff5700 10 client.16491 handle_command_reply: tid=0 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.067+0000 7f0e3cb3fb80 1 client.16491 shutdown 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.067+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 msgr2=0x55a6b008c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.067+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cc2e0 tx=0x7f0e140976a0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 shutdown_connections 2023-07-13T22:23:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00979f0 0x55a6b0099dd0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0093730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b00824d0 0x55a6b008c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/2168993095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b0082e20 0x55a6b008c6f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 >> 172.21.15.26:0/2168993095 conn(0x55a6b0077da0 msgr2=0x55a6b007c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e06ff5700 0 client.16491 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 shutdown_connections 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.068+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/2168993095 wait complete. 2023-07-13T22:23:48.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.070+0000 7f0e3cb3fb80 20 client.16491 trim_cache size 0 max 16384 2023-07-13T22:23:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.105+0000 7f0e3cb3fb80 1 Processor -- start 2023-07-13T22:23:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.105+0000 7f0e3cb3fb80 1 -- start start 2023-07-13T22:23:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b008aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b0097860 con 0x55a6b00824d0 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b0093b20 con 0x55a6b0079310 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b0089db0 con 0x55a6b0083c70 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e2259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:37200/0 (socket says 172.21.15.26:37200) 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e2259c700 1 -- 172.21.15.26:0/112093244 learned_addr learned my addr 172.21.15.26:0/112093244 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:48.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.106+0000 7f0e2359e700 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b008aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 -- 172.21.15.26:0/112093244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 msgr2=0x55a6b008aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b008aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 -- 172.21.15.26:0/112093244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 msgr2=0x55a6b00a75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 -- 172.21.15.26:0/112093244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6afe28250 con 0x55a6b0083c70 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.107+0000 7f0e2259c700 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e14099340 tx=0x7f0e140a1890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cd73c779a349ae server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.108+0000 7f0e09ffb700 1 -- 172.21.15.26:0/112093244 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140988a0 con 0x55a6b0083c70 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.108+0000 7f0e09ffb700 1 -- 172.21.15.26:0/112093244 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a6aff2d4f0 con 0x55a6b0083c70 2023-07-13T22:23:48.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.108+0000 7f0e09ffb700 1 -- 172.21.15.26:0/112093244 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140cf330 con 0x55a6b0083c70 2023-07-13T22:23:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/112093244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 msgr2=0x55a6b0094020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0e14099340 tx=0x7f0e140a1890 comp rx=0 tx=0).stop 2023-07-13T22:23:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/112093244 shutdown_connections 2023-07-13T22:23:48.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b008aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/112093244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b0094020 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/112093244 >> 172.21.15.26:0/112093244 conn(0x55a6b0077da0 msgr2=0x55a6b009e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.109+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/112093244 shutdown_connections 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.110+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/112093244 wait complete. 2023-07-13T22:23:48.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.110+0000 7f0e3cb3fb80 1 -- start start 2023-07-13T22:23:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 Processor -- start 2023-07-13T22:23:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b00a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b00ee030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b00ee570 con 0x55a6b00824d0 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b00ee6e0 con 0x55a6b0079310 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.111+0000 7f0e3cb3fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6b00ee8a0 con 0x55a6b0083c70 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.112+0000 7f0e3cb3fb80 10 client.0 _ll_register_callbacks cb 0x7f0e18009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.112+0000 7f0e2359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b00ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.112+0000 7f0e2359e700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b00ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:37214/0 (socket says 172.21.15.26:37214) 2023-07-13T22:23:48.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.112+0000 7f0e2359e700 1 -- 172.21.15.26:0/4183423950 learned_addr learned my addr 172.21.15.26:0/4183423950 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.112+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b00a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.113+0000 7f0e22d9d700 1 -- 172.21.15.26:0/4183423950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 msgr2=0x55a6b00ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.113+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b00ee030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.113+0000 7f0e22d9d700 1 -- 172.21.15.26:0/4183423950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 msgr2=0x55a6b00a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.113+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.113+0000 7f0e22d9d700 1 -- 172.21.15.26:0/4183423950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6afdeac60 con 0x55a6b0079310 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.114+0000 7f0e22d9d700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b00a68c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00afd0 tx=0x7f0e1c018550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ffcf75a71b49b30 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.114+0000 7f0e06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.114+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c017c00 con 0x55a6b0079310 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.114+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0e1c006e40 con 0x55a6b0079310 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.115+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e140044f0 con 0x55a6b0079310 2023-07-13T22:23:48.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.115+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1c017d60 con 0x55a6b0079310 2023-07-13T22:23:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.117+0000 7f0e3cb3fb80 10 client.27035 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:48.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.117+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a6afe002b0 con 0x55a6b0079310 2023-07-13T22:23:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e140044f0 con 0x55a6b0079310 2023-07-13T22:23:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0e1c028070 con 0x55a6b0079310 2023-07-13T22:23:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 10 client.27035 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 10 client.27035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 20 client.27035 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00f0220 0x55a6b00f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 4 client.27035 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:23:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6aff8f050 con 0x55a6b00f0220 2023-07-13T22:23:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.118+0000 7f0e2259c700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00f0220 0x55a6b00f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.120+0000 7f0e2259c700 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00f0220 0x55a6b00f2630 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aac0e0a00ebec5f server_cookie=1b0df41875f175fc in_seq=0 out_seq=0 2023-07-13T22:23:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.120+0000 7f0e06ff5700 10 client.27035 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.120+0000 7f0e06ff5700 1 -- 172.21.15.26:0/4183423950 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a6aff8f050 con 0x55a6b00f0220 2023-07-13T22:23:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.120+0000 7f0e06ff5700 10 client.27035 handle_command_reply: tid=0 2023-07-13T22:23:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.120+0000 7f0e3cb3fb80 1 client.27035 shutdown 2023-07-13T22:23:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 msgr2=0x55a6b00a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b00a68c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00afd0 tx=0x7f0e1c018550 comp rx=0 tx=0).stop 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 shutdown_connections 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55a6b00f0220 0x55a6b00f2630 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6b0083c70 0x55a6b00ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6b0079310 0x55a6b00a68c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 --2- 172.21.15.26:0/4183423950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a6b00824d0 0x55a6b00a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 >> 172.21.15.26:0/4183423950 conn(0x55a6b0077da0 msgr2=0x55a6b008aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.121+0000 7f0e06ff5700 0 client.27035 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:48.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.122+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 shutdown_connections 2023-07-13T22:23:48.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.122+0000 7f0e3cb3fb80 1 -- 172.21.15.26:0/4183423950 wait complete. 2023-07-13T22:23:48.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.123+0000 7f0e3cb3fb80 20 client.27035 trim_cache size 0 max 16384 2023-07-13T22:23:48.107 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:48.107 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:23:48.107 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:23:48.107 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.124+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 msgr2=0x7f0e0c0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.124+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 0x7f0e0c0a37d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e18006010 tx=0x7f0e18009310 comp rx=0 tx=0).stop 2023-07-13T22:23:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.124+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 msgr2=0x7f0e24117670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.124+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 0x7f0e24117670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400d740 tx=0x7f0e1400da50 comp rx=0 tx=0).stop 2023-07-13T22:23:48.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 shutdown_connections 2023-07-13T22:23:48.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0e24109d50 0x7f0e2411c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e24070ee0 0x7f0e24117130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f0e0c0a1320 0x7f0e0c0a37d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 --2- 172.21.15.26:0/1074128136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e24071830 0x7f0e24117670 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.125+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 >> 172.21.15.26:0/1074128136 conn(0x7f0e2406ca90 msgr2=0x7f0e2406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.126+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 shutdown_connections 2023-07-13T22:23:48.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.126+0000 7f0e291ff700 1 -- 172.21.15.26:0/1074128136 wait complete. 2023-07-13T22:23:48.145 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:23:48.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:23:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.664+0000 7f68dabff700 1 -- 172.21.15.26:0/3476635558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d4071820 msgr2=0x7f68d410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.664+0000 7f68dabff700 1 --2- 172.21.15.26:0/3476635558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d4071820 0x7f68d410b7d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009990 tx=0x7f68c8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 -- 172.21.15.26:0/3476635558 shutdown_connections 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 --2- 172.21.15.26:0/3476635558 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d410bd60 0x7f68d410e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 --2- 172.21.15.26:0/3476635558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4070ed0 0x7f68d40712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 --2- 172.21.15.26:0/3476635558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d4071820 0x7f68d410b7d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 -- 172.21.15.26:0/3476635558 >> 172.21.15.26:0/3476635558 conn(0x7f68d406ca40 msgr2=0x7f68d406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:48.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.665+0000 7f68dabff700 1 -- 172.21.15.26:0/3476635558 shutdown_connections 2023-07-13T22:23:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.666+0000 7f68dabff700 1 -- 172.21.15.26:0/3476635558 wait complete. 2023-07-13T22:23:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.667+0000 7f68dabff700 1 Processor -- start 2023-07-13T22:23:48.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.668+0000 7f68dabff700 1 -- start start 2023-07-13T22:23:48.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4071820 0x7f68d4113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d410bd60 0x7f68d41b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d4117060 con 0x7f68d4070ed0 2023-07-13T22:23:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41171d0 con 0x7f68d410bd60 2023-07-13T22:23:48.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68dabff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41b4dc0 con 0x7f68d4071820 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.673+0000 7f68d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60476/0 (socket says 172.21.15.26:60476) 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 -- 172.21.15.26:0/2007634887 learned_addr learned my addr 172.21.15.26:0/2007634887 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d3fff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4071820 0x7f68d4113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d919c700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d410bd60 0x7f68d41b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 -- 172.21.15.26:0/2007634887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4071820 msgr2=0x7f68d4113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4071820 0x7f68d4113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 -- 172.21.15.26:0/2007634887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d410bd60 msgr2=0x7f68d41b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d410bd60 0x7f68d41b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 -- 172.21.15.26:0/2007634887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80095f0 con 0x7f68d4070ed0 2023-07-13T22:23:48.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.674+0000 7f68d899b700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f68c400d950 tx=0x7f68c400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.675+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4009ae0 con 0x7f68d4070ed0 2023-07-13T22:23:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.675+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68c4009c40 con 0x7f68d4070ed0 2023-07-13T22:23:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.675+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400f620 con 0x7f68d4070ed0 2023-07-13T22:23:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.675+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d41b4fc0 con 0x7f68d4070ed0 2023-07-13T22:23:48.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.675+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d41b5390 con 0x7f68d4070ed0 2023-07-13T22:23:48.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.677+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d411a130 con 0x7f68d4070ed0 2023-07-13T22:23:48.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.681+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f68c4010460 con 0x7f68d4070ed0 2023-07-13T22:23:48.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.684+0000 7f68d1ffb700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 0x7f68bc0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:48.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.684+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f68c40cc7f0 con 0x7f68d4070ed0 2023-07-13T22:23:48.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.684+0000 7f68d3fff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 0x7f68bc0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:48.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.685+0000 7f68d3fff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 0x7f68bc0a3700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f68c80098e0 tx=0x7f68c800bf40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:48.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:48.691+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f68c4095ef0 con 0x7f68d4070ed0 2023-07-13T22:23:49.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.304+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68d411a2f0 con 0x7f68d4070ed0 2023-07-13T22:23:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.313+0000 7f68d1ffb700 1 -- 172.21.15.26:0/2007634887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f68c4099510 con 0x7f68d4070ed0 2023-07-13T22:23:49.297 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:23:49.298 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:23:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.340+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 msgr2=0x7f68bc0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.340+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 0x7f68bc0a3700 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f68c80098e0 tx=0x7f68c800bf40 comp rx=0 tx=0).stop 2023-07-13T22:23:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.340+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 msgr2=0x7f68d4113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.340+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f68c400d950 tx=0x7f68c400dd10 comp rx=0 tx=0).stop 2023-07-13T22:23:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 shutdown_connections 2023-07-13T22:23:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f68bc0a1250 0x7f68bc0a3700 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4070ed0 0x7f68d4113a40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68d410bd60 0x7f68d41b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 --2- 172.21.15.26:0/2007634887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f68d4071820 0x7f68d4113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.342+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 >> 172.21.15.26:0/2007634887 conn(0x7f68d406ca40 msgr2=0x7f68d406dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.344+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 shutdown_connections 2023-07-13T22:23:49.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:49.344+0000 7f68dabff700 1 -- 172.21.15.26:0/2007634887 wait complete. 2023-07-13T22:23:49.368 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:23:49.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:23:49.656 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:49 smithi026 ceph-mon[95647]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:49.656 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2007634887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:49.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:49 smithi121 ceph-mon[129187]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:49.737 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2007634887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:49.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:49 smithi117 ceph-mon[129074]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 299 MiB/s wr, 97 op/s 2023-07-13T22:23:49.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2007634887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:23:50.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.042+0000 7fa831dff700 1 -- 172.21.15.26:0/1838103537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c10bd60 msgr2=0x7fa82c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.042+0000 7fa831dff700 1 --2- 172.21.15.26:0/1838103537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c10bd60 0x7fa82c10e1f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b0a0 tx=0x7fa82400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 -- 172.21.15.26:0/1838103537 shutdown_connections 2023-07-13T22:23:50.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 --2- 172.21.15.26:0/1838103537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c10bd60 0x7fa82c10e1f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 --2- 172.21.15.26:0/1838103537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c070ee0 0x7fa82c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 --2- 172.21.15.26:0/1838103537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa82c071830 0x7fa82c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 -- 172.21.15.26:0/1838103537 >> 172.21.15.26:0/1838103537 conn(0x7fa82c06ca90 msgr2=0x7fa82c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.043+0000 7fa831dff700 1 -- 172.21.15.26:0/1838103537 shutdown_connections 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.044+0000 7fa831dff700 1 -- 172.21.15.26:0/1838103537 wait complete. 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.045+0000 7fa831dff700 1 Processor -- start 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 -- start start 2023-07-13T22:23:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c071830 0x7fa82c113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa82c10bd60 0x7fa82c1b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c117040 con 0x7fa82c10bd60 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1171b0 con 0x7fa82c070ee0 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.046+0000 7fa831dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1b4e20 con 0x7fa82c071830 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.047+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.047+0000 7fa82b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43706/0 (socket says 172.21.15.26:43706) 2023-07-13T22:23:50.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.047+0000 7fa82b7fe700 1 -- 172.21.15.26:0/2080128568 learned_addr learned my addr 172.21.15.26:0/2080128568 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:50.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.048+0000 7fa82affd700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c071830 0x7fa82c113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 -- 172.21.15.26:0/2080128568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c071830 msgr2=0x7fa82c113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c071830 0x7fa82c113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 -- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa82c10bd60 msgr2=0x7fa82c1b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa82c10bd60 0x7fa82c1b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 -- 172.21.15.26:0/2080128568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009d00 con 0x7fa82c070ee0 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.049+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00e9a0 tx=0x7fa81c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.050+0000 7fa828ff9700 1 -- 172.21.15.26:0/2080128568 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009790 con 0x7fa82c070ee0 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.050+0000 7fa828ff9700 1 -- 172.21.15.26:0/2080128568 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c00cb70 con 0x7fa82c070ee0 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.050+0000 7fa828ff9700 1 -- 172.21.15.26:0/2080128568 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c018680 con 0x7fa82c070ee0 2023-07-13T22:23:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.050+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c1b4fc0 con 0x7fa82c070ee0 2023-07-13T22:23:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.051+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1b5470 con 0x7fa82c070ee0 2023-07-13T22:23:50.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.061+0000 7fa8456d0b80 1 Processor -- start 2023-07-13T22:23:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.062+0000 7fa828ff9700 1 -- 172.21.15.26:0/2080128568 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa81c010430 con 0x7fa82c070ee0 2023-07-13T22:23:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.065+0000 7fa828ff9700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 0x7fa8140a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.066+0000 7fa828ff9700 1 -- 172.21.15.26:0/2080128568 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa81c014070 con 0x7fa82c070ee0 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.067+0000 7fa8456d0b80 1 -- start start 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.067+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a6f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.067+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6fe20 0x55ff83a706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.067+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.086+0000 7fa82bfff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 0x7fa8140a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.086+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83974cb0 con 0x55ff83a6fe20 2023-07-13T22:23:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.086+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff839c7a40 con 0x55ff83a70c70 2023-07-13T22:23:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.086+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff839c0ed0 con 0x55ff83a6f4d0 2023-07-13T22:23:50.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.089+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.089+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43710/0 (socket says 172.21.15.26:43710) 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.089+0000 7fa82bfff700 1 -- 172.21.15.26:0/1617569256 learned_addr learned my addr 172.21.15.26:0/1617569256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a6f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 -- 172.21.15.26:0/1617569256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 msgr2=0x55ff83a6f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a6f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 -- 172.21.15.26:0/1617569256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6fe20 msgr2=0x55ff83a706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6fe20 0x55ff83a706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 -- 172.21.15.26:0/1617569256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff837eede0 con 0x55ff83a70c70 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.090+0000 7fa82bfff700 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa82400bb30 tx=0x7fa82400bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f450167d9e6804 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.093+0000 7fa82bfff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 0x7fa8140a3800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa824007ac0 tx=0x7fa82401cb90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.093+0000 7fa811ffb700 1 -- 172.21.15.26:0/1617569256 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400e030 con 0x55ff83a70c70 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.094+0000 7fa811ffb700 1 -- 172.21.15.26:0/1617569256 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa824023070 con 0x55ff83a70c70 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.094+0000 7fa811ffb700 1 -- 172.21.15.26:0/1617569256 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82401c930 con 0x55ff83a70c70 2023-07-13T22:23:50.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.095+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1617569256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 msgr2=0x55ff83a710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.095+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa82400bb30 tx=0x7fa82400bb60 comp rx=0 tx=0).stop 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.099+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1617569256 shutdown_connections 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.099+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a710e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.099+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a6f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.099+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1617569256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6fe20 0x55ff83a706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.099+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1617569256 >> 172.21.15.26:0/1617569256 conn(0x55ff83a64da0 msgr2=0x55ff83a6a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.112+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1617569256 shutdown_connections 2023-07-13T22:23:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.112+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1617569256 wait complete. 2023-07-13T22:23:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.113+0000 7fa8456d0b80 1 -- start start 2023-07-13T22:23:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.114+0000 7fa8456d0b80 1 Processor -- start 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a791b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a70c70 0x55ff83a80730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83960ac0 con 0x55ff83a70c70 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff839607c0 con 0x55ff83a6fe20 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.119+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83844920 con 0x55ff83a6f4d0 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.120+0000 7fa8456d0b80 10 client.0 _ll_register_callbacks cb 0x55ff8395e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.125+0000 7fa82affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.125+0000 7fa82affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43714/0 (socket says 172.21.15.26:43714) 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.125+0000 7fa82affd700 1 -- 172.21.15.26:0/1113642831 learned_addr learned my addr 172.21.15.26:0/1113642831 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82bfff700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a70c70 0x55ff83a80730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a791b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 -- 172.21.15.26:0/1113642831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 msgr2=0x55ff83a791b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a791b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 -- 172.21.15.26:0/1113642831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a70c70 msgr2=0x55ff83a80730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a70c70 0x55ff83a80730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 -- 172.21.15.26:0/1113642831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c117640 con 0x55ff83a6fe20 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.126+0000 7fa82affd700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d160 tx=0x7fa82000d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6a1370489c8fd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.134+0000 7fa80affd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.134+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82000de60 con 0x55ff83a6fe20 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.134+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa820013480 con 0x55ff83a6fe20 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.134+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff8391a4f0 con 0x55ff83a6fe20 2023-07-13T22:23:50.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.145+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa820013640 con 0x55ff83a6fe20 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.145+0000 7fa8456d0b80 10 client.27050 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.146+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff837d7c60 con 0x55ff83a6fe20 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.159+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff8391a4f0 con 0x55ff83a6fe20 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.159+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa820012940 con 0x55ff83a6fe20 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 10 client.27050 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 10 client.27050 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 20 client.27050 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:50.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83a85080 0x55ff83a87490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 4 client.27050 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.160+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff839b9eb0 con 0x55ff83a85080 2023-07-13T22:23:50.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.164+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83a85080 0x55ff83a87490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.181+0000 7fa80affd700 10 client.27050 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:50.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.180+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83a85080 0x55ff83a87490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23a029f541c62e5 server_cookie=44c9c75323546426 in_seq=0 out_seq=0 2023-07-13T22:23:50.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.192+0000 7fa80affd700 1 -- 172.21.15.26:0/1113642831 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff839b9eb0 con 0x55ff83a85080 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.192+0000 7fa80affd700 10 client.27050 handle_command_reply: tid=0 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.192+0000 7fa8456d0b80 1 client.27050 shutdown 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.192+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 msgr2=0x55ff83a796f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.192+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d160 tx=0x7fa82000d520 comp rx=0 tx=0).stop 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 shutdown_connections 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83a85080 0x55ff83a87490 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a70c70 0x55ff83a80730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6f4d0 0x55ff83a791b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1113642831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a6fe20 0x55ff83a796f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 >> 172.21.15.26:0/1113642831 conn(0x55ff83a64da0 msgr2=0x55ff83a698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.193+0000 7fa80affd700 0 client.27050 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.195+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 shutdown_connections 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.198+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1113642831 wait complete. 2023-07-13T22:23:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.206+0000 7fa8456d0b80 20 client.27050 trim_cache size 0 max 16384 2023-07-13T22:23:50.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 Processor -- start 2023-07-13T22:23:50.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 -- start start 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a77ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a6bcc0 con 0x55ff83a6f4d0 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a6b270 con 0x55ff83a70c70 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.248+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a79e30 con 0x55ff83a6fe20 2023-07-13T22:23:50.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.250+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a77ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.250+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a77ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43728/0 (socket says 172.21.15.26:43728) 2023-07-13T22:23:50.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.250+0000 7fa82bfff700 1 -- 172.21.15.26:0/1801108887 learned_addr learned my addr 172.21.15.26:0/1801108887 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:50.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.250+0000 7fa82affd700 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.261+0000 7fa82affd700 1 -- 172.21.15.26:0/1801108887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 msgr2=0x55ff83a77ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.261+0000 7fa82affd700 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a77ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.261+0000 7fa82affd700 1 -- 172.21.15.26:0/1801108887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 msgr2=0x55ff83a81020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:23:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.261+0000 7fa82affd700 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a81020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.261+0000 7fa82affd700 1 -- 172.21.15.26:0/1801108887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff83815250 con 0x55ff83a6fe20 2023-07-13T22:23:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.269+0000 7fa82affd700 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a806f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa820012cd0 tx=0x7fa820006e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668431c547afe358 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.274+0000 7fa8127fc700 1 -- 172.21.15.26:0/1801108887 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820007000 con 0x55ff83a6fe20 2023-07-13T22:23:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.274+0000 7fa8127fc700 1 -- 172.21.15.26:0/1801108887 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82001b350 con 0x55ff83a6fe20 2023-07-13T22:23:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.274+0000 7fa8127fc700 1 -- 172.21.15.26:0/1801108887 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820012750 con 0x55ff83a6fe20 2023-07-13T22:23:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.275+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1801108887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 msgr2=0x55ff83a806f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.276+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a806f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa820012cd0 tx=0x7fa820006e10 comp rx=0 tx=0).stop 2023-07-13T22:23:50.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.292+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1801108887 shutdown_connections 2023-07-13T22:23:50.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.292+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a81020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.292+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a77ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.292+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/1801108887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a806f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.292+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1801108887 >> 172.21.15.26:0/1801108887 conn(0x55ff83a64da0 msgr2=0x55ff83a8bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.306+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1801108887 shutdown_connections 2023-07-13T22:23:50.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.306+0000 7fa8456d0b80 1 -- 172.21.15.26:0/1801108887 wait complete. 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.307+0000 7fa8456d0b80 1 -- start start 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.307+0000 7fa8456d0b80 1 Processor -- start 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a80050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a80590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a7cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a7d1f0 con 0x55ff83a6f4d0 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a7d360 con 0x55ff83a70c70 2023-07-13T22:23:50.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff83a7d520 con 0x55ff83a6fe20 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.308+0000 7fa8456d0b80 10 client.0 _ll_register_callbacks cb 0x7fa820007820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.312+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a7cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.312+0000 7fa82bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a7cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43738/0 (socket says 172.21.15.26:43738) 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.312+0000 7fa82bfff700 1 -- 172.21.15.26:0/3442752860 learned_addr learned my addr 172.21.15.26:0/3442752860 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.312+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a80050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.318+0000 7fa82affd700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a80590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.327+0000 7fa82b7fe700 1 -- 172.21.15.26:0/3442752860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 msgr2=0x55ff83a80590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.327+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a80590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.327+0000 7fa82b7fe700 1 -- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 msgr2=0x55ff83a7cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.327+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a7cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.327+0000 7fa82b7fe700 1 -- 172.21.15.26:0/3442752860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff837d7c60 con 0x55ff83a6f4d0 2023-07-13T22:23:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.334+0000 7fa82b7fe700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a80050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c096220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed6ab75e3ccac0c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.334+0000 7fa80affd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.339+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c099350 con 0x55ff83a6f4d0 2023-07-13T22:23:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.339+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa81c0b1070 con 0x55ff83a6f4d0 2023-07-13T22:23:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.340+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa824023070 con 0x55ff83a6f4d0 2023-07-13T22:23:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.352+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c0b8250 con 0x55ff83a6f4d0 2023-07-13T22:23:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.353+0000 7fa8456d0b80 10 client.16503 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:50.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.353+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff837ed2b0 con 0x55ff83a6f4d0 2023-07-13T22:23:50.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa824023070 con 0x55ff83a6f4d0 2023-07-13T22:23:50.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa81c0adb60 con 0x55ff83a6f4d0 2023-07-13T22:23:50.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 10 client.16503 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 20 client.16503 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83ada470 0x55ff83adc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 4 client.16503 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.374+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff8397c050 con 0x55ff83ada470 2023-07-13T22:23:50.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.381+0000 7fa82affd700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83ada470 0x55ff83adc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:50.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.384+0000 7fa82affd700 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83ada470 0x55ff83adc880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a92fcdd05e3e472 server_cookie=4bdfe5e98160466c in_seq=0 out_seq=0 2023-07-13T22:23:50.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.385+0000 7fa80affd700 10 client.16503 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.400+0000 7fa80affd700 1 -- 172.21.15.26:0/3442752860 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff8397c050 con 0x55ff83ada470 2023-07-13T22:23:50.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.400+0000 7fa80affd700 10 client.16503 handle_command_reply: tid=0 2023-07-13T22:23:50.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.400+0000 7fa8456d0b80 1 client.16503 shutdown 2023-07-13T22:23:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.405+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 msgr2=0x55ff83a80050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.405+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a80050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa81c000c00 tx=0x7fa81c096220 comp rx=0 tx=0).stop 2023-07-13T22:23:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 shutdown_connections 2023-07-13T22:23:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x55ff83ada470 0x55ff83adc880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff83a6f4d0 0x55ff83a80050 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff83a70c70 0x55ff83a7cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 --2- 172.21.15.26:0/3442752860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff83a6fe20 0x55ff83a80590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.419+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 >> 172.21.15.26:0/3442752860 conn(0x55ff83a64da0 msgr2=0x55ff83a81020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.430+0000 7fa80affd700 0 client.16503 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:23:50.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.444+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 shutdown_connections 2023-07-13T22:23:50.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.466+0000 7fa8456d0b80 1 -- 172.21.15.26:0/3442752860 wait complete. 2023-07-13T22:23:50.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.477+0000 7fa8456d0b80 20 client.16503 trim_cache size 0 max 16384 2023-07-13T22:23:50.464 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.484+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 msgr2=0x7fa8140a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.484+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 0x7fa8140a3800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa824007ac0 tx=0x7fa82401cb90 comp rx=0 tx=0).stop 2023-07-13T22:23:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.484+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 msgr2=0x7fa82c113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:50.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.484+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa81c00e9a0 tx=0x7fa81c00ed60 comp rx=0 tx=0).stop 2023-07-13T22:23:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 shutdown_connections 2023-07-13T22:23:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa8140a1350 0x7fa8140a3800 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa82c10bd60 0x7fa82c1b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa82c070ee0 0x7fa82c113a20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 --2- 172.21.15.26:0/2080128568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa82c071830 0x7fa82c113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 >> 172.21.15.26:0/2080128568 conn(0x7fa82c06ca90 msgr2=0x7fa82c06dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.485+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 shutdown_connections 2023-07-13T22:23:50.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:50.486+0000 7fa831dff700 1 -- 172.21.15.26:0/2080128568 wait complete. 2023-07-13T22:23:50.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:23:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 -- 172.21.15.26:0/3127479292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0071830 msgr2=0x7f6dc010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/3127479292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0071830 0x7f6dc010b7d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009990 tx=0x7f6db0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 -- 172.21.15.26:0/3127479292 shutdown_connections 2023-07-13T22:23:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/3127479292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc010bd60 0x7f6dc010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/3127479292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0070ee0 0x7f6dc00712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/3127479292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0071830 0x7f6dc010b7d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 -- 172.21.15.26:0/3127479292 >> 172.21.15.26:0/3127479292 conn(0x7f6dc006ca90 msgr2=0x7f6dc006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 -- 172.21.15.26:0/3127479292 shutdown_connections 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.458+0000 7f6dc53ff700 1 -- 172.21.15.26:0/3127479292 wait complete. 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.460+0000 7f6dc53ff700 1 Processor -- start 2023-07-13T22:23:51.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.460+0000 7f6dc53ff700 1 -- start start 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0070ee0 0x7f6dc01a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc010bd60 0x7f6dc01abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01ac240 con 0x7f6dc010bd60 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc010ec40 con 0x7f6dc0071830 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.461+0000 7f6dc53ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc010edd0 con 0x7f6dc0070ee0 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.462+0000 7f6db67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.462+0000 7f6db67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43758/0 (socket says 172.21.15.26:43758) 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.462+0000 7f6db67fc700 1 -- 172.21.15.26:0/578208 learned_addr learned my addr 172.21.15.26:0/578208 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.463+0000 7f6db67fc700 1 -- 172.21.15.26:0/578208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0070ee0 msgr2=0x7f6dc01a46c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.463+0000 7f6db67fc700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0070ee0 0x7f6dc01a46c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.463+0000 7f6db67fc700 1 -- 172.21.15.26:0/578208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc010bd60 msgr2=0x7f6dc01abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.463+0000 7f6db67fc700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc010bd60 0x7f6dc01abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.463+0000 7f6db67fc700 1 -- 172.21.15.26:0/578208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00095f0 con 0x7f6dc0071830 2023-07-13T22:23:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.464+0000 7f6db67fc700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6db0005f50 tx=0x7f6db000f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.465+0000 7f6dbcff9700 1 -- 172.21.15.26:0/578208 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db001d070 con 0x7f6dc0071830 2023-07-13T22:23:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.465+0000 7f6dbcff9700 1 -- 172.21.15.26:0/578208 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db000fc20 con 0x7f6dc0071830 2023-07-13T22:23:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.465+0000 7f6dbcff9700 1 -- 172.21.15.26:0/578208 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0017650 con 0x7f6dc0071830 2023-07-13T22:23:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.467+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc010f050 con 0x7f6dc0071830 2023-07-13T22:23:51.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.467+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc010f500 con 0x7f6dc0071830 2023-07-13T22:23:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.475+0000 7f6dd8c87b80 1 Processor -- start 2023-07-13T22:23:51.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.479+0000 7f6dbcff9700 1 -- 172.21.15.26:0/578208 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6db0022470 con 0x7f6dc0071830 2023-07-13T22:23:51.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.490+0000 7f6dbcff9700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 0x7f6dac0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.490+0000 7f6dd8c87b80 1 -- start start 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.490+0000 7f6dbcff9700 1 -- 172.21.15.26:0/578208 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6db00cdad0 con 0x7f6dc0071830 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaaa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bf9afcb0 con 0x5577bfaaae20 2023-07-13T22:23:51.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfa02a40 con 0x5577bfaaa4d0 2023-07-13T22:23:51.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.491+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bf9fbed0 con 0x5577bfaabc70 2023-07-13T22:23:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.492+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 0x7f6dac0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.492+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaaa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60562/0 (socket says 172.21.15.26:60562) 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 -- 172.21.15.26:0/1910664787 learned_addr learned my addr 172.21.15.26:0/1910664787 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 -- 172.21.15.26:0/1910664787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 msgr2=0x5577bfaac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 -- 172.21.15.26:0/1910664787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 msgr2=0x5577bfaaa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaaa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.493+0000 7f6db67fc700 1 -- 172.21.15.26:0/1910664787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577bf829de0 con 0x5577bfaaae20 2023-07-13T22:23:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.494+0000 7f6db67fc700 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6db009aeb0 tx=0x7f6db0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fddf110e888726 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.494+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 0x7f6dac0a37d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009ad0 tx=0x7f6db80091c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.494+0000 7f6db57fa700 1 -- 172.21.15.26:0/1910664787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db009ef50 con 0x5577bfaaae20 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.495+0000 7f6db57fa700 1 -- 172.21.15.26:0/1910664787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db00a32e0 con 0x5577bfaaae20 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.495+0000 7f6db57fa700 1 -- 172.21.15.26:0/1910664787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00a2e50 con 0x5577bfaaae20 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.495+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/1910664787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 msgr2=0x5577bfaab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.495+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6db009aeb0 tx=0x7f6db0000bc0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/1910664787 shutdown_connections 2023-07-13T22:23:51.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaaa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/1910664787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfaab6e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/1910664787 >> 172.21.15.26:0/1910664787 conn(0x5577bfa9fda0 msgr2=0x5577bfaa51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/1910664787 shutdown_connections 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.496+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/1910664787 wait complete. 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.497+0000 7f6dd8c87b80 1 -- start start 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.497+0000 7f6dd8c87b80 1 Processor -- start 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.498+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.498+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 0x5577bfab4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.498+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfabb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bf99bac0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bf99b7c0 con 0x5577bfaaae20 2023-07-13T22:23:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bf87f920 con 0x5577bfaabc70 2023-07-13T22:23:51.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dd8c87b80 10 client.0 _ll_register_callbacks cb 0x5577bf999760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60568/0 (socket says 172.21.15.26:60568) 2023-07-13T22:23:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dbeffd700 1 -- 172.21.15.26:0/3171036396 learned_addr learned my addr 172.21.15.26:0/3171036396 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:51.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.499+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfabb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6db67fc700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 0x5577bfab4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 -- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 msgr2=0x5577bfabb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfabb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 -- 172.21.15.26:0/3171036396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 msgr2=0x5577bfab4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 0x5577bfab4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 -- 172.21.15.26:0/3171036396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc010f500 con 0x5577bfaaa4d0 2023-07-13T22:23:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6db67fc700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 0x5577bfab4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6da8011690 tx=0x7f6da8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ddb35862dff27e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.500+0000 7f6da67fc700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8009790 con 0x5577bfaaa4d0 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6da8012920 con 0x5577bfaaa4d0 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577bf9554f0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da8012ae0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6dd8c87b80 10 client.16509 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.501+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5577bf812c60 con 0x5577bfaaa4d0 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577bf9554f0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6da801a3f0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 10 client.16509 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 10 client.16509 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 20 client.16509 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfabfc10 0x5577bfac2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 4 client.16509 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.502+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577bf9f4eb0 con 0x5577bfabfc10 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.503+0000 7f6db67fc700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfabfc10 0x5577bfac2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.503+0000 7f6db67fc700 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfabfc10 0x5577bfac2020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8bfa16056f918937 server_cookie=78bc894360c002c in_seq=0 out_seq=0 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.503+0000 7f6da67fc700 10 client.16509 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6da67fc700 1 -- 172.21.15.26:0/3171036396 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577bf9f4eb0 con 0x5577bfabfc10 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6da67fc700 10 client.16509 handle_command_reply: tid=0 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 client.16509 shutdown 2023-07-13T22:23:51.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 msgr2=0x5577bfab3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6da8011690 tx=0x7f6da8011a50 comp rx=0 tx=0).stop 2023-07-13T22:23:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 shutdown_connections 2023-07-13T22:23:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfabfc10 0x5577bfac2020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaa4d0 0x5577bfab3dd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfabb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/3171036396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaae20 0x5577bfab4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 >> 172.21.15.26:0/3171036396 conn(0x5577bfa9fda0 msgr2=0x5577bfaa48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.504+0000 7f6da67fc700 0 client.16509 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.505+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 shutdown_connections 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.505+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/3171036396 wait complete. 2023-07-13T22:23:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.505+0000 7f6dd8c87b80 20 client.16509 trim_cache size 0 max 16384 2023-07-13T22:23:51.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.528+0000 7f6dd8c87b80 1 Processor -- start 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.528+0000 7f6dd8c87b80 1 -- start start 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.528+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.528+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.528+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.529+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfabf910 con 0x5577bfaaae20 2023-07-13T22:23:51.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.529+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfaaa8a0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.529+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfaa4350 con 0x5577bfaabc70 2023-07-13T22:23:51.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.530+0000 7f6dbeffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.530+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.530+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:37326/0 (socket says 172.21.15.26:37326) 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.530+0000 7f6dbf7fe700 1 -- 172.21.15.26:0/642725767 learned_addr learned my addr 172.21.15.26:0/642725767 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6db67fc700 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6dbf7fe700 1 -- 172.21.15.26:0/642725767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 msgr2=0x5577bfaac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6dbf7fe700 1 -- 172.21.15.26:0/642725767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 msgr2=0x5577bfabb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.531+0000 7f6dbf7fe700 1 -- 172.21.15.26:0/642725767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577bf850250 con 0x5577bfaabc70 2023-07-13T22:23:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.532+0000 7f6dbf7fe700 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6db8003eb0 tx=0x7f6db8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3ad709d90fbb55 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.533+0000 7f6db5ffb700 1 -- 172.21.15.26:0/642725767 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db801d070 con 0x5577bfaabc70 2023-07-13T22:23:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.533+0000 7f6db5ffb700 1 -- 172.21.15.26:0/642725767 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db800ee80 con 0x5577bfaabc70 2023-07-13T22:23:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.533+0000 7f6db5ffb700 1 -- 172.21.15.26:0/642725767 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8006670 con 0x5577bfaabc70 2023-07-13T22:23:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.534+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/642725767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 msgr2=0x5577bfaa3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.534+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6db8003eb0 tx=0x7f6db8011000 comp rx=0 tx=0).stop 2023-07-13T22:23:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/642725767 shutdown_connections 2023-07-13T22:23:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaabc70 0x5577bfaa3e10 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaaa4d0 0x5577bfaac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/642725767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/642725767 >> 172.21.15.26:0/642725767 conn(0x5577bfa9fda0 msgr2=0x5577bfac6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/642725767 shutdown_connections 2023-07-13T22:23:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.535+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/642725767 wait complete. 2023-07-13T22:23:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.536+0000 7f6dd8c87b80 1 -- start start 2023-07-13T22:23:51.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.536+0000 7f6dd8c87b80 1 Processor -- start 2023-07-13T22:23:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 0x5577bfabacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 0x5577bfab77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfab7ee0 con 0x5577bfaaae20 2023-07-13T22:23:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfab8050 con 0x5577bfaabc70 2023-07-13T22:23:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577bfab8210 con 0x5577bfaaa4d0 2023-07-13T22:23:51.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.537+0000 7f6dd8c87b80 10 client.0 _ll_register_callbacks cb 0x7f6da8009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.538+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 0x5577bfab77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.538+0000 7f6db67fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.538+0000 7f6dbf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 0x5577bfab77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43794/0 (socket says 172.21.15.26:43794) 2023-07-13T22:23:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.538+0000 7f6dbf7fe700 1 -- 172.21.15.26:0/932471048 learned_addr learned my addr 172.21.15.26:0/932471048 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:51.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.538+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 0x5577bfabacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 -- 172.21.15.26:0/932471048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 msgr2=0x5577bfab77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 0x5577bfab77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 -- 172.21.15.26:0/932471048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 msgr2=0x5577bfabb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 -- 172.21.15.26:0/932471048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db8003b60 con 0x5577bfaaa4d0 2023-07-13T22:23:51.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.539+0000 7f6dbeffd700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 0x5577bfabacf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6da801a8a0 tx=0x7f6da8009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54516c710a0888c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.556+0000 7f6da67fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.556+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8005310 con 0x5577bfaaa4d0 2023-07-13T22:23:51.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.556+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6da801a4f0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.556+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6db00a32e0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.562+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da8005470 con 0x5577bfaaa4d0 2023-07-13T22:23:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.562+0000 7f6dd8c87b80 10 client.27073 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:51.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.562+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5577bf8282b0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.585+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db00a32e0 con 0x5577bfaaa4d0 2023-07-13T22:23:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.585+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6da8037070 con 0x5577bfaaa4d0 2023-07-13T22:23:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 10 client.27073 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 10 client.27073 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:23:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 20 client.27073 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfb150f0 0x5577bfb17500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 4 client.27073 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.586+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577bf9b7050 con 0x5577bfb150f0 2023-07-13T22:23:51.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.594+0000 7f6db67fc700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfb150f0 0x5577bfb17500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.614+0000 7f6db67fc700 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfb150f0 0x5577bfb17500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=922aa24e11633605 server_cookie=1bcfb3cb530b7981 in_seq=0 out_seq=0 2023-07-13T22:23:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.614+0000 7f6da67fc700 10 client.27073 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.614+0000 7f6da67fc700 1 -- 172.21.15.26:0/932471048 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577bf9b7050 con 0x5577bfb150f0 2023-07-13T22:23:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.614+0000 7f6da67fc700 10 client.27073 handle_command_reply: tid=0 2023-07-13T22:23:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.614+0000 7f6dd8c87b80 1 client.27073 shutdown 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 msgr2=0x5577bfabacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 0x5577bfabacf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6da801a8a0 tx=0x7f6da8009f00 comp rx=0 tx=0).stop 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 shutdown_connections 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x5577bfb150f0 0x5577bfb17500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5577bfaabc70 0x5577bfab77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6da67fc700 0 client.27073 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:23:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577bfaaa4d0 0x5577bfabacf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 --2- 172.21.15.26:0/932471048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5577bfaaae20 0x5577bfabb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.615+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 >> 172.21.15.26:0/932471048 conn(0x5577bfa9fda0 msgr2=0x5577bfab2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.616+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 shutdown_connections 2023-07-13T22:23:51.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.616+0000 7f6dd8c87b80 1 -- 172.21.15.26:0/932471048 wait complete. 2023-07-13T22:23:51.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.617+0000 7f6dd8c87b80 20 client.27073 trim_cache size 0 max 16384 2023-07-13T22:23:51.602 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.622+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 msgr2=0x7f6dac0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.622+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 0x7f6dac0a37d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009ad0 tx=0x7f6db80091c0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.622+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 msgr2=0x7f6dc01a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:51.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.622+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6db0005f50 tx=0x7f6db000f460 comp rx=0 tx=0).stop 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 shutdown_connections 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f6dac0a1320 0x7f6dac0a37d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6dc010bd60 0x7f6dc01abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc0070ee0 0x7f6dc01a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 --2- 172.21.15.26:0/578208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dc0071830 0x7f6dc01a4c00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:51.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.623+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 >> 172.21.15.26:0/578208 conn(0x7f6dc006ca90 msgr2=0x7f6dc006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.624+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 shutdown_connections 2023-07-13T22:23:51.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:51.625+0000 7f6dc53ff700 1 -- 172.21.15.26:0/578208 wait complete. 2023-07-13T22:23:51.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:23:51.817 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:51 smithi026 ceph-mon[95647]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 94 op/s 2023-07-13T22:23:51.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:51 smithi117 ceph-mon[129074]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 94 op/s 2023-07-13T22:23:51.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:51 smithi121 ceph-mon[129187]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 296 MiB/s wr, 94 op/s 2023-07-13T22:23:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.193+0000 7f9da47ff700 1 -- 172.21.15.26:0/4284966925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 msgr2=0x7f9d9c0712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.193+0000 7f9da47ff700 1 --2- 172.21.15.26:0/4284966925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c0712f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009990 tx=0x7f9d98009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 -- 172.21.15.26:0/4284966925 shutdown_connections 2023-07-13T22:23:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 --2- 172.21.15.26:0/4284966925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d9c10bd60 0x7f9d9c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 --2- 172.21.15.26:0/4284966925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c0712f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 --2- 172.21.15.26:0/4284966925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c071830 0x7f9d9c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 -- 172.21.15.26:0/4284966925 >> 172.21.15.26:0/4284966925 conn(0x7f9d9c06ca90 msgr2=0x7f9d9c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.195+0000 7f9da47ff700 1 -- 172.21.15.26:0/4284966925 shutdown_connections 2023-07-13T22:23:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.196+0000 7f9da47ff700 1 -- 172.21.15.26:0/4284966925 wait complete. 2023-07-13T22:23:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.197+0000 7f9da47ff700 1 Processor -- start 2023-07-13T22:23:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 -- start start 2023-07-13T22:23:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d9c071830 0x7f9d9c113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c10bd60 0x7f9d9c1b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c117080 con 0x7f9d9c10bd60 2023-07-13T22:23:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c1171f0 con 0x7f9d9c070ee0 2023-07-13T22:23:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.198+0000 7f9da47ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c1b4e70 con 0x7f9d9c071830 2023-07-13T22:23:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.199+0000 7f9da259b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.199+0000 7f9da259b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43814/0 (socket says 172.21.15.26:43814) 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.199+0000 7f9da259b700 1 -- 172.21.15.26:0/1940678300 learned_addr learned my addr 172.21.15.26:0/1940678300 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.200+0000 7f9da259b700 1 -- 172.21.15.26:0/1940678300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d9c071830 msgr2=0x7f9d9c113fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.200+0000 7f9da259b700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d9c071830 0x7f9d9c113fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.200+0000 7f9da259b700 1 -- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c10bd60 msgr2=0x7f9d9c1b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.200+0000 7f9da259b700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c10bd60 0x7f9d9c1b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.200+0000 7f9da259b700 1 -- 172.21.15.26:0/1940678300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980095f0 con 0x7f9d9c070ee0 2023-07-13T22:23:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.202+0000 7f9da259b700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005950 tx=0x7f9d9800bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.213+0000 7f9d937fe700 1 -- 172.21.15.26:0/1940678300 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801d070 con 0x7f9d9c070ee0 2023-07-13T22:23:52.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.214+0000 7f9d937fe700 1 -- 172.21.15.26:0/1940678300 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9800f460 con 0x7f9d9c070ee0 2023-07-13T22:23:52.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.214+0000 7f9d937fe700 1 -- 172.21.15.26:0/1940678300 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98021670 con 0x7f9d9c070ee0 2023-07-13T22:23:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.214+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c1b5010 con 0x7f9d9c070ee0 2023-07-13T22:23:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.214+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c1b5460 con 0x7f9d9c070ee0 2023-07-13T22:23:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.221+0000 7f9db81cdb80 1 Processor -- start 2023-07-13T22:23:52.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.222+0000 7f9db81cdb80 1 -- start start 2023-07-13T22:23:52.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.222+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51ba08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.223+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.223+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.223+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51aa5cb0 con 0x555b51ba0e20 2023-07-13T22:23:52.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.223+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51af8a40 con 0x555b51ba04d0 2023-07-13T22:23:52.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.223+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51af1ed0 con 0x555b51ba1c70 2023-07-13T22:23:52.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.248+0000 7f9da2d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51ba08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.248+0000 7f9da259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.248+0000 7f9da259b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60600/0 (socket says 172.21.15.26:60600) 2023-07-13T22:23:52.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.248+0000 7f9da259b700 1 -- 172.21.15.26:0/28930862 learned_addr learned my addr 172.21.15.26:0/28930862 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:52.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.253+0000 7f9d937fe700 1 -- 172.21.15.26:0/1940678300 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9d9800f8b0 con 0x7f9d9c070ee0 2023-07-13T22:23:52.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.255+0000 7f9d937fe700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 0x7f9d880a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.255+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.256+0000 7f9d937fe700 1 -- 172.21.15.26:0/1940678300 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d980d4840 con 0x7f9d9c070ee0 2023-07-13T22:23:52.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.256+0000 7f9da259b700 1 -- 172.21.15.26:0/28930862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 msgr2=0x555b51ba20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.256+0000 7f9da259b700 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.256+0000 7f9da259b700 1 -- 172.21.15.26:0/28930862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 msgr2=0x555b51ba08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.256+0000 7f9da259b700 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51ba08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.257+0000 7f9da259b700 1 -- 172.21.15.26:0/28930862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b5191fde0 con 0x555b51ba0e20 2023-07-13T22:23:52.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.257+0000 7f9da259b700 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d980d22f0 tx=0x7f9d9800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d30e374f49e12d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.258+0000 7f9d90ff9700 1 -- 172.21.15.26:0/28930862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9809deb0 con 0x555b51ba0e20 2023-07-13T22:23:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.258+0000 7f9d90ff9700 1 -- 172.21.15.26:0/28930862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d980a19d0 con 0x555b51ba0e20 2023-07-13T22:23:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.258+0000 7f9d90ff9700 1 -- 172.21.15.26:0/28930862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980a1350 con 0x555b51ba0e20 2023-07-13T22:23:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.259+0000 7f9db81cdb80 1 -- 172.21.15.26:0/28930862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 msgr2=0x555b51ba16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.259+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d980d22f0 tx=0x7f9d9800fc50 comp rx=0 tx=0).stop 2023-07-13T22:23:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.260+0000 7f9db81cdb80 1 -- 172.21.15.26:0/28930862 shutdown_connections 2023-07-13T22:23:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.260+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.260+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51ba08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.260+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/28930862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51ba16e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.260+0000 7f9db81cdb80 1 -- 172.21.15.26:0/28930862 >> 172.21.15.26:0/28930862 conn(0x555b51b95da0 msgr2=0x555b51b9b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.268+0000 7f9da2d9c700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 0x7f9d880a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.273+0000 7f9da2d9c700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 0x7f9d880a3800 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9d940093d0 tx=0x7f9d94004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.355+0000 7f9db81cdb80 1 -- 172.21.15.26:0/28930862 shutdown_connections 2023-07-13T22:23:52.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.355+0000 7f9db81cdb80 1 -- 172.21.15.26:0/28930862 wait complete. 2023-07-13T22:23:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.356+0000 7f9db81cdb80 1 -- start start 2023-07-13T22:23:52.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.356+0000 7f9db81cdb80 1 Processor -- start 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba04d0 0x555b51ba9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba1c70 0x555b51bb12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51a91ac0 con 0x555b51ba04d0 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51a917c0 con 0x555b51ba1c70 2023-07-13T22:23:52.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.357+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51975920 con 0x555b51ba0e20 2023-07-13T22:23:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.358+0000 7f9db81cdb80 10 client.0 _ll_register_callbacks cb 0x555b51a8f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.359+0000 7f9da2d9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba1c70 0x555b51bb12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.359+0000 7f9da1d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.359+0000 7f9da1d9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:37372/0 (socket says 172.21.15.26:37372) 2023-07-13T22:23:52.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.359+0000 7f9da1d9a700 1 -- 172.21.15.26:0/1175284994 learned_addr learned my addr 172.21.15.26:0/1175284994 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.363+0000 7f9da1d9a700 1 -- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba1c70 msgr2=0x555b51bb12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.363+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba1c70 0x555b51bb12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.363+0000 7f9da1d9a700 1 -- 172.21.15.26:0/1175284994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba04d0 msgr2=0x555b51ba9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.363+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba04d0 0x555b51ba9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.363+0000 7f9da1d9a700 1 -- 172.21.15.26:0/1175284994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c1b5460 con 0x555b51ba0e20 2023-07-13T22:23:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.364+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e480 tx=0x7f9d8c00e790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df75f062721d59ad server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.371+0000 7f9d85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.371+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c013470 con 0x555b51ba0e20 2023-07-13T22:23:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.371+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d8c0135d0 con 0x555b51ba0e20 2023-07-13T22:23:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.372+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b51a4b4f0 con 0x555b51ba0e20 2023-07-13T22:23:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.390+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c013790 con 0x555b51ba0e20 2023-07-13T22:23:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.391+0000 7f9db81cdb80 10 client.27082 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.391+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555b51908c60 con 0x555b51ba0e20 2023-07-13T22:23:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b51a4b4f0 con 0x555b51ba0e20 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d8c011070 con 0x555b51ba0e20 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 10 client.27082 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 10 client.27082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 20 client.27082 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51bb5c00 0x555b51bb8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 4 client.27082 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.410+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b51aeaeb0 con 0x555b51bb5c00 2023-07-13T22:23:52.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.422+0000 7f9da259b700 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51bb5c00 0x555b51bb8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.429+0000 7f9da259b700 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51bb5c00 0x555b51bb8010 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8969a30846f890ce server_cookie=808cd242b84f1e20 in_seq=0 out_seq=0 2023-07-13T22:23:52.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.429+0000 7f9d85ffb700 10 client.27082 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.441+0000 7f9d85ffb700 1 -- 172.21.15.26:0/1175284994 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b51aeaeb0 con 0x555b51bb5c00 2023-07-13T22:23:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.441+0000 7f9d85ffb700 10 client.27082 handle_command_reply: tid=0 2023-07-13T22:23:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.441+0000 7f9db81cdb80 1 client.27082 shutdown 2023-07-13T22:23:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 msgr2=0x555b51baa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e480 tx=0x7f9d8c00e790 comp rx=0 tx=0).stop 2023-07-13T22:23:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 shutdown_connections 2023-07-13T22:23:52.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba04d0 0x555b51ba9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51bb5c00 0x555b51bb8010 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba1c70 0x555b51bb12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1175284994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba0e20 0x555b51baa2b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 >> 172.21.15.26:0/1175284994 conn(0x555b51b95da0 msgr2=0x555b51b9a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.442+0000 7f9d85ffb700 0 client.27082 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.443+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 shutdown_connections 2023-07-13T22:23:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.443+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1175284994 wait complete. 2023-07-13T22:23:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.443+0000 7f9db81cdb80 20 client.27082 trim_cache size 0 max 16384 2023-07-13T22:23:52.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 Processor -- start 2023-07-13T22:23:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 -- start start 2023-07-13T22:23:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51bb1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51ba9220 con 0x555b51ba0e20 2023-07-13T22:23:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51b92ef0 con 0x555b51ba04d0 2023-07-13T22:23:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.546+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51ba8680 con 0x555b51ba1c70 2023-07-13T22:23:52.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.547+0000 7f9da259b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.547+0000 7f9da259b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43840/0 (socket says 172.21.15.26:43840) 2023-07-13T22:23:52.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.547+0000 7f9da259b700 1 -- 172.21.15.26:0/1635355388 learned_addr learned my addr 172.21.15.26:0/1635355388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:52.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.550+0000 7f9da259b700 1 -- 172.21.15.26:0/1635355388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 msgr2=0x555b51ba4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.550+0000 7f9da259b700 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.550+0000 7f9da259b700 1 -- 172.21.15.26:0/1635355388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 msgr2=0x555b51bb1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.550+0000 7f9da259b700 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51bb1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.550+0000 7f9da259b700 1 -- 172.21.15.26:0/1635355388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b51946250 con 0x555b51ba04d0 2023-07-13T22:23:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.551+0000 7f9da259b700 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800fc00 tx=0x7f9d980a01b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41baa376819a2dc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.551+0000 7f9d917fa700 1 -- 172.21.15.26:0/1635355388 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9809fde0 con 0x555b51ba04d0 2023-07-13T22:23:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.551+0000 7f9d917fa700 1 -- 172.21.15.26:0/1635355388 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d980a1740 con 0x555b51ba04d0 2023-07-13T22:23:52.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.551+0000 7f9d917fa700 1 -- 172.21.15.26:0/1635355388 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980ba260 con 0x555b51ba04d0 2023-07-13T22:23:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.552+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1635355388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 msgr2=0x555b51bb7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.552+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800fc00 tx=0x7f9d980a01b0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1635355388 shutdown_connections 2023-07-13T22:23:52.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba1c70 0x555b51ba4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba04d0 0x555b51bb7c10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/1635355388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba0e20 0x555b51bb1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1635355388 >> 172.21.15.26:0/1635355388 conn(0x555b51b95da0 msgr2=0x555b51ba7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1635355388 shutdown_connections 2023-07-13T22:23:52.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.553+0000 7f9db81cdb80 1 -- 172.21.15.26:0/1635355388 wait complete. 2023-07-13T22:23:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.554+0000 7f9db81cdb80 1 -- start start 2023-07-13T22:23:52.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.554+0000 7f9db81cdb80 1 Processor -- start 2023-07-13T22:23:52.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba04d0 0x555b51bad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba1c70 0x555b51bb06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51bb0c20 con 0x555b51ba1c70 2023-07-13T22:23:52.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51bb0d90 con 0x555b51ba0e20 2023-07-13T22:23:52.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b51bb0f00 con 0x555b51ba04d0 2023-07-13T22:23:52.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.555+0000 7f9db81cdb80 10 client.0 _ll_register_callbacks cb 0x7f9d8c0099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:52.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.556+0000 7f9da259b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba04d0 0x555b51bad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.556+0000 7f9da1d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.556+0000 7f9da1d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:43850/0 (socket says 172.21.15.26:43850) 2023-07-13T22:23:52.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.556+0000 7f9da1d9a700 1 -- 172.21.15.26:0/637953739 learned_addr learned my addr 172.21.15.26:0/637953739 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:52.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.557+0000 7f9da1d9a700 1 -- 172.21.15.26:0/637953739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba04d0 msgr2=0x555b51bad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.557+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba04d0 0x555b51bad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.557+0000 7f9da1d9a700 1 -- 172.21.15.26:0/637953739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba1c70 msgr2=0x555b51bb06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:52.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.557+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba1c70 0x555b51bb06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.557+0000 7f9da1d9a700 1 -- 172.21.15.26:0/637953739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b5191fde0 con 0x555b51ba0e20 2023-07-13T22:23:52.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.558+0000 7f9da1d9a700 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e450 tx=0x7f9d8c01c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2836b7eb1fef177a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:52.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.558+0000 7f9d85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:52.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.558+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005e60 con 0x555b51ba0e20 2023-07-13T22:23:52.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.558+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d8c013470 con 0x555b51ba0e20 2023-07-13T22:23:52.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.559+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d980a19d0 con 0x555b51ba0e20 2023-07-13T22:23:52.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.560+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d8c009280 con 0x555b51ba0e20 2023-07-13T22:23:52.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.560+0000 7f9db81cdb80 10 client.27092 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:52.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.560+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555b5191e2b0 con 0x555b51ba0e20 2023-07-13T22:23:52.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.562+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d980a19d0 con 0x555b51ba0e20 2023-07-13T22:23:52.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.562+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9d8c011070 con 0x555b51ba0e20 2023-07-13T22:23:52.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 10 client.27092 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 10 client.27092 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 20 client.27092 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51c0abd0 0x555b51c0cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 4 client.27092 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.563+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555b51aad050 con 0x555b51c0abd0 2023-07-13T22:23:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.564+0000 7f9da259b700 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51c0abd0 0x555b51c0cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.570+0000 7f9da259b700 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51c0abd0 0x555b51c0cfe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2956f0ecf06b9936 server_cookie=85d159f2cf8d6c67 in_seq=0 out_seq=0 2023-07-13T22:23:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.570+0000 7f9d85ffb700 10 client.27092 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.572+0000 7f9d85ffb700 1 -- 172.21.15.26:0/637953739 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555b51aad050 con 0x555b51c0abd0 2023-07-13T22:23:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.572+0000 7f9d85ffb700 10 client.27092 handle_command_reply: tid=0 2023-07-13T22:23:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.572+0000 7f9db81cdb80 1 client.27092 shutdown 2023-07-13T22:23:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.573+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 msgr2=0x555b51bad9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.573+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e450 tx=0x7f9d8c01c9e0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 shutdown_connections 2023-07-13T22:23:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555b51ba1c70 0x555b51bb06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x555b51c0abd0 0x555b51c0cfe0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b51ba04d0 0x555b51bad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 --2- 172.21.15.26:0/637953739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b51ba0e20 0x555b51bad9e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.576+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 >> 172.21.15.26:0/637953739 conn(0x555b51b95da0 msgr2=0x555b51bb7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.577+0000 7f9d85ffb700 0 client.27092 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:23:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.577+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 shutdown_connections 2023-07-13T22:23:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.577+0000 7f9db81cdb80 1 -- 172.21.15.26:0/637953739 wait complete. 2023-07-13T22:23:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.578+0000 7f9db81cdb80 20 client.27092 trim_cache size 0 max 16384 2023-07-13T22:23:52.561 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.579+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 msgr2=0x7f9d880a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.579+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 0x7f9d880a3800 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9d940093d0 tx=0x7f9d94004ad0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.579+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 msgr2=0x7f9d9c113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.579+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005950 tx=0x7f9d9800bbd0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 shutdown_connections 2023-07-13T22:23:52.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f9d880a1350 0x7f9d880a3800 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9c10bd60 0x7f9d9c1b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d9c070ee0 0x7f9d9c113a60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 --2- 172.21.15.26:0/1940678300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d9c071830 0x7f9d9c113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:52.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.582+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 >> 172.21.15.26:0/1940678300 conn(0x7f9d9c06ca90 msgr2=0x7f9d9c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:52.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.597+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 shutdown_connections 2023-07-13T22:23:52.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:52.598+0000 7f9da47ff700 1 -- 172.21.15.26:0/1940678300 wait complete. 2023-07-13T22:23:52.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.409+0000 7f06bc3ff700 1 -- 172.21.15.26:0/3175651175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b4103670 msgr2=0x7f06b4103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.409+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/3175651175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b4103670 0x7f06b4103ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f06ac00b0a0 tx=0x7f06ac00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.410+0000 7f06bc3ff700 1 -- 172.21.15.26:0/3175651175 shutdown_connections 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.410+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/3175651175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b4103670 0x7f06b4103ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.410+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/3175651175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b40fe350 0x7f06b4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.410+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/3175651175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b4103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.410+0000 7f06bc3ff700 1 -- 172.21.15.26:0/3175651175 >> 172.21.15.26:0/3175651175 conn(0x7f06b40fa000 msgr2=0x7f06b40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.411+0000 7f06bc3ff700 1 -- 172.21.15.26:0/3175651175 shutdown_connections 2023-07-13T22:23:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.411+0000 7f06bc3ff700 1 -- 172.21.15.26:0/3175651175 wait complete. 2023-07-13T22:23:53.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.413+0000 7f06bc3ff700 1 Processor -- start 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.413+0000 7f06bc3ff700 1 -- start start 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b40fe350 0x7f06b410ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4103670 0x7f06b410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b410e560 con 0x7f06b4100cb0 2023-07-13T22:23:53.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b410e6d0 con 0x7f06b4103670 2023-07-13T22:23:53.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.414+0000 7f06bc3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b410e890 con 0x7f06b40fe350 2023-07-13T22:23:53.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.415+0000 7f06b999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.415+0000 7f06b999a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54612/0 (socket says 172.21.15.26:54612) 2023-07-13T22:23:53.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.415+0000 7f06b999a700 1 -- 172.21.15.26:0/2523777970 learned_addr learned my addr 172.21.15.26:0/2523777970 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:53.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 -- 172.21.15.26:0/2523777970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b40fe350 msgr2=0x7f06b410ca10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:23:53.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b40fe350 0x7f06b410ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 -- 172.21.15.26:0/2523777970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4103670 msgr2=0x7f06b410df90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:23:53.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4103670 0x7f06b410df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 -- 172.21.15.26:0/2523777970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac009d00 con 0x7f06b4100cb0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.416+0000 7f06b999a700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b530 tx=0x7f06b000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.417+0000 7f06ab7fe700 1 -- 172.21.15.26:0/2523777970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0011830 con 0x7f06b4100cb0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.417+0000 7f06ab7fe700 1 -- 172.21.15.26:0/2523777970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06b0011e50 con 0x7f06b4100cb0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.417+0000 7f06ab7fe700 1 -- 172.21.15.26:0/2523777970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000f2e0 con 0x7f06b4100cb0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.417+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b410eb10 con 0x7f06b4100cb0 2023-07-13T22:23:53.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.417+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41acf80 con 0x7f06b4100cb0 2023-07-13T22:23:53.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.425+0000 7f06ab7fe700 1 -- 172.21.15.26:0/2523777970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f06b0011990 con 0x7f06b4100cb0 2023-07-13T22:23:53.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.426+0000 7f06cfde8b80 1 Processor -- start 2023-07-13T22:23:53.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.428+0000 7f06ab7fe700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 0x7f06a00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.428+0000 7f06ab7fe700 1 -- 172.21.15.26:0/2523777970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f06b00cc2f0 con 0x7f06b4100cb0 2023-07-13T22:23:53.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.429+0000 7f06cfde8b80 1 -- start start 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.429+0000 7f06ba99c700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 0x7f06a00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb2a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2ae20 0x560b6bb2b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6ba2fcb0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6ba82a40 con 0x560b6bb2ae20 2023-07-13T22:23:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6ba7bed0 con 0x560b6bb2bc70 2023-07-13T22:23:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06ba99c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06ba19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb2a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06ba99c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb2c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41308/0 (socket says 172.21.15.26:41308) 2023-07-13T22:23:53.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.430+0000 7f06ba99c700 1 -- 172.21.15.26:0/643190731 learned_addr learned my addr 172.21.15.26:0/643190731 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.431+0000 7f06ba19b700 1 -- 172.21.15.26:0/643190731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 msgr2=0x560b6bb2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.431+0000 7f06ba19b700 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.431+0000 7f06ba19b700 1 -- 172.21.15.26:0/643190731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2ae20 msgr2=0x560b6bb2b6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.431+0000 7f06ba19b700 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2ae20 0x560b6bb2b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.431+0000 7f06ba19b700 1 -- 172.21.15.26:0/643190731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6b8a9de0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.432+0000 7f06ba19b700 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb2a8e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f06a400ba30 tx=0x7f06a400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cd285d7cf9436f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.432+0000 7f06ba99c700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 0x7f06a00a3800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006070 tx=0x7f06ac0093d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.432+0000 7f06a8bfe700 1 -- 172.21.15.26:0/643190731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4004a30 con 0x560b6bb2a4d0 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.432+0000 7f06a8bfe700 1 -- 172.21.15.26:0/643190731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06a400dc90 con 0x560b6bb2a4d0 2023-07-13T22:23:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.432+0000 7f06a8bfe700 1 -- 172.21.15.26:0/643190731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a40053e0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 -- 172.21.15.26:0/643190731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 msgr2=0x560b6bb2a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb2a8e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f06a400ba30 tx=0x7f06a400bd40 comp rx=0 tx=0).stop 2023-07-13T22:23:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 -- 172.21.15.26:0/643190731 shutdown_connections 2023-07-13T22:23:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb2a8e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/643190731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2ae20 0x560b6bb2b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.433+0000 7f06cfde8b80 1 -- 172.21.15.26:0/643190731 >> 172.21.15.26:0/643190731 conn(0x560b6bb1fda0 msgr2=0x560b6bb251d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.434+0000 7f06cfde8b80 1 -- 172.21.15.26:0/643190731 shutdown_connections 2023-07-13T22:23:53.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.434+0000 7f06cfde8b80 1 -- 172.21.15.26:0/643190731 wait complete. 2023-07-13T22:23:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.435+0000 7f06cfde8b80 1 -- start start 2023-07-13T22:23:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.435+0000 7f06cfde8b80 1 Processor -- start 2023-07-13T22:23:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2bc70 0x560b6bb34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb39830 0x560b6bb3bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6ba1bac0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6ba1b7c0 con 0x560b6bb2bc70 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6b8ff920 con 0x560b6bb39830 2023-07-13T22:23:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.436+0000 7f06cfde8b80 10 client.0 _ll_register_callbacks cb 0x560b6ba19760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.437+0000 7f06ba19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.437+0000 7f06ba19b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54622/0 (socket says 172.21.15.26:54622) 2023-07-13T22:23:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.437+0000 7f06ba19b700 1 -- 172.21.15.26:0/1336086215 learned_addr learned my addr 172.21.15.26:0/1336086215 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 -- 172.21.15.26:0/1336086215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb39830 msgr2=0x560b6bb3bc10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:23:53.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb39830 0x560b6bb3bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 -- 172.21.15.26:0/1336086215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2bc70 msgr2=0x560b6bb34800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2bc70 0x560b6bb34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 -- 172.21.15.26:0/1336086215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a400b690 con 0x560b6bb2a4d0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f06ba19b700 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004ce0 tx=0x7f06a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bf25faf9b2c04a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.438+0000 7f069dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.439+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4005320 con 0x560b6bb2a4d0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.439+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f06a4005480 con 0x560b6bb2a4d0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.439+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b6b9d54f0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.440+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a4005640 con 0x560b6bb2a4d0 2023-07-13T22:23:53.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.440+0000 7f06cfde8b80 10 client.16527 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:53.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.440+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b6b892c60 con 0x560b6bb2a4d0 2023-07-13T22:23:53.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.440+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b6b9d54f0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.441+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f06a402f7f0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 10 client.16527 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 10 client.16527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 20 client.16527 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb40340 0x560b6bb427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 4 client.16527 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:53.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.442+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b6ba74eb0 con 0x560b6bb40340 2023-07-13T22:23:53.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.444+0000 7f06b999a700 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb40340 0x560b6bb427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.446+0000 7f06b999a700 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb40340 0x560b6bb427d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3a7895f09bc0785 server_cookie=46da4114aa659cb3 in_seq=0 out_seq=0 2023-07-13T22:23:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.446+0000 7f069dffb700 10 client.16527 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:53.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.447+0000 7f069dffb700 1 -- 172.21.15.26:0/1336086215 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b6ba74eb0 con 0x560b6bb40340 2023-07-13T22:23:53.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.447+0000 7f069dffb700 10 client.16527 handle_command_reply: tid=0 2023-07-13T22:23:53.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.447+0000 7f06cfde8b80 1 client.16527 shutdown 2023-07-13T22:23:53.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 msgr2=0x560b6bb342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004ce0 tx=0x7f06a4005f90 comp rx=0 tx=0).stop 2023-07-13T22:23:53.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 shutdown_connections 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb40340 0x560b6bb427d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb342c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb39830 0x560b6bb3bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/1336086215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb2bc70 0x560b6bb34800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 >> 172.21.15.26:0/1336086215 conn(0x560b6bb1fda0 msgr2=0x560b6bb248b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.448+0000 7f069dffb700 0 client.16527 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:53.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.449+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 shutdown_connections 2023-07-13T22:23:53.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.449+0000 7f06cfde8b80 1 -- 172.21.15.26:0/1336086215 wait complete. 2023-07-13T22:23:53.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.449+0000 7f06cfde8b80 20 client.16527 trim_cache size 0 max 16384 2023-07-13T22:23:53.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.509+0000 7f06cfde8b80 1 Processor -- start 2023-07-13T22:23:53.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 -- start start 2023-07-13T22:23:53.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb2b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb351c0 con 0x560b6bb2a4d0 2023-07-13T22:23:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb25f20 con 0x560b6bb39830 2023-07-13T22:23:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.510+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb40120 con 0x560b6bb2bc70 2023-07-13T22:23:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.517+0000 7f06ba99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb2b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.517+0000 7f06ba99c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb2b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56134/0 (socket says 172.21.15.26:56134) 2023-07-13T22:23:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.517+0000 7f06ba99c700 1 -- 172.21.15.26:0/3971968494 learned_addr learned my addr 172.21.15.26:0/3971968494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:53.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.531+0000 7f06ba19b700 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.538+0000 7f06b999a700 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.541+0000 7f06b999a700 1 -- 172.21.15.26:0/3971968494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 msgr2=0x560b6bb2b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.541+0000 7f06b999a700 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb2b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.541+0000 7f06b999a700 1 -- 172.21.15.26:0/3971968494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 msgr2=0x560b6bb50090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.541+0000 7f06b999a700 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb50090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.541+0000 7f06b999a700 1 -- 172.21.15.26:0/3971968494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b6b8d0250 con 0x560b6bb2bc70 2023-07-13T22:23:53.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.543+0000 7f06b999a700 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb321c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x560b6bb2b240 tx=0x7f06b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76090cf5c041626e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.546+0000 7f06a93ff700 1 -- 172.21.15.26:0/3971968494 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b009d770 con 0x560b6bb2bc70 2023-07-13T22:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.546+0000 7f06a93ff700 1 -- 172.21.15.26:0/3971968494 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06b00a1b90 con 0x560b6bb2bc70 2023-07-13T22:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.546+0000 7f06a93ff700 1 -- 172.21.15.26:0/3971968494 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0010610 con 0x560b6bb2bc70 2023-07-13T22:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.548+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3971968494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 msgr2=0x560b6bb321c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.548+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb321c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x560b6bb2b240 tx=0x7f06b0000bc0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.559+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3971968494 shutdown_connections 2023-07-13T22:23:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.559+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2a4d0 0x560b6bb50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.559+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb2b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.559+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3971968494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2bc70 0x560b6bb321c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.559+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3971968494 >> 172.21.15.26:0/3971968494 conn(0x560b6bb1fda0 msgr2=0x560b6bb3c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.569+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3971968494 shutdown_connections 2023-07-13T22:23:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.577+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3971968494 wait complete. 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.578+0000 7f06cfde8b80 1 -- start start 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.578+0000 7f06cfde8b80 1 Processor -- start 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.589+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 0x560b6bb4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.590+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 0x560b6bb4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.590+0000 7f06cfde8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb4f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.590+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb4fb80 con 0x560b6bb2bc70 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.590+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb4fcf0 con 0x560b6bb39830 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.590+0000 7f06cfde8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6bb4fe60 con 0x560b6bb2a4d0 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.591+0000 7f06cfde8b80 10 client.0 _ll_register_callbacks cb 0x7f06ac00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.592+0000 7f06ba19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 0x560b6bb4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.593+0000 7f06ba19b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 0x560b6bb4c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41336/0 (socket says 172.21.15.26:41336) 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.593+0000 7f06ba19b700 1 -- 172.21.15.26:0/3414771809 learned_addr learned my addr 172.21.15.26:0/3414771809 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 0x560b6bb4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06ba99c700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb4f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 -- 172.21.15.26:0/3414771809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 msgr2=0x560b6bb4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 0x560b6bb4c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 -- 172.21.15.26:0/3414771809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 msgr2=0x560b6bb4f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb4f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.594+0000 7f06b999a700 1 -- 172.21.15.26:0/3414771809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41acf80 con 0x560b6bb2bc70 2023-07-13T22:23:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.595+0000 7f06b999a700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 0x560b6bb4c800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x560b6bb3cd10 tx=0x7f06b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c90c22e5557a24 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.595+0000 7f069dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.622+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b009d770 con 0x560b6bb2bc70 2023-07-13T22:23:53.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.622+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f06b00c9e40 con 0x560b6bb2bc70 2023-07-13T22:23:53.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.622+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06a400dc90 con 0x560b6bb2bc70 2023-07-13T22:23:53.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.642+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b00ca000 con 0x560b6bb2bc70 2023-07-13T22:23:53.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.642+0000 7f06cfde8b80 10 client.16533 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:53.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.642+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560b6b8a82b0 con 0x560b6bb2bc70 2023-07-13T22:23:53.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.643+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06a400dc90 con 0x560b6bb2bc70 2023-07-13T22:23:53.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.643+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f06b00c1070 con 0x560b6bb2bc70 2023-07-13T22:23:53.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.644+0000 7f06cfde8b80 10 client.16533 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:53.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.644+0000 7f06cfde8b80 10 client.16533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:23:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.644+0000 7f06cfde8b80 20 client.16533 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.644+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb90f20 0x560b6bb93330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.645+0000 7f06cfde8b80 4 client.16533 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.645+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560b6ba37050 con 0x560b6bb90f20 2023-07-13T22:23:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.645+0000 7f06ba19b700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb90f20 0x560b6bb93330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:53.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f06ba19b700 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb90f20 0x560b6bb93330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f63237d0dd17e416 server_cookie=1f5b2699351a16b0 in_seq=0 out_seq=0 2023-07-13T22:23:53.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f069dffb700 10 client.16533 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:53.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f069dffb700 1 -- 172.21.15.26:0/3414771809 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560b6ba37050 con 0x560b6bb90f20 2023-07-13T22:23:53.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f069dffb700 10 client.16533 handle_command_reply: tid=0 2023-07-13T22:23:53.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f06cfde8b80 1 client.16533 shutdown 2023-07-13T22:23:53.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 msgr2=0x560b6bb4c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.647+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 0x560b6bb4c800 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x560b6bb3cd10 tx=0x7f06b0004750 comp rx=0 tx=0).stop 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 shutdown_connections 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560b6bb39830 0x560b6bb4f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560b6bb2a4d0 0x560b6bb4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x560b6bb90f20 0x560b6bb93330 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 --2- 172.21.15.26:0/3414771809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560b6bb2bc70 0x560b6bb4c800 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 >> 172.21.15.26:0/3414771809 conn(0x560b6bb1fda0 msgr2=0x560b6bb321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f069dffb700 0 client.16533 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:23:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 shutdown_connections 2023-07-13T22:23:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.648+0000 7f06cfde8b80 1 -- 172.21.15.26:0/3414771809 wait complete. 2023-07-13T22:23:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.649+0000 7f06cfde8b80 20 client.16533 trim_cache size 0 max 16384 2023-07-13T22:23:53.633 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.674+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 msgr2=0x7f06a00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.674+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 0x7f06a00a3800 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006070 tx=0x7f06ac0093d0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.674+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 msgr2=0x7f06b410cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.674+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f06b000b530 tx=0x7f06b000b8f0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 shutdown_connections 2023-07-13T22:23:53.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06b4103670 0x7f06b410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06b40fe350 0x7f06b410ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f06a00a1350 0x7f06a00a3800 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 --2- 172.21.15.26:0/2523777970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06b4100cb0 0x7f06b410cf50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:53.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 >> 172.21.15.26:0/2523777970 conn(0x7f06b40fa000 msgr2=0x7f06b40fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:53.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 shutdown_connections 2023-07-13T22:23:53.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:53.717+0000 7f06bc3ff700 1 -- 172.21.15.26:0/2523777970 wait complete. 2023-07-13T22:23:53.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:23:53.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:53 smithi117 ceph-mon[129074]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 82 op/s 2023-07-13T22:23:53.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:53 smithi026 ceph-mon[95647]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 82 op/s 2023-07-13T22:23:53.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:53 smithi121 ceph-mon[129187]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 252 MiB/s wr, 82 op/s 2023-07-13T22:23:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.528+0000 7f03c93ff700 1 -- 172.21.15.26:0/1714122631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 msgr2=0x7f03c40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.528+0000 7f03c93ff700 1 --2- 172.21.15.26:0/1714122631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c40712f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f03b4009990 tx=0x7f03b4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 -- 172.21.15.26:0/1714122631 shutdown_connections 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 --2- 172.21.15.26:0/1714122631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c410bd60 0x7f03c410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 --2- 172.21.15.26:0/1714122631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c40712f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 --2- 172.21.15.26:0/1714122631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c4071830 0x7f03c410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 -- 172.21.15.26:0/1714122631 >> 172.21.15.26:0/1714122631 conn(0x7f03c406ca90 msgr2=0x7f03c406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 -- 172.21.15.26:0/1714122631 shutdown_connections 2023-07-13T22:23:54.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.529+0000 7f03c93ff700 1 -- 172.21.15.26:0/1714122631 wait complete. 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.531+0000 7f03c93ff700 1 Processor -- start 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.531+0000 7f03c93ff700 1 -- start start 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.531+0000 7f03c93ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c41a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.532+0000 7f03c93ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 0x7f03c41a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.532+0000 7f03c93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 0x7f03c41abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.532+0000 7f03c93ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41ac1d0 con 0x7f03c4071830 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.532+0000 7f03c93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c410ec40 con 0x7f03c410bd60 2023-07-13T22:23:54.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.532+0000 7f03c93ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c410edd0 con 0x7f03c4070ee0 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.534+0000 7f03c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c41a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.534+0000 7f03c37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 0x7f03c41abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.534+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 0x7f03c41a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.534+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 0x7f03c41a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54666/0 (socket says 172.21.15.26:54666) 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.534+0000 7f03c27fc700 1 -- 172.21.15.26:0/2936345472 learned_addr learned my addr 172.21.15.26:0/2936345472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.535+0000 7f03c37fe700 1 -- 172.21.15.26:0/2936345472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 msgr2=0x7f03c41a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.535+0000 7f03c37fe700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c41a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.535+0000 7f03c37fe700 1 -- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 msgr2=0x7f03c41a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.535+0000 7f03c37fe700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 0x7f03c41a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.535+0000 7f03c37fe700 1 -- 172.21.15.26:0/2936345472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b40095f0 con 0x7f03c410bd60 2023-07-13T22:23:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.542+0000 7f03c37fe700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 0x7f03c41abc00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03bc007f10 tx=0x7f03bc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.546+0000 7f03abfff700 1 -- 172.21.15.26:0/2936345472 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc004db0 con 0x7f03c410bd60 2023-07-13T22:23:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.546+0000 7f03abfff700 1 -- 172.21.15.26:0/2936345472 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03bc00f040 con 0x7f03c410bd60 2023-07-13T22:23:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.546+0000 7f03abfff700 1 -- 172.21.15.26:0/2936345472 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0058b0 con 0x7f03c410bd60 2023-07-13T22:23:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.546+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c410f0b0 con 0x7f03c410bd60 2023-07-13T22:23:54.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.546+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c410f570 con 0x7f03c410bd60 2023-07-13T22:23:54.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.556+0000 7f03dcd68b80 1 Processor -- start 2023-07-13T22:23:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 -- start start 2023-07-13T22:23:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 0x55685191d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55685191ec70 0x55685191f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556851822cb0 con 0x55685191ec70 2023-07-13T22:23:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x556851875a40 con 0x55685191d4d0 2023-07-13T22:23:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.561+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685186eed0 con 0x55685191de20 2023-07-13T22:23:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.562+0000 7f03c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.562+0000 7f03c2ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41370/0 (socket says 172.21.15.26:41370) 2023-07-13T22:23:54.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.562+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1299629825 learned_addr learned my addr 172.21.15.26:0/1299629825 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:54.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.563+0000 7f03c27fc700 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 0x55685191d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.563+0000 7f03abfff700 1 -- 172.21.15.26:0/2936345472 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f03bc00db20 con 0x7f03c410bd60 2023-07-13T22:23:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.566+0000 7f03abfff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 0x7f03ac0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.566+0000 7f03abfff700 1 -- 172.21.15.26:0/2936345472 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03bc0cc0b0 con 0x7f03c410bd60 2023-07-13T22:23:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1299629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 msgr2=0x55685191d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 0x55685191d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1299629825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55685191ec70 msgr2=0x55685191f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55685191ec70 0x55685191f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1299629825 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55685169cde0 con 0x55685191de20 2023-07-13T22:23:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.568+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03b40098e0 tx=0x7f03b400b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30d345e2b9d5d05a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.569+0000 7f03a97fa700 1 -- 172.21.15.26:0/1299629825 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b401d070 con 0x55685191de20 2023-07-13T22:23:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.569+0000 7f03a97fa700 1 -- 172.21.15.26:0/1299629825 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b401e040 con 0x55685191de20 2023-07-13T22:23:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.569+0000 7f03a97fa700 1 -- 172.21.15.26:0/1299629825 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4004c00 con 0x55685191de20 2023-07-13T22:23:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.570+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1299629825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 msgr2=0x55685191e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.570+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f03b40098e0 tx=0x7f03b400b900 comp rx=0 tx=0).stop 2023-07-13T22:23:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.571+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1299629825 shutdown_connections 2023-07-13T22:23:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.571+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55685191ec70 0x55685191f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.571+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 0x55685191d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.571+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685191e6e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.571+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1299629825 >> 172.21.15.26:0/1299629825 conn(0x556851912da0 msgr2=0x5568519181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.572+0000 7f03c37fe700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 0x7f03ac0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.573+0000 7f03c37fe700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 0x7f03ac0a3800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f03bc095970 tx=0x7f03bc009e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.580+0000 7f03c27fc700 1 --2- 172.21.15.26:0/1299629825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191d4d0 0x55685191d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:54.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.580+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1299629825 shutdown_connections 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.580+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1299629825 wait complete. 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.581+0000 7f03dcd68b80 1 -- start start 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.581+0000 7f03dcd68b80 1 Processor -- start 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685180eac0 con 0x556851927cf0 2023-07-13T22:23:54.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685180e7c0 con 0x556851927880 2023-07-13T22:23:54.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568516f2920 con 0x55685191de20 2023-07-13T22:23:54.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.582+0000 7f03dcd68b80 10 client.0 _ll_register_callbacks cb 0x55685180c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:54.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.584+0000 7f03c2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.584+0000 7f03c2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56170/0 (socket says 172.21.15.26:56170) 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.584+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1927998650 learned_addr learned my addr 172.21.15.26:0/1927998650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c27fc700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1927998650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 msgr2=0x556851927340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1927998650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 msgr2=0x55685192ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.585+0000 7f03c2ffd700 1 -- 172.21.15.26:0/1927998650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c4117640 con 0x556851927880 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.586+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b4004850 tx=0x7f03b4023ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5515da3f4700f2bb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.586+0000 7f03a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.586+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b401d070 con 0x556851927880 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.586+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03b400f660 con 0x556851927880 2023-07-13T22:23:54.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.586+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568517c84f0 con 0x556851927880 2023-07-13T22:23:54.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.587+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03b4005950 con 0x556851927880 2023-07-13T22:23:54.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.587+0000 7f03dcd68b80 10 client.27113 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:54.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.587+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556851685c60 con 0x556851927880 2023-07-13T22:23:54.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5568517c84f0 con 0x556851927880 2023-07-13T22:23:54.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f03b4035070 con 0x556851927880 2023-07-13T22:23:54.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 10 client.27113 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:54.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 10 client.27113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:54.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 20 client.27113 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:54.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568519332e0 0x5568519356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 4 client.27113 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:54.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.588+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556851867eb0 con 0x5568519332e0 2023-07-13T22:23:54.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.593+0000 7f03c27fc700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568519332e0 0x5568519356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.594+0000 7f03c27fc700 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568519332e0 0x5568519356c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237331e331d43c47 server_cookie=124febf430da309 in_seq=0 out_seq=0 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.594+0000 7f03a6ff5700 10 client.27113 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.594+0000 7f03a6ff5700 1 -- 172.21.15.26:0/1927998650 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556851867eb0 con 0x5568519332e0 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.594+0000 7f03a6ff5700 10 client.27113 handle_command_reply: tid=0 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.594+0000 7f03dcd68b80 1 client.27113 shutdown 2023-07-13T22:23:54.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.595+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 msgr2=0x55685192c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.595+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b4004850 tx=0x7f03b4023ac0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 shutdown_connections 2023-07-13T22:23:54.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5568519332e0 0x5568519356c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/1927998650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192c8b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 >> 172.21.15.26:0/1927998650 conn(0x556851912da0 msgr2=0x5568519178b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03a6ff5700 0 client.27113 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:54.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 shutdown_connections 2023-07-13T22:23:54.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.596+0000 7f03dcd68b80 1 -- 172.21.15.26:0/1927998650 wait complete. 2023-07-13T22:23:54.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.597+0000 7f03dcd68b80 20 client.27113 trim_cache size 0 max 16384 2023-07-13T22:23:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.625+0000 7f03dcd68b80 1 Processor -- start 2023-07-13T22:23:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 -- start start 2023-07-13T22:23:54.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927880 0x55685192c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x556851867eb0 con 0x556851927880 2023-07-13T22:23:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568517c84f0 con 0x55685191de20 2023-07-13T22:23:54.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.626+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x556851914ec0 con 0x556851927cf0 2023-07-13T22:23:54.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41390/0 (socket says 172.21.15.26:41390) 2023-07-13T22:23:54.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 -- 172.21.15.26:0/3228490830 learned_addr learned my addr 172.21.15.26:0/3228490830 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927880 0x55685192c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 -- 172.21.15.26:0/3228490830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191de20 msgr2=0x556851927340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 -- 172.21.15.26:0/3228490830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927880 msgr2=0x55685192c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927880 0x55685192c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.627+0000 7f03c27fc700 1 -- 172.21.15.26:0/3228490830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568516c3250 con 0x556851927cf0 2023-07-13T22:23:54.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.628+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03b80096b0 tx=0x7f03b8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cfa9f73001859e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.628+0000 7f03a9ffb700 1 -- 172.21.15.26:0/3228490830 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8005230 con 0x556851927cf0 2023-07-13T22:23:54.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.628+0000 7f03a9ffb700 1 -- 172.21.15.26:0/3228490830 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b8023040 con 0x556851927cf0 2023-07-13T22:23:54.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.628+0000 7f03a9ffb700 1 -- 172.21.15.26:0/3228490830 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8010430 con 0x556851927cf0 2023-07-13T22:23:54.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.629+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3228490830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 msgr2=0x55685192f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.629+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03b80096b0 tx=0x7f03b8009260 comp rx=0 tx=0).stop 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3228490830 shutdown_connections 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556851927cf0 0x55685192f960 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55685191de20 0x556851927340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3228490830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927880 0x55685192c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3228490830 >> 172.21.15.26:0/3228490830 conn(0x556851912da0 msgr2=0x556851925260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3228490830 shutdown_connections 2023-07-13T22:23:54.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.630+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3228490830 wait complete. 2023-07-13T22:23:54.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.631+0000 7f03dcd68b80 1 -- start start 2023-07-13T22:23:54.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.631+0000 7f03dcd68b80 1 Processor -- start 2023-07-13T22:23:54.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685192b7a0 con 0x556851927cf0 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685192b910 con 0x556851927880 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55685192bad0 con 0x55685191de20 2023-07-13T22:23:54.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.632+0000 7f03dcd68b80 10 client.0 _ll_register_callbacks cb 0x7f03b400b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.633+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.633+0000 7f03c27fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41392/0 (socket says 172.21.15.26:41392) 2023-07-13T22:23:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.633+0000 7f03c27fc700 1 -- 172.21.15.26:0/3160624370 learned_addr learned my addr 172.21.15.26:0/3160624370 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:54.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.634+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 -- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 msgr2=0x55685192ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 -- 172.21.15.26:0/3160624370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 msgr2=0x55685192b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.635+0000 7f03c27fc700 1 -- 172.21.15.26:0/3160624370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b801d9d0 con 0x55685191de20 2023-07-13T22:23:54.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03c27fc700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005be0 tx=0x7f03b8009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e00f370e8de50a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:23:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8010650 con 0x55685191de20 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f03b80107b0 con 0x55685191de20 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03b401e040 con 0x55685191de20 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.643+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03b8010970 con 0x55685191de20 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.644+0000 7f03dcd68b80 10 client.27115 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.644+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55685169b2b0 con 0x55685191de20 2023-07-13T22:23:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.644+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03b401e040 con 0x55685191de20 2023-07-13T22:23:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f03b8029420 con 0x55685191de20 2023-07-13T22:23:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 10 client.27115 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 20 client.27115 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556851988660 0x55685198aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 4 client.27115 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:23:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.645+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55685182a050 con 0x556851988660 2023-07-13T22:23:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.646+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556851988660 0x55685198aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.646+0000 7f03a6ff5700 10 client.27115 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.646+0000 7f03c2ffd700 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556851988660 0x55685198aa70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a346ff55873143e3 server_cookie=bf467dcd5cc3574b in_seq=0 out_seq=0 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03a6ff5700 1 -- 172.21.15.26:0/3160624370 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55685182a050 con 0x556851988660 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03a6ff5700 10 client.27115 handle_command_reply: tid=0 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 client.27115 shutdown 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 msgr2=0x55685192c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005be0 tx=0x7f03b8009350 comp rx=0 tx=0).stop 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 shutdown_connections 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x556851927cf0 0x55685192b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x556851988660 0x55685198aa70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55685191de20 0x55685192c710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 --2- 172.21.15.26:0/3160624370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556851927880 0x55685192ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 >> 172.21.15.26:0/3160624370 conn(0x556851912da0 msgr2=0x55685192f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.647+0000 7f03a6ff5700 0 client.27115 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:54.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.648+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 shutdown_connections 2023-07-13T22:23:54.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.648+0000 7f03dcd68b80 1 -- 172.21.15.26:0/3160624370 wait complete. 2023-07-13T22:23:54.633 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:23:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.648+0000 7f03dcd68b80 20 client.27115 trim_cache size 0 max 16384 2023-07-13T22:23:54.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.661+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 msgr2=0x7f03ac0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.661+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 0x7f03ac0a3800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f03bc095970 tx=0x7f03bc009e20 comp rx=0 tx=0).stop 2023-07-13T22:23:54.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.661+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 msgr2=0x7f03c41abc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:54.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.661+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 0x7f03c41abc00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f03bc007f10 tx=0x7f03bc00d3b0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 shutdown_connections 2023-07-13T22:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03c410bd60 0x7f03c41abc00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03c4070ee0 0x7f03c41a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f03ac0a1350 0x7f03ac0a3800 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 --2- 172.21.15.26:0/2936345472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03c4071830 0x7f03c41a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 >> 172.21.15.26:0/2936345472 conn(0x7f03c406ca90 msgr2=0x7f03c406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.672+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 shutdown_connections 2023-07-13T22:23:54.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:54.673+0000 7f03c93ff700 1 -- 172.21.15.26:0/2936345472 wait complete. 2023-07-13T22:23:54.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:54 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:23:54.971 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:54 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:23:54.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:54 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:23:55.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:23:55.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:23:55.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:55 smithi117 ceph-mon[129074]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 316 MiB/s wr, 99 op/s 2023-07-13T22:23:55.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:55 smithi026 ceph-mon[95647]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 316 MiB/s wr, 99 op/s 2023-07-13T22:23:55.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:55 smithi121 ceph-mon[129187]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 316 MiB/s wr, 99 op/s 2023-07-13T22:23:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.282+0000 7f782cdff700 1 -- 172.21.15.26:0/38774111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 msgr2=0x7f782810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.282+0000 7f782cdff700 1 --2- 172.21.15.26:0/38774111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f782810b7d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f781c009990 tx=0x7f781c009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 -- 172.21.15.26:0/38774111 shutdown_connections 2023-07-13T22:23:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 --2- 172.21.15.26:0/38774111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782810bd60 0x7f782810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 --2- 172.21.15.26:0/38774111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7828070ee0 0x7f78280712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 --2- 172.21.15.26:0/38774111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f782810b7d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 -- 172.21.15.26:0/38774111 >> 172.21.15.26:0/38774111 conn(0x7f782806ca90 msgr2=0x7f782806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 -- 172.21.15.26:0/38774111 shutdown_connections 2023-07-13T22:23:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.283+0000 7f782cdff700 1 -- 172.21.15.26:0/38774111 wait complete. 2023-07-13T22:23:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.285+0000 7f782cdff700 1 Processor -- start 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 -- start start 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7828070ee0 0x7f78281a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f78281a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 0x7f78281abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78281ac230 con 0x7f7828070ee0 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782810ec40 con 0x7f782810bd60 2023-07-13T22:23:56.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.286+0000 7f782cdff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782810edd0 con 0x7f7828071830 2023-07-13T22:23:56.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.287+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 0x7f78281abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.287+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 0x7f78281abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56208/0 (socket says 172.21.15.26:56208) 2023-07-13T22:23:56.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.287+0000 7f7826d9d700 1 -- 172.21.15.26:0/169496083 learned_addr learned my addr 172.21.15.26:0/169496083 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:56.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.287+0000 7f7825d9b700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f78281a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.288+0000 7f7825d9b700 1 -- 172.21.15.26:0/169496083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 msgr2=0x7f78281abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.288+0000 7f7825d9b700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 0x7f78281abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.288+0000 7f7825d9b700 1 -- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7828070ee0 msgr2=0x7f78281a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.288+0000 7f7825d9b700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7828070ee0 0x7f78281a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.288+0000 7f7825d9b700 1 -- 172.21.15.26:0/169496083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0095f0 con 0x7f7828071830 2023-07-13T22:23:56.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.291+0000 7f7825d9b700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f78281a4bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f781c000c00 tx=0x7f781c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.302+0000 7f78177fe700 1 -- 172.21.15.26:0/169496083 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c01d070 con 0x7f7828071830 2023-07-13T22:23:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.303+0000 7f78177fe700 1 -- 172.21.15.26:0/169496083 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c021410 con 0x7f7828071830 2023-07-13T22:23:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.303+0000 7f78177fe700 1 -- 172.21.15.26:0/169496083 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c017650 con 0x7f7828071830 2023-07-13T22:23:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.307+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782810f050 con 0x7f7828071830 2023-07-13T22:23:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.307+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782810f480 con 0x7f7828071830 2023-07-13T22:23:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.314+0000 7f78177fe700 1 -- 172.21.15.26:0/169496083 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f781c021580 con 0x7f7828071830 2023-07-13T22:23:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.317+0000 7f78177fe700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 0x7f78100a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.317+0000 7f78177fe700 1 -- 172.21.15.26:0/169496083 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f781c0cd270 con 0x7f7828071830 2023-07-13T22:23:56.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.324+0000 7f782659c700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 0x7f78100a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.326+0000 7f782659c700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 0x7f78100a3750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f782000b420 tx=0x7f7820008540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.327+0000 7f78407a0b80 1 Processor -- start 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 -- start start 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6a6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e5a9d30 con 0x55634e6a4550 2023-07-13T22:23:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e5fcac0 con 0x55634e6a4ea0 2023-07-13T22:23:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.328+0000 7f78407a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e5f5f50 con 0x55634e6a5cf0 2023-07-13T22:23:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.336+0000 7f782659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.336+0000 7f782659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54740/0 (socket says 172.21.15.26:54740) 2023-07-13T22:23:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.336+0000 7f782659c700 1 -- 172.21.15.26:0/440985950 learned_addr learned my addr 172.21.15.26:0/440985950 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.336+0000 7f7825d9b700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f7826d9d700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6a6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 -- 172.21.15.26:0/440985950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 msgr2=0x55634e6a6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6a6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 -- 172.21.15.26:0/440985950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 msgr2=0x55634e6a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 -- 172.21.15.26:0/440985950 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634e423e60 con 0x55634e6a4550 2023-07-13T22:23:56.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f7826d9d700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:23:56.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f7825d9b700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:23:56.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.337+0000 7f782659c700 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78200176e0 tx=0x7f7820010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0975c88801f26e server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.363+0000 7f7814ff9700 1 -- 172.21.15.26:0/440985950 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782001c070 con 0x55634e6a4550 2023-07-13T22:23:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.363+0000 7f7814ff9700 1 -- 172.21.15.26:0/440985950 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78200228e0 con 0x55634e6a4550 2023-07-13T22:23:56.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.363+0000 7f7814ff9700 1 -- 172.21.15.26:0/440985950 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820010070 con 0x55634e6a4550 2023-07-13T22:23:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.364+0000 7f78407a0b80 1 -- 172.21.15.26:0/440985950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 msgr2=0x55634e6a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.364+0000 7f78407a0b80 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78200176e0 tx=0x7f7820010000 comp rx=0 tx=0).stop 2023-07-13T22:23:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.403+0000 7f78407a0b80 1 -- 172.21.15.26:0/440985950 shutdown_connections 2023-07-13T22:23:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.403+0000 7f78407a0b80 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6a4960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.403+0000 7f78407a0b80 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.403+0000 7f78407a0b80 1 --2- 172.21.15.26:0/440985950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.403+0000 7f78407a0b80 1 -- 172.21.15.26:0/440985950 >> 172.21.15.26:0/440985950 conn(0x55634e699e20 msgr2=0x55634e69f250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.419+0000 7f78407a0b80 1 -- 172.21.15.26:0/440985950 shutdown_connections 2023-07-13T22:23:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.424+0000 7f78407a0b80 1 -- 172.21.15.26:0/440985950 wait complete. 2023-07-13T22:23:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.425+0000 7f78407a0b80 1 -- start start 2023-07-13T22:23:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.425+0000 7f78407a0b80 1 Processor -- start 2023-07-13T22:23:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e595b40 con 0x55634e6a4550 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e595840 con 0x55634e6a4ea0 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.428+0000 7f78407a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e4799a0 con 0x55634e6a5cf0 2023-07-13T22:23:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.429+0000 7f78407a0b80 10 client.0 _ll_register_callbacks cb 0x55634e5937e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.430+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.431+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41414/0 (socket says 172.21.15.26:41414) 2023-07-13T22:23:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.431+0000 7f7826d9d700 1 -- 172.21.15.26:0/544486995 learned_addr learned my addr 172.21.15.26:0/544486995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.431+0000 7f782659c700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.432+0000 7f782659c700 1 -- 172.21.15.26:0/544486995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 msgr2=0x55634e6b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.432+0000 7f782659c700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.432+0000 7f782659c700 1 -- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 msgr2=0x55634e6ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.432+0000 7f782659c700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.432+0000 7f782659c700 1 -- 172.21.15.26:0/544486995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820017420 con 0x55634e6a4550 2023-07-13T22:23:56.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.433+0000 7f782659c700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6ae240 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f782000bbb0 tx=0x7f7820022a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed1d8176a3e3736 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.433+0000 7f780dffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:23:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.433+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782001c040 con 0x55634e6a4550 2023-07-13T22:23:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.433+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7820015740 con 0x55634e6a4550 2023-07-13T22:23:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.434+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55634e54f570 con 0x55634e6a4550 2023-07-13T22:23:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.434+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7820015900 con 0x55634e6a4550 2023-07-13T22:23:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.452+0000 7f78407a0b80 10 client.16554 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.452+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55634e40cce0 con 0x55634e6a4550 2023-07-13T22:23:56.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.454+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f782003a030 con 0x55634e6a4550 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.454+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f782002dc10 con 0x55634e6a4550 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 20 client.16554 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e6b9e10 0x55634e6bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 4 client.16554 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:56.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.455+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55634e5eef30 con 0x55634e6b9e10 2023-07-13T22:23:56.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.456+0000 7f7825d9b700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e6b9e10 0x55634e6bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.495+0000 7f7825d9b700 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e6b9e10 0x55634e6bc220 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830a93b1caa4d7c4 server_cookie=d09340002453e66f in_seq=0 out_seq=0 2023-07-13T22:23:56.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.495+0000 7f780dffb700 10 client.16554 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:56.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.497+0000 7f780dffb700 1 -- 172.21.15.26:0/544486995 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55634e5eef30 con 0x55634e6b9e10 2023-07-13T22:23:56.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.497+0000 7f780dffb700 10 client.16554 handle_command_reply: tid=0 2023-07-13T22:23:56.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.498+0000 7f78407a0b80 1 client.16554 shutdown 2023-07-13T22:23:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.505+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 msgr2=0x55634e6ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.505+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6ae240 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f782000bbb0 tx=0x7f7820022a50 comp rx=0 tx=0).stop 2023-07-13T22:23:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.505+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 shutdown_connections 2023-07-13T22:23:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.505+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6ae240 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e6b9e10 0x55634e6bc220 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a5cf0 0x55634e6b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 --2- 172.21.15.26:0/544486995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a4ea0 0x55634e6ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 >> 172.21.15.26:0/544486995 conn(0x55634e699e20 msgr2=0x55634e69e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f780dffb700 0 client.16554 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 shutdown_connections 2023-07-13T22:23:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.506+0000 7f78407a0b80 1 -- 172.21.15.26:0/544486995 wait complete. 2023-07-13T22:23:56.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.507+0000 7f78407a0b80 20 client.16554 trim_cache size 0 max 16384 2023-07-13T22:23:56.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.550+0000 7f78407a0b80 1 Processor -- start 2023-07-13T22:23:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.556+0000 7f78407a0b80 1 -- start start 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.556+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e69e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.556+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.556+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.556+0000 7f78407a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e6a4ce0 con 0x55634e6a4550 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.557+0000 7f78407a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e6999f0 con 0x55634e6a5cf0 2023-07-13T22:23:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.557+0000 7f78407a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e6abe90 con 0x55634e6a4ea0 2023-07-13T22:23:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f7825d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56232/0 (socket says 172.21.15.26:56232) 2023-07-13T22:23:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f7825d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41430/0 (socket says 172.21.15.26:41430) 2023-07-13T22:23:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f7826d9d700 1 -- 172.21.15.26:0/1571801755 learned_addr learned my addr 172.21.15.26:0/1571801755 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.564+0000 7f782659c700 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e69e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.565+0000 7f7825d9b700 1 -- 172.21.15.26:0/1571801755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 msgr2=0x55634e6bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.565+0000 7f7825d9b700 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.565+0000 7f7825d9b700 1 -- 172.21.15.26:0/1571801755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 msgr2=0x55634e69e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.565+0000 7f7825d9b700 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e69e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.565+0000 7f7825d9b700 1 -- 172.21.15.26:0/1571801755 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55634e69b660 con 0x55634e6a4ea0 2023-07-13T22:23:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.566+0000 7f7825d9b700 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f781c09a3c0 tx=0x7f781c0a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8e80f80331e642 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.596+0000 7f78157fa700 1 -- 172.21.15.26:0/1571801755 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c09e6f0 con 0x55634e6a4ea0 2023-07-13T22:23:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.596+0000 7f78157fa700 1 -- 172.21.15.26:0/1571801755 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781c0a2a80 con 0x55634e6a4ea0 2023-07-13T22:23:56.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.596+0000 7f78157fa700 1 -- 172.21.15.26:0/1571801755 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c09a920 con 0x55634e6a4ea0 2023-07-13T22:23:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.598+0000 7f78407a0b80 1 -- 172.21.15.26:0/1571801755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 msgr2=0x55634e6b5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.598+0000 7f78407a0b80 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f781c09a3c0 tx=0x7f781c0a0040 comp rx=0 tx=0).stop 2023-07-13T22:23:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.599+0000 7f78407a0b80 1 -- 172.21.15.26:0/1571801755 shutdown_connections 2023-07-13T22:23:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.599+0000 7f78407a0b80 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e69e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.599+0000 7f78407a0b80 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.599+0000 7f78407a0b80 1 --2- 172.21.15.26:0/1571801755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b5780 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.599+0000 7f78407a0b80 1 -- 172.21.15.26:0/1571801755 >> 172.21.15.26:0/1571801755 conn(0x55634e699e20 msgr2=0x55634e6acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.610+0000 7f78407a0b80 1 -- 172.21.15.26:0/1571801755 shutdown_connections 2023-07-13T22:23:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.610+0000 7f78407a0b80 1 -- 172.21.15.26:0/1571801755 wait complete. 2023-07-13T22:23:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.611+0000 7f78407a0b80 1 -- start start 2023-07-13T22:23:56.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.612+0000 7f78407a0b80 1 Processor -- start 2023-07-13T22:23:56.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6b5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e6b2d20 con 0x55634e6a4550 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e70dcc0 con 0x55634e6a5cf0 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55634e70de50 con 0x55634e6a4ea0 2023-07-13T22:23:56.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.613+0000 7f78407a0b80 10 client.0 _ll_register_callbacks cb 0x7f78200175b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56246/0 (socket says 172.21.15.26:56246) 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 -- 172.21.15.26:0/433437662 learned_addr learned my addr 172.21.15.26:0/433437662 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f782659c700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7825d9b700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 -- 172.21.15.26:0/433437662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 msgr2=0x55634e6b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 -- 172.21.15.26:0/433437662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 msgr2=0x55634e6b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:56.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6b5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.614+0000 7f7826d9d700 1 -- 172.21.15.26:0/433437662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782810f480 con 0x55634e6a5cf0 2023-07-13T22:23:56.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.615+0000 7f7826d9d700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7818012700 tx=0x7f7818012ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed3e5ef17ebd012 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.623+0000 7f780dffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.623+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818020070 con 0x55634e6a5cf0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.623+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7818004830 con 0x55634e6a5cf0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.623+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f78200228e0 con 0x55634e6a5cf0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.624+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78180049f0 con 0x55634e6a5cf0 2023-07-13T22:23:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.624+0000 7f78407a0b80 10 client.27128 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.624+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55634e44a2d0 con 0x55634e6a5cf0 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78200228e0 con 0x55634e6a5cf0 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7818023030 con 0x55634e6a5cf0 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 10 client.27128 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 10 client.27128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 20 client.27128 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e70f970 0x55634e711d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 4 client.27128 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:23:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55634e5b10d0 con 0x55634e70f970 2023-07-13T22:23:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.625+0000 7f782659c700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e70f970 0x55634e711d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.626+0000 7f780dffb700 10 client.27128 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:56.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.626+0000 7f782659c700 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e70f970 0x55634e711d80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6ced5e5ef9487 server_cookie=fd015c6754b18213 in_seq=0 out_seq=0 2023-07-13T22:23:56.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.646+0000 7f780dffb700 1 -- 172.21.15.26:0/433437662 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55634e5b10d0 con 0x55634e70f970 2023-07-13T22:23:56.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.646+0000 7f780dffb700 10 client.27128 handle_command_reply: tid=0 2023-07-13T22:23:56.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.658+0000 7f78407a0b80 1 client.27128 shutdown 2023-07-13T22:23:56.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.668+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 msgr2=0x55634e6b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.668+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7818012700 tx=0x7f7818012ac0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 shutdown_connections 2023-07-13T22:23:56.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55634e6a4550 0x55634e6b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55634e70f970 0x55634e711d80 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55634e6a5cf0 0x55634e6b2640 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 --2- 172.21.15.26:0/433437662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55634e6a4ea0 0x55634e6b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 >> 172.21.15.26:0/433437662 conn(0x55634e699e20 msgr2=0x55634e69e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f780dffb700 0 client.27128 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:56.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.669+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 shutdown_connections 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.670+0000 7f78407a0b80 1 -- 172.21.15.26:0/433437662 wait complete. 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.670+0000 7f78407a0b80 20 client.27128 trim_cache size 0 max 16384 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:23:56.654 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "09c8e361-d8e9-4105-9ee0-14c363f05314", 2023-07-13T22:23:56.655 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:23:56.655 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:56.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.696+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 msgr2=0x7f78100a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.696+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 0x7f78100a3750 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f782000b420 tx=0x7f7820008540 comp rx=0 tx=0).stop 2023-07-13T22:23:56.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.696+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 msgr2=0x7f78281a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:56.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.696+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f78281a4bf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f781c000c00 tx=0x7f781c00f690 comp rx=0 tx=0).stop 2023-07-13T22:23:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 shutdown_connections 2023-07-13T22:23:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f78100a12a0 0x7f78100a3750 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7828070ee0 0x7f78281a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f782810bd60 0x7f78281abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 --2- 172.21.15.26:0/169496083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7828071830 0x7f78281a4bf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:56.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.710+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 >> 172.21.15.26:0/169496083 conn(0x7f782806ca90 msgr2=0x7f782806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.734+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 shutdown_connections 2023-07-13T22:23:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:56.734+0000 7f782cdff700 1 -- 172.21.15.26:0/169496083 wait complete. 2023-07-13T22:23:56.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:23:56.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:23:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:23:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:56 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:23:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:56 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:23:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:56 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:56.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:23:56.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:23:56.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:56 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:23:56.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:56 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:23:56.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:56 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:56.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:23:56.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:23:56.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:56 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:23:56.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:56 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:23:56.988 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:56 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 -- 172.21.15.26:0/1464583905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae500710c0 msgr2=0x7fae500714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 --2- 172.21.15.26:0/1464583905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae500710c0 0x7fae500714d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae40009990 tx=0x7fae40009ca0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 -- 172.21.15.26:0/1464583905 shutdown_connections 2023-07-13T22:23:57.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 --2- 172.21.15.26:0/1464583905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae50076d80 0x7fae50079230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 --2- 172.21.15.26:0/1464583905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae500710c0 0x7fae500714d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 --2- 172.21.15.26:0/1464583905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae50071a10 0x7fae50076710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.641+0000 7fae4ffff700 1 -- 172.21.15.26:0/1464583905 >> 172.21.15.26:0/1464583905 conn(0x7fae5006c8b0 msgr2=0x7fae5006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.646+0000 7fae4ffff700 1 -- 172.21.15.26:0/1464583905 shutdown_connections 2023-07-13T22:23:57.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.648+0000 7fae4ffff700 1 -- 172.21.15.26:0/1464583905 wait complete. 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.650+0000 7fae4ffff700 1 Processor -- start 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.650+0000 7fae4ffff700 1 -- start start 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae50071a10 0x7fae50082680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae50076d80 0x7fae501b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501b47a0 con 0x7fae50071a10 2023-07-13T22:23:57.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501b4910 con 0x7fae500710c0 2023-07-13T22:23:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4ffff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae501b4ad0 con 0x7fae50076d80 2023-07-13T22:23:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56262/0 (socket says 172.21.15.26:56262) 2023-07-13T22:23:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.651+0000 7fae4effd700 1 -- 172.21.15.26:0/3524360671 learned_addr learned my addr 172.21.15.26:0/3524360671 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:57.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.652+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae50076d80 0x7fae501b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.655+0000 7fae4effd700 1 -- 172.21.15.26:0/3524360671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae50076d80 msgr2=0x7fae501b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.655+0000 7fae4effd700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae50076d80 0x7fae501b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.655+0000 7fae4effd700 1 -- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae50071a10 msgr2=0x7fae50082680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.655+0000 7fae4effd700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae50071a10 0x7fae50082680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.655+0000 7fae4effd700 1 -- 172.21.15.26:0/3524360671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400095f0 con 0x7fae500710c0 2023-07-13T22:23:57.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.656+0000 7fae4effd700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae400094b0 tx=0x7fae400047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.656+0000 7fae549fd700 1 -- 172.21.15.26:0/3524360671 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae40023070 con 0x7fae500710c0 2023-07-13T22:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.657+0000 7fae549fd700 1 -- 172.21.15.26:0/3524360671 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae40004830 con 0x7fae500710c0 2023-07-13T22:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.657+0000 7fae549fd700 1 -- 172.21.15.26:0/3524360671 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400055c0 con 0x7fae500710c0 2023-07-13T22:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.657+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae501b4d50 con 0x7fae500710c0 2023-07-13T22:23:57.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.657+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae501b51b0 con 0x7fae500710c0 2023-07-13T22:23:57.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.664+0000 7fae69333b80 1 Processor -- start 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.666+0000 7fae69333b80 1 -- start start 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1a070 0x555e0be1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1ae30 0x555e0be1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bd1ecb0 con 0x555e0be1a070 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bd71a40 con 0x555e0be1ae30 2023-07-13T22:23:57.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae69333b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bd6aed0 con 0x555e0be19720 2023-07-13T22:23:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:41470/0 (socket says 172.21.15.26:41470) 2023-07-13T22:23:57.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.667+0000 7fae4f7fe700 1 -- 172.21.15.26:0/4200836283 learned_addr learned my addr 172.21.15.26:0/4200836283 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1ae30 0x555e0be1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4f7fe700 1 -- 172.21.15.26:0/4200836283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1ae30 msgr2=0x555e0be1b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1ae30 0x555e0be1b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4f7fe700 1 -- 172.21.15.26:0/4200836283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1a070 msgr2=0x555e0be1a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1a070 0x555e0be1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.668+0000 7fae4f7fe700 1 -- 172.21.15.26:0/4200836283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0bb98de0 con 0x555e0be19720 2023-07-13T22:23:57.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.669+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fae480093c0 tx=0x7fae48009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c17cfa22e244612 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.670+0000 7fae35ffb700 1 -- 172.21.15.26:0/4200836283 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4801a070 con 0x555e0be19720 2023-07-13T22:23:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.670+0000 7fae35ffb700 1 -- 172.21.15.26:0/4200836283 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae48012040 con 0x555e0be19720 2023-07-13T22:23:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.670+0000 7fae35ffb700 1 -- 172.21.15.26:0/4200836283 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48011050 con 0x555e0be19720 2023-07-13T22:23:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.671+0000 7fae549fd700 1 -- 172.21.15.26:0/3524360671 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae40004ca0 con 0x7fae500710c0 2023-07-13T22:23:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.671+0000 7fae69333b80 1 -- 172.21.15.26:0/4200836283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 msgr2=0x555e0be19b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.671+0000 7fae69333b80 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fae480093c0 tx=0x7fae48009920 comp rx=0 tx=0).stop 2023-07-13T22:23:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.674+0000 7fae549fd700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.674+0000 7fae549fd700 1 -- 172.21.15.26:0/3524360671 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae400d0990 con 0x7fae500710c0 2023-07-13T22:23:57.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.680+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.681+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 0x7fae380a37f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae48009690 tx=0x7fae48009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 -- 172.21.15.26:0/4200836283 shutdown_connections 2023-07-13T22:23:57.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1ae30 0x555e0be1b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be19720 0x555e0be19b30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 --2- 172.21.15.26:0/4200836283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1a070 0x555e0be1a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 -- 172.21.15.26:0/4200836283 >> 172.21.15.26:0/4200836283 conn(0x555e0be0eff0 msgr2=0x555e0be14460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.682+0000 7fae69333b80 1 -- 172.21.15.26:0/4200836283 shutdown_connections 2023-07-13T22:23:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.683+0000 7fae69333b80 1 -- 172.21.15.26:0/4200836283 wait complete. 2023-07-13T22:23:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.683+0000 7fae69333b80 1 -- start start 2023-07-13T22:23:57.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.684+0000 7fae69333b80 1 Processor -- start 2023-07-13T22:23:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be23390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be2a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bd0aac0 con 0x555e0be19720 2023-07-13T22:23:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bd0a7c0 con 0x555e0be1a070 2023-07-13T22:23:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0bbee920 con 0x555e0be1ae30 2023-07-13T22:23:57.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.685+0000 7fae69333b80 10 client.0 _ll_register_callbacks cb 0x555e0bd08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.686+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.686+0000 7fae4f7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be2a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.686+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56278/0 (socket says 172.21.15.26:56278) 2023-07-13T22:23:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.686+0000 7fae4e7fc700 1 -- 172.21.15.26:0/1703259991 learned_addr learned my addr 172.21.15.26:0/1703259991 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:57.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.686+0000 7fae4effd700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be23390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.687+0000 7fae4e7fc700 1 -- 172.21.15.26:0/1703259991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 msgr2=0x555e0be2a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.687+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be2a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.687+0000 7fae4e7fc700 1 -- 172.21.15.26:0/1703259991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 msgr2=0x555e0be23390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.687+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be23390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.687+0000 7fae4e7fc700 1 -- 172.21.15.26:0/1703259991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae48005e70 con 0x555e0be1a070 2023-07-13T22:23:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.688+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e450 tx=0x7fae4400e760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78f3104d57d8a84 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.688+0000 7fae337f6700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.688+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44009530 con 0x555e0be1a070 2023-07-13T22:23:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.688+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae44013470 con 0x555e0be1a070 2023-07-13T22:23:57.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.689+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e0bcc44f0 con 0x555e0be1a070 2023-07-13T22:23:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.689+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae4400eed0 con 0x555e0be1a070 2023-07-13T22:23:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.689+0000 7fae69333b80 10 client.27140 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.689+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e0bb81c60 con 0x555e0be1a070 2023-07-13T22:23:57.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.690+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e0bcc44f0 con 0x555e0be1a070 2023-07-13T22:23:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.690+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae44011070 con 0x555e0be1a070 2023-07-13T22:23:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 10 client.27140 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 10 client.27140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 20 client.27140 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:57.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be2f180 0x555e0be31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 4 client.27140 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:23:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e0bd63eb0 con 0x555e0be2f180 2023-07-13T22:23:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.691+0000 7fae4effd700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be2f180 0x555e0be31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.692+0000 7fae4effd700 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be2f180 0x555e0be31590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806b47c884c0a8ba server_cookie=d157a74a5df4e242 in_seq=0 out_seq=0 2023-07-13T22:23:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.692+0000 7fae337f6700 10 client.27140 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.693+0000 7fae337f6700 1 -- 172.21.15.26:0/1703259991 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e0bd63eb0 con 0x555e0be2f180 2023-07-13T22:23:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.693+0000 7fae337f6700 10 client.27140 handle_command_reply: tid=0 2023-07-13T22:23:57.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.693+0000 7fae69333b80 1 client.27140 shutdown 2023-07-13T22:23:57.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 msgr2=0x555e0be238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e450 tx=0x7fae4400e760 comp rx=0 tx=0).stop 2023-07-13T22:23:57.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 shutdown_connections 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be23390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be2f180 0x555e0be31590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be2a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 --2- 172.21.15.26:0/1703259991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be238d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 >> 172.21.15.26:0/1703259991 conn(0x555e0be0eff0 msgr2=0x555e0be13b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.694+0000 7fae337f6700 0 client.27140 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.695+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 shutdown_connections 2023-07-13T22:23:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.695+0000 7fae69333b80 1 -- 172.21.15.26:0/1703259991 wait complete. 2023-07-13T22:23:57.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.696+0000 7fae69333b80 20 client.27140 trim_cache size 0 max 16384 2023-07-13T22:23:57.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.761+0000 7fae69333b80 1 Processor -- start 2023-07-13T22:23:57.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.762+0000 7fae69333b80 1 -- start start 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be2a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be22f00 con 0x555e0be19720 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be1a440 con 0x555e0be1a070 2023-07-13T22:23:57.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.763+0000 7fae69333b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be24010 con 0x555e0be1ae30 2023-07-13T22:23:57.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.764+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be2a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.764+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be2a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56280/0 (socket says 172.21.15.26:56280) 2023-07-13T22:23:57.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.764+0000 7fae4e7fc700 1 -- 172.21.15.26:0/2178940917 learned_addr learned my addr 172.21.15.26:0/2178940917 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:57.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.764+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.765+0000 7fae4f7fe700 1 -- 172.21.15.26:0/2178940917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 msgr2=0x555e0be2a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.765+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be2a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.765+0000 7fae4f7fe700 1 -- 172.21.15.26:0/2178940917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 msgr2=0x555e0be21210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.765+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.765+0000 7fae4f7fe700 1 -- 172.21.15.26:0/2178940917 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0bbbf250 con 0x555e0be1ae30 2023-07-13T22:23:57.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.766+0000 7fae4f7fe700 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be22040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae48009390 tx=0x7fae48027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9d63400549d512 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.772+0000 7fae367fc700 1 -- 172.21.15.26:0/2178940917 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4801a040 con 0x555e0be1ae30 2023-07-13T22:23:57.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.772+0000 7fae367fc700 1 -- 172.21.15.26:0/2178940917 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae48026070 con 0x555e0be1ae30 2023-07-13T22:23:57.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.772+0000 7fae367fc700 1 -- 172.21.15.26:0/2178940917 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48016780 con 0x555e0be1ae30 2023-07-13T22:23:57.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.777+0000 7fae69333b80 1 -- 172.21.15.26:0/2178940917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 msgr2=0x555e0be22040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.777+0000 7fae69333b80 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be22040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae48009390 tx=0x7fae48027000 comp rx=0 tx=0).stop 2023-07-13T22:23:57.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.778+0000 7fae69333b80 1 -- 172.21.15.26:0/2178940917 shutdown_connections 2023-07-13T22:23:57.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.778+0000 7fae69333b80 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be19720 0x555e0be21210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.778+0000 7fae69333b80 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1ae30 0x555e0be22040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.778+0000 7fae69333b80 1 --2- 172.21.15.26:0/2178940917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be1a070 0x555e0be2a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.778+0000 7fae69333b80 1 -- 172.21.15.26:0/2178940917 >> 172.21.15.26:0/2178940917 conn(0x555e0be0eff0 msgr2=0x555e0be31190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.791+0000 7fae69333b80 1 -- 172.21.15.26:0/2178940917 shutdown_connections 2023-07-13T22:23:57.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.791+0000 7fae69333b80 1 -- 172.21.15.26:0/2178940917 wait complete. 2023-07-13T22:23:57.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.792+0000 7fae69333b80 1 -- start start 2023-07-13T22:23:57.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.792+0000 7fae69333b80 1 Processor -- start 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1a070 0x555e0be26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1ae30 0x555e0be29a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be29fd0 con 0x555e0be1ae30 2023-07-13T22:23:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be2a140 con 0x555e0be19720 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0be2a300 con 0x555e0be1a070 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.793+0000 7fae69333b80 10 client.0 _ll_register_callbacks cb 0x7fae44009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.794+0000 7fae4e7fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1a070 0x555e0be26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.794+0000 7fae4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.794+0000 7fae4effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56294/0 (socket says 172.21.15.26:56294) 2023-07-13T22:23:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.794+0000 7fae4effd700 1 -- 172.21.15.26:0/2314285011 learned_addr learned my addr 172.21.15.26:0/2314285011 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 -- 172.21.15.26:0/2314285011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1a070 msgr2=0x555e0be26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1a070 0x555e0be26a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 -- 172.21.15.26:0/2314285011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1ae30 msgr2=0x555e0be29a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1ae30 0x555e0be29a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 -- 172.21.15.26:0/2314285011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e0bb98de0 con 0x555e0be19720 2023-07-13T22:23:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.795+0000 7fae4effd700 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae40004c50 tx=0x7fae40005820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49da2798fb540f94 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:23:57.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.800+0000 7fae337f6700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:23:57.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.800+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4009dde0 con 0x555e0be19720 2023-07-13T22:23:57.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.800+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae400a1e10 con 0x555e0be19720 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.800+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae48012040 con 0x555e0be19720 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.801+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae400a61a0 con 0x555e0be19720 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.801+0000 7fae69333b80 10 client.27146 fetch_fsmap learned FSMap version 29 2023-07-13T22:23:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.801+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e0bb972b0 con 0x555e0be19720 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.801+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae48012040 con 0x555e0be19720 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.801+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fae400cdf90 con 0x555e0be19720 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 10 client.27146 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:23:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 20 client.27146 populate_metadata read hostname 'smithi026' 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be83a40 0x555e0be85e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 4 client.27146 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:23:57.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.802+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e0bd26050 con 0x555e0be83a40 2023-07-13T22:23:57.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.804+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be83a40 0x555e0be85e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.805+0000 7fae337f6700 10 client.27146 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.805+0000 7fae4e7fc700 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be83a40 0x555e0be85e50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a14d5c6b36afcee server_cookie=9fb8dfdd46017de8 in_seq=0 out_seq=0 2023-07-13T22:23:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.805+0000 7fae337f6700 1 -- 172.21.15.26:0/2314285011 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555e0bd26050 con 0x555e0be83a40 2023-07-13T22:23:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.805+0000 7fae337f6700 10 client.27146 handle_command_reply: tid=0 2023-07-13T22:23:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.805+0000 7fae69333b80 1 client.27146 shutdown 2023-07-13T22:23:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 msgr2=0x555e0be26530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fae40004c50 tx=0x7fae40005820 comp rx=0 tx=0).stop 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 shutdown_connections 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e0be1ae30 0x555e0be29a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x555e0be83a40 0x555e0be85e50 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555e0be19720 0x555e0be26530 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 --2- 172.21.15.26:0/2314285011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e0be1a070 0x555e0be26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae337f6700 0 client.27146 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:23:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 >> 172.21.15.26:0/2314285011 conn(0x555e0be0eff0 msgr2=0x555e0be21210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.806+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 shutdown_connections 2023-07-13T22:23:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.807+0000 7fae69333b80 1 -- 172.21.15.26:0/2314285011 wait complete. 2023-07-13T22:23:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.807+0000 7fae69333b80 20 client.27146 trim_cache size 0 max 16384 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "09c8e361-d8e9-4105-9ee0-14c363f05314": { 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "09c8e361-d8e9-4105-9ee0-14c363f05314", 2023-07-13T22:23:57.792 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:23:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:23:57.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:23:57.793 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:23:57.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.808+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 msgr2=0x7fae380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.808+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 0x7fae380a37f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae48009690 tx=0x7fae48009600 comp rx=0 tx=0).stop 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.808+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 msgr2=0x7fae50082140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.808+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fae400094b0 tx=0x7fae400047a0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 shutdown_connections 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae50076d80 0x7fae501b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae500710c0 0x7fae50082140 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae380a1340 0x7fae380a37f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 --2- 172.21.15.26:0/3524360671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae50071a10 0x7fae50082680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:23:57.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 >> 172.21.15.26:0/3524360671 conn(0x7fae5006c8b0 msgr2=0x7fae5006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:23:57.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 shutdown_connections 2023-07-13T22:23:57.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:23:57.809+0000 7fae4ffff700 1 -- 172.21.15.26:0/3524360671 wait complete. 2023-07-13T22:23:57.834 INFO:tasks.cephfs.filesystem:scrub status for tag:09c8e361-d8e9-4105-9ee0-14c363f05314 - {'path': '/', 'tag': '09c8e361-d8e9-4105-9ee0-14c363f05314', 'options': 'recursive,force'} 2023-07-13T22:23:57.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:57 smithi117 ceph-mon[129074]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:23:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:57 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:23:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:57 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:57 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:23:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:57 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:57 smithi026 ceph-mon[95647]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:23:57.948 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:57 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:23:57.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:57 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:57 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:23:57.949 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:57 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:57 smithi121 ceph-mon[129187]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 281 MiB/s wr, 89 op/s 2023-07-13T22:23:57.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:57 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:23:57.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:57 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:57.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:57 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:23:57.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:57 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:23:59.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:23:59 smithi121 ceph-mon[129187]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 314 MiB/s wr, 102 op/s 2023-07-13T22:24:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:23:59 smithi117 ceph-mon[129074]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 314 MiB/s wr, 102 op/s 2023-07-13T22:24:00.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:23:59 smithi026 ceph-mon[95647]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 314 MiB/s wr, 102 op/s 2023-07-13T22:24:02.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:01 smithi117 ceph-mon[129074]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 100 op/s 2023-07-13T22:24:02.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:01 smithi026 ceph-mon[95647]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 100 op/s 2023-07-13T22:24:02.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:01 smithi121 ceph-mon[129187]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 314 MiB/s wr, 100 op/s 2023-07-13T22:24:04.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:03 smithi117 ceph-mon[129074]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:24:04.207 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:03 smithi026 ceph-mon[95647]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:24:04.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:03 smithi121 ceph-mon[129187]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 276 MiB/s wr, 91 op/s 2023-07-13T22:24:05.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:04 smithi117 ceph-mon[129074]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 107 op/s 2023-07-13T22:24:05.206 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:04 smithi026 ceph-mon[95647]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 107 op/s 2023-07-13T22:24:05.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:04 smithi121 ceph-mon[129187]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 107 op/s 2023-07-13T22:24:07.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:07 smithi117 ceph-mon[129074]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:24:07.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:07 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:24:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:07 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:24:07.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:07 smithi026 ceph-mon[95647]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:24:07.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:07 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:24:07.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:07 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:24:07.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:07 smithi121 ceph-mon[129187]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 286 MiB/s wr, 93 op/s 2023-07-13T22:24:07.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:07 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:24:07.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:07 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:24:09.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:09 smithi117 ceph-mon[129074]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 312 MiB/s wr, 103 op/s 2023-07-13T22:24:09.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:09 smithi026 ceph-mon[95647]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 312 MiB/s wr, 103 op/s 2023-07-13T22:24:09.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:09 smithi121 ceph-mon[129187]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 312 MiB/s wr, 103 op/s 2023-07-13T22:24:11.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:11 smithi117 ceph-mon[129074]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 310 MiB/s wr, 99 op/s 2023-07-13T22:24:11.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:11 smithi026 ceph-mon[95647]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 310 MiB/s wr, 99 op/s 2023-07-13T22:24:11.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:11 smithi121 ceph-mon[129187]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 310 MiB/s wr, 99 op/s 2023-07-13T22:24:13.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:13 smithi117 ceph-mon[129074]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 88 op/s 2023-07-13T22:24:13.947 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:13 smithi026 ceph-mon[95647]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 88 op/s 2023-07-13T22:24:13.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:13 smithi121 ceph-mon[129187]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 269 MiB/s wr, 88 op/s 2023-07-13T22:24:15.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:15 smithi117 ceph-mon[129074]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 323 MiB/s wr, 104 op/s 2023-07-13T22:24:15.950 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:15 smithi026 ceph-mon[95647]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 323 MiB/s wr, 104 op/s 2023-07-13T22:24:15.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:15 smithi121 ceph-mon[129187]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 323 MiB/s wr, 104 op/s 2023-07-13T22:24:17.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:17 smithi121 ceph-mon[129187]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 277 MiB/s wr, 90 op/s 2023-07-13T22:24:18.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:17 smithi117 ceph-mon[129074]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 277 MiB/s wr, 90 op/s 2023-07-13T22:24:18.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:17 smithi026 ceph-mon[95647]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 277 MiB/s wr, 90 op/s 2023-07-13T22:24:20.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:19 smithi117 ceph-mon[129074]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 310 MiB/s wr, 103 op/s 2023-07-13T22:24:20.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:19 smithi026 ceph-mon[95647]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 310 MiB/s wr, 103 op/s 2023-07-13T22:24:20.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:19 smithi121 ceph-mon[129187]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 310 MiB/s wr, 103 op/s 2023-07-13T22:24:22.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:21 smithi117 ceph-mon[129074]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 316 MiB/s wr, 103 op/s 2023-07-13T22:24:22.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:21 smithi026 ceph-mon[95647]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 316 MiB/s wr, 103 op/s 2023-07-13T22:24:22.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:21 smithi121 ceph-mon[129187]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 316 MiB/s wr, 103 op/s 2023-07-13T22:24:24.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:23 smithi117 ceph-mon[129074]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 270 MiB/s wr, 91 op/s 2023-07-13T22:24:24.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:23 smithi026 ceph-mon[95647]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 270 MiB/s wr, 91 op/s 2023-07-13T22:24:24.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:23 smithi121 ceph-mon[129187]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 270 MiB/s wr, 91 op/s 2023-07-13T22:24:25.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:24 smithi117 ceph-mon[129074]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 321 MiB/s wr, 104 op/s 2023-07-13T22:24:25.229 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:24 smithi026 ceph-mon[95647]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 321 MiB/s wr, 104 op/s 2023-07-13T22:24:25.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:24 smithi121 ceph-mon[129187]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 321 MiB/s wr, 104 op/s 2023-07-13T22:24:27.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:24:27.946 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:27 smithi026 ceph-mon[95647]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 92 op/s 2023-07-13T22:24:27.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:27 smithi121 ceph-mon[129187]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 92 op/s 2023-07-13T22:24:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:27 smithi117 ceph-mon[129074]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 284 MiB/s wr, 92 op/s 2023-07-13T22:24:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 -- 172.21.15.26:0/3156320836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65981057a0 msgr2=0x7f6598107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 --2- 172.21.15.26:0/3156320836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65981057a0 0x7f6598107c00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6590009f90 tx=0x7f659000b330 comp rx=0 tx=0).stop 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 -- 172.21.15.26:0/3156320836 shutdown_connections 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 --2- 172.21.15.26:0/3156320836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f6598102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 --2- 172.21.15.26:0/3156320836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65981057a0 0x7f6598107c00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 --2- 172.21.15.26:0/3156320836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598102e80 0x7f6598105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 -- 172.21.15.26:0/3156320836 >> 172.21.15.26:0/3156320836 conn(0x7f65980fa000 msgr2=0x7f65980fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.491+0000 7f659e9ff700 1 -- 172.21.15.26:0/3156320836 shutdown_connections 2023-07-13T22:24:28.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.492+0000 7f659e9ff700 1 -- 172.21.15.26:0/3156320836 wait complete. 2023-07-13T22:24:28.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.493+0000 7f659e9ff700 1 Processor -- start 2023-07-13T22:24:28.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.493+0000 7f659e9ff700 1 -- start start 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f659806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6598102e80 0x7f659806f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 0x7f65981a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659806f9b0 con 0x7f6598100560 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659806fb20 con 0x7f65981057a0 2023-07-13T22:24:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659e9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65981a5fc0 con 0x7f6598102e80 2023-07-13T22:24:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f6597fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f659806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f6597fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f659806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47974/0 (socket says 172.21.15.26:47974) 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f6597fff700 1 -- 172.21.15.26:0/1986461224 learned_addr learned my addr 172.21.15.26:0/1986461224 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f659cf9c700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 0x7f65981a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.494+0000 7f65977fe700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6598102e80 0x7f659806f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 -- 172.21.15.26:0/1986461224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6598102e80 msgr2=0x7f659806f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6598102e80 0x7f659806f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 -- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 msgr2=0x7f659806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f659806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 -- 172.21.15.26:0/1986461224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590009c40 con 0x7f65981057a0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.495+0000 7f659cf9c700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 0x7f65981a5a30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f659000b6c0 tx=0x7f6590004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.496+0000 7f65957fa700 1 -- 172.21.15.26:0/1986461224 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659000e040 con 0x7f65981057a0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.496+0000 7f65957fa700 1 -- 172.21.15.26:0/1986461224 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6590004e50 con 0x7f65981057a0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.496+0000 7f65957fa700 1 -- 172.21.15.26:0/1986461224 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659001dea0 con 0x7f65981057a0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.496+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65981a6240 con 0x7f65981057a0 2023-07-13T22:24:28.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.496+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65981a6670 con 0x7f65981057a0 2023-07-13T22:24:28.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.500+0000 7f65957fa700 1 -- 172.21.15.26:0/1986461224 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f659001d470 con 0x7f65981057a0 2023-07-13T22:24:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.502+0000 7f65957fa700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 0x7f65800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.503+0000 7f65957fa700 1 -- 172.21.15.26:0/1986461224 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f65900cdcd0 con 0x7f65981057a0 2023-07-13T22:24:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.503+0000 7f65b23cfb80 1 Processor -- start 2023-07-13T22:24:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.503+0000 7f6597fff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 0x7f65800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.503+0000 7f65b23cfb80 1 -- start start 2023-07-13T22:24:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d9196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d919c70 0x55b99d91a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d81dcb0 con 0x55b99d918dd0 2023-07-13T22:24:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d870a40 con 0x55b99d918480 2023-07-13T22:24:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d869ed0 con 0x55b99d919c70 2023-07-13T22:24:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f6597fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f6597fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58004/0 (socket says 172.21.15.26:58004) 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.504+0000 7f6597fff700 1 -- 172.21.15.26:0/2020660859 learned_addr learned my addr 172.21.15.26:0/2020660859 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f659cf9c700 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d919c70 0x55b99d91a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f65977fe700 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d9196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 0x7f65800a3680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6588005950 tx=0x7f6588009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 -- 172.21.15.26:0/2020660859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d919c70 msgr2=0x55b99d91a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d919c70 0x55b99d91a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 -- 172.21.15.26:0/2020660859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 msgr2=0x55b99d9196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d9196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 -- 172.21.15.26:0/2020660859 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99d697de0 con 0x55b99d918480 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.505+0000 7f6597fff700 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f658800e570 tx=0x7f6588014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8bb48ddbc6f74da server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.506+0000 7f657e7fc700 1 -- 172.21.15.26:0/2020660859 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588004030 con 0x55b99d918480 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.506+0000 7f657e7fc700 1 -- 172.21.15.26:0/2020660859 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6588003680 con 0x55b99d918480 2023-07-13T22:24:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.506+0000 7f657e7fc700 1 -- 172.21.15.26:0/2020660859 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880266f0 con 0x55b99d918480 2023-07-13T22:24:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2020660859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 msgr2=0x55b99d918890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f658800e570 tx=0x7f6588014000 comp rx=0 tx=0).stop 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2020660859 shutdown_connections 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d919c70 0x55b99d91a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918480 0x55b99d918890 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2020660859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d9196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2020660859 >> 172.21.15.26:0/2020660859 conn(0x55b99d90dda0 msgr2=0x55b99d9131d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2020660859 shutdown_connections 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.507+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2020660859 wait complete. 2023-07-13T22:24:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.508+0000 7f65b23cfb80 1 -- start start 2023-07-13T22:24:28.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 Processor -- start 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918dd0 0x55b99d92af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d809ac0 con 0x55b99d918480 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d8097c0 con 0x55b99d919c70 2023-07-13T22:24:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d6ed920 con 0x55b99d918dd0 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.509+0000 7f65b23cfb80 10 client.0 _ll_register_callbacks cb 0x55b99d807760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f6597fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f6597fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48006/0 (socket says 172.21.15.26:48006) 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f6597fff700 1 -- 172.21.15.26:0/214596036 learned_addr learned my addr 172.21.15.26:0/214596036 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f659cf9c700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f65977fe700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918dd0 0x55b99d92af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f659cf9c700 1 -- 172.21.15.26:0/214596036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918dd0 msgr2=0x55b99d92af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f659cf9c700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918dd0 0x55b99d92af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f659cf9c700 1 -- 172.21.15.26:0/214596036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 msgr2=0x55b99d92aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.510+0000 7f659cf9c700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f659cf9c700 1 -- 172.21.15.26:0/214596036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6588017070 con 0x55b99d919c70 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f6597fff700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f659cf9c700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6590097620 tx=0x7f659009af70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb38ec7cdf9558f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65777fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659009a810 con 0x55b99d919c70 2023-07-13T22:24:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f659009b120 con 0x55b99d919c70 2023-07-13T22:24:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b99d7c34f0 con 0x55b99d919c70 2023-07-13T22:24:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659009f150 con 0x55b99d919c70 2023-07-13T22:24:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65b23cfb80 10 client.27158 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.511+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b99d680c60 con 0x55b99d919c70 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b99d7c34f0 con 0x55b99d919c70 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f65900b91d0 con 0x55b99d919c70 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 10 client.27158 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 10 client.27158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 20 client.27158 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d9365b0 0x55b99d9389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 4 client.27158 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b99d862eb0 con 0x55b99d9365b0 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.512+0000 7f6597fff700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d9365b0 0x55b99d9389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.513+0000 7f6597fff700 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d9365b0 0x55b99d9389c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af4a4af3afe2901 server_cookie=3ca7cba065b4bf69 in_seq=0 out_seq=0 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.513+0000 7f65777fe700 10 client.27158 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.513+0000 7f65777fe700 1 -- 172.21.15.26:0/214596036 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b99d862eb0 con 0x55b99d9365b0 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.513+0000 7f65777fe700 10 client.27158 handle_command_reply: tid=0 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.513+0000 7f65b23cfb80 1 client.27158 shutdown 2023-07-13T22:24:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 msgr2=0x55b99d931f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931f90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6590097620 tx=0x7f659009af70 comp rx=0 tx=0).stop 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 shutdown_connections 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918480 0x55b99d92aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d9365b0 0x55b99d9389c0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931f90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/214596036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918dd0 0x55b99d92af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 >> 172.21.15.26:0/214596036 conn(0x55b99d90dda0 msgr2=0x55b99d9128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65777fe700 0 client.27158 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 shutdown_connections 2023-07-13T22:24:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.514+0000 7f65b23cfb80 1 -- 172.21.15.26:0/214596036 wait complete. 2023-07-13T22:24:28.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.515+0000 7f65b23cfb80 20 client.27158 trim_cache size 0 max 16384 2023-07-13T22:24:28.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 Processor -- start 2023-07-13T22:24:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 -- start start 2023-07-13T22:24:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918dd0 0x55b99d931f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 0x55b99d92aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d913ed0 con 0x55b99d919c70 2023-07-13T22:24:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d914ca0 con 0x55b99d918dd0 2023-07-13T22:24:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.533+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d92a580 con 0x55b99d918480 2023-07-13T22:24:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50962/0 (socket says 172.21.15.26:50962) 2023-07-13T22:24:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 -- 172.21.15.26:0/264374681 learned_addr learned my addr 172.21.15.26:0/264374681 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f65977fe700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918dd0 0x55b99d931f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 -- 172.21.15.26:0/264374681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918dd0 msgr2=0x55b99d931f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918dd0 0x55b99d931f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 -- 172.21.15.26:0/264374681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 msgr2=0x55b99d92aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f659cf9c700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 0x55b99d92aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 0x55b99d92aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.534+0000 7f6597fff700 1 -- 172.21.15.26:0/264374681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b99d91a310 con 0x55b99d918480 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.535+0000 7f659cf9c700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 0x55b99d92aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.535+0000 7f6597fff700 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f658800eb60 tx=0x7f6588003b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a6aa32434503965 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.535+0000 7f657effd700 1 -- 172.21.15.26:0/264374681 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658802a040 con 0x55b99d918480 2023-07-13T22:24:28.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.535+0000 7f657effd700 1 -- 172.21.15.26:0/264374681 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658800ed10 con 0x55b99d918480 2023-07-13T22:24:28.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.535+0000 7f657effd700 1 -- 172.21.15.26:0/264374681 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65880042e0 con 0x55b99d918480 2023-07-13T22:24:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 -- 172.21.15.26:0/264374681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 msgr2=0x55b99d9385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f658800eb60 tx=0x7f6588003b60 comp rx=0 tx=0).stop 2023-07-13T22:24:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 -- 172.21.15.26:0/264374681 shutdown_connections 2023-07-13T22:24:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d919c70 0x55b99d92aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d9385c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/264374681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d918dd0 0x55b99d931f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 -- 172.21.15.26:0/264374681 >> 172.21.15.26:0/264374681 conn(0x55b99d90dda0 msgr2=0x55b99d9128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 -- 172.21.15.26:0/264374681 shutdown_connections 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.536+0000 7f65b23cfb80 1 -- 172.21.15.26:0/264374681 wait complete. 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.537+0000 7f65b23cfb80 1 -- start start 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.537+0000 7f65b23cfb80 1 Processor -- start 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.537+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d92df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.537+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d92e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65b23cfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d9319a0 con 0x55b99d918dd0 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d931b10 con 0x55b99d919c70 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65b23cfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b99d931cd0 con 0x55b99d918480 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65b23cfb80 10 client.0 _ll_register_callbacks cb 0x7f6590011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f659cf9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f659cf9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58030/0 (socket says 172.21.15.26:58030) 2023-07-13T22:24:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f659cf9c700 1 -- 172.21.15.26:0/2005805650 learned_addr learned my addr 172.21.15.26:0/2005805650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:28.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f65977fe700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d92e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.538+0000 7f6597fff700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d92df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 -- 172.21.15.26:0/2005805650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 msgr2=0x55b99d92df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d92df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 -- 172.21.15.26:0/2005805650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 msgr2=0x55b99d92e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d92e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 -- 172.21.15.26:0/2005805650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b99d697de0 con 0x55b99d919c70 2023-07-13T22:24:28.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f65977fe700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d92e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.540+0000 7f65777fe700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.539+0000 7f659cf9c700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6590009f90 tx=0x7f6590097510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6c189f93dc7f33 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.540+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900a3250 con 0x55b99d919c70 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.540+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f659009b120 con 0x55b99d919c70 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.540+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6588003680 con 0x55b99d919c70 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.541+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659009f150 con 0x55b99d919c70 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.541+0000 7f65b23cfb80 10 client.27164 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:28.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.541+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b99d6be250 con 0x55b99d919c70 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.541+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6588003680 con 0x55b99d919c70 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f65900b9b40 con 0x55b99d919c70 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 10 client.27164 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 20 client.27164 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d98b0d0 0x55b99d98d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 4 client.27164 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:24:28.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.542+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b99d825050 con 0x55b99d98b0d0 2023-07-13T22:24:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.543+0000 7f6597fff700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d98b0d0 0x55b99d98d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.543+0000 7f6597fff700 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d98b0d0 0x55b99d98d4b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed96288a8450516 server_cookie=689650db26d66c2 in_seq=0 out_seq=0 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.543+0000 7f65777fe700 10 client.27164 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65777fe700 1 -- 172.21.15.26:0/2005805650 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b99d825050 con 0x55b99d98b0d0 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65777fe700 10 client.27164 handle_command_reply: tid=0 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 client.27164 shutdown 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 msgr2=0x55b99d931460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6590009f90 tx=0x7f6590097510 comp rx=0 tx=0).stop 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 shutdown_connections 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b99d98b0d0 0x55b99d98d4b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b99d919c70 0x55b99d931460 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b99d918480 0x55b99d92df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65777fe700 0 client.27164 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 --2- 172.21.15.26:0/2005805650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b99d918dd0 0x55b99d92e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.544+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 >> 172.21.15.26:0/2005805650 conn(0x55b99d90dda0 msgr2=0x55b99d9385c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.545+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 shutdown_connections 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.545+0000 7f65b23cfb80 1 -- 172.21.15.26:0/2005805650 wait complete. 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.545+0000 7f65b23cfb80 20 client.27164 trim_cache size 0 max 16384 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:24:28.530 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:24:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 msgr2=0x7f65800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 0x7f65800a3680 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6588005950 tx=0x7f6588009310 comp rx=0 tx=0).stop 2023-07-13T22:24:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 msgr2=0x7f65981a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 0x7f65981a5a30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f659000b6c0 tx=0x7f6590004c60 comp rx=0 tx=0).stop 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 shutdown_connections 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f65800a11d0 0x7f65800a3680 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6598100560 0x7f659806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65981057a0 0x7f65981a5a30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 --2- 172.21.15.26:0/1986461224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6598102e80 0x7f659806f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.546+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 >> 172.21.15.26:0/1986461224 conn(0x7f65980fa000 msgr2=0x7f6598101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:28.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.547+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 shutdown_connections 2023-07-13T22:24:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:28.547+0000 7f659e9ff700 1 -- 172.21.15.26:0/1986461224 wait complete. 2023-07-13T22:24:28.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:24:28.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:24:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.135+0000 7fd878fff700 1 -- 172.21.15.26:0/453889826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874101d20 msgr2=0x7fd874102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.135+0000 7fd878fff700 1 --2- 172.21.15.26:0/453889826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874101d20 0x7fd874102130 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b0a0 tx=0x7fd86c00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.136+0000 7fd878fff700 1 -- 172.21.15.26:0/453889826 shutdown_connections 2023-07-13T22:24:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.136+0000 7fd878fff700 1 --2- 172.21.15.26:0/453889826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd87410a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.136+0000 7fd878fff700 1 --2- 172.21.15.26:0/453889826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874101d20 0x7fd874102130 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.136+0000 7fd878fff700 1 --2- 172.21.15.26:0/453889826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd874102f10 0x7fd874103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.136+0000 7fd878fff700 1 -- 172.21.15.26:0/453889826 >> 172.21.15.26:0/453889826 conn(0x7fd8740fd360 msgr2=0x7fd8740ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.137+0000 7fd878fff700 1 -- 172.21.15.26:0/453889826 shutdown_connections 2023-07-13T22:24:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.137+0000 7fd878fff700 1 -- 172.21.15.26:0/453889826 wait complete. 2023-07-13T22:24:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.139+0000 7fd878fff700 1 Processor -- start 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 -- start start 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd874101d20 0x7fd87406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874102f10 0x7fd87406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87406f990 con 0x7fd874101d20 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87406fb00 con 0x7fd874102f10 2023-07-13T22:24:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.140+0000 7fd878fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741a8220 con 0x7fd8741038a0 2023-07-13T22:24:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50986/0 (socket says 172.21.15.26:50986) 2023-07-13T22:24:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 -- 172.21.15.26:0/3087546687 learned_addr learned my addr 172.21.15.26:0/3087546687 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd871d9b700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874102f10 0x7fd87406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 -- 172.21.15.26:0/3087546687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874102f10 msgr2=0x7fd87406f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874102f10 0x7fd87406f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 -- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd874101d20 msgr2=0x7fd87406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.142+0000 7fd872d9d700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd874101d20 0x7fd87406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.143+0000 7fd872d9d700 1 -- 172.21.15.26:0/3087546687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c009d00 con 0x7fd8741038a0 2023-07-13T22:24:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.143+0000 7fd872d9d700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00d980 tx=0x7fd85c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.144+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c009940 con 0x7fd8741038a0 2023-07-13T22:24:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.144+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd85c004830 con 0x7fd8741038a0 2023-07-13T22:24:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.144+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0054e0 con 0x7fd8741038a0 2023-07-13T22:24:29.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.145+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8741a8500 con 0x7fd8741038a0 2023-07-13T22:24:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.146+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8741a8990 con 0x7fd8741038a0 2023-07-13T22:24:29.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.148+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874109260 con 0x7fd8741038a0 2023-07-13T22:24:29.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.152+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd85c004ca0 con 0x7fd8741038a0 2023-07-13T22:24:29.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.155+0000 7fd86b7fe700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 0x7fd8600a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:29.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.155+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd85c0cc6b0 con 0x7fd8741038a0 2023-07-13T22:24:29.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.175+0000 7fd87259c700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 0x7fd8600a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:29.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.185+0000 7fd87259c700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 0x7fd8600a3800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b070 tx=0x7fd86c00b7b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:29.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.189+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd85c095eb0 con 0x7fd8741038a0 2023-07-13T22:24:29.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.982+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd87402c850 con 0x7fd8741038a0 2023-07-13T22:24:29.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.985+0000 7fd86b7fe700 1 -- 172.21.15.26:0/3087546687 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7fd85c017070 con 0x7fd8741038a0 2023-07-13T22:24:29.971 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:24:29.971 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:24:29.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.990+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 msgr2=0x7fd8600a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.990+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 0x7fd8600a3800 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd86c00b070 tx=0x7fd86c00b7b0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.990+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 msgr2=0x7fd8741a7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.990+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00d980 tx=0x7fd85c00dd40 comp rx=0 tx=0).stop 2023-07-13T22:24:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 shutdown_connections 2023-07-13T22:24:29.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd8600a1350 0x7fd8600a3800 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd874101d20 0x7fd87406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8741038a0 0x7fd8741a7c90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 --2- 172.21.15.26:0/3087546687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd874102f10 0x7fd87406f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:29.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:29.991+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 >> 172.21.15.26:0/3087546687 conn(0x7fd8740fd360 msgr2=0x7fd8741063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.015+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 shutdown_connections 2023-07-13T22:24:30.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.020+0000 7fd878fff700 1 -- 172.21.15.26:0/3087546687 wait complete. 2023-07-13T22:24:30.021 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:24:30.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:24:30.061 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:29 smithi026 ceph-mon[95647]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 319 MiB/s wr, 106 op/s 2023-07-13T22:24:30.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:29 smithi117 ceph-mon[129074]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 319 MiB/s wr, 106 op/s 2023-07-13T22:24:30.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:29 smithi121 ceph-mon[129187]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 319 MiB/s wr, 106 op/s 2023-07-13T22:24:30.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.604+0000 7f7a30bff700 1 -- 172.21.15.26:0/2491266677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 msgr2=0x7f7a2c10b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.604+0000 7f7a30bff700 1 --2- 172.21.15.26:0/2491266677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c10b7d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b0a0 tx=0x7f7a2400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.624+0000 7f7a30bff700 1 -- 172.21.15.26:0/2491266677 shutdown_connections 2023-07-13T22:24:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.624+0000 7f7a30bff700 1 --2- 172.21.15.26:0/2491266677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c10bd60 0x7f7a2c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.624+0000 7f7a30bff700 1 --2- 172.21.15.26:0/2491266677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a2c070ee0 0x7f7a2c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.624+0000 7f7a30bff700 1 --2- 172.21.15.26:0/2491266677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c10b7d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.624+0000 7f7a30bff700 1 -- 172.21.15.26:0/2491266677 >> 172.21.15.26:0/2491266677 conn(0x7f7a2c06ca90 msgr2=0x7f7a2c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.639+0000 7f7a30bff700 1 -- 172.21.15.26:0/2491266677 shutdown_connections 2023-07-13T22:24:30.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.648+0000 7f7a30bff700 1 -- 172.21.15.26:0/2491266677 wait complete. 2023-07-13T22:24:30.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.649+0000 7f7a30bff700 1 Processor -- start 2023-07-13T22:24:30.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 -- start start 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c070ee0 0x7f7a2c114c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a2c10bd60 0x7f7a2c1161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c1167a0 con 0x7f7a2c070ee0 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c116910 con 0x7f7a2c071830 2023-07-13T22:24:30.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.654+0000 7f7a30bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c116ad0 con 0x7f7a2c10bd60 2023-07-13T22:24:30.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.655+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.655+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58056/0 (socket says 172.21.15.26:58056) 2023-07-13T22:24:30.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.655+0000 7f7a29d9b700 1 -- 172.21.15.26:0/67035731 learned_addr learned my addr 172.21.15.26:0/67035731 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:30.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 -- 172.21.15.26:0/67035731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a2c10bd60 msgr2=0x7f7a2c1161d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-13T22:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a2c10bd60 0x7f7a2c1161d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 -- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c070ee0 msgr2=0x7f7a2c114c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c070ee0 0x7f7a2c114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 -- 172.21.15.26:0/67035731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24009d00 con 0x7f7a2c071830 2023-07-13T22:24:30.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.656+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b070 tx=0x7f7a240124a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.657+0000 7f7a1b7fe700 1 -- 172.21.15.26:0/67035731 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2400e040 con 0x7f7a2c071830 2023-07-13T22:24:30.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.657+0000 7f7a1b7fe700 1 -- 172.21.15.26:0/67035731 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a24003e10 con 0x7f7a2c071830 2023-07-13T22:24:30.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.657+0000 7f7a1b7fe700 1 -- 172.21.15.26:0/67035731 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24005750 con 0x7f7a2c071830 2023-07-13T22:24:30.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.657+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2c116cf0 con 0x7f7a2c071830 2023-07-13T22:24:30.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.657+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2c1171d0 con 0x7f7a2c071830 2023-07-13T22:24:30.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.664+0000 7f7a44475b80 1 Processor -- start 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 -- start start 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254be20 0x562cf254c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf254d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf2450cb0 con 0x562cf254be20 2023-07-13T22:24:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf24a3a40 con 0x562cf254cc70 2023-07-13T22:24:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.665+0000 7f7a44475b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf249ced0 con 0x562cf254b4d0 2023-07-13T22:24:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.666+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.666+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51002/0 (socket says 172.21.15.26:51002) 2023-07-13T22:24:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.666+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/2144710289 learned_addr learned my addr 172.21.15.26:0/2144710289 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/2144710289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 msgr2=0x562cf254d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf254d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/2144710289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254be20 msgr2=0x562cf254c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254be20 0x562cf254c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/2144710289 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cf22cade0 con 0x562cf254b4d0 2023-07-13T22:24:30.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.670+0000 7f7a1b7fe700 1 -- 172.21.15.26:0/67035731 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7a240126c0 con 0x7f7a2c071830 2023-07-13T22:24:30.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.673+0000 7f7a1b7fe700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 0x7f7a140a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.673+0000 7f7a1b7fe700 1 -- 172.21.15.26:0/67035731 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a240cdde0 con 0x7f7a2c071830 2023-07-13T22:24:30.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.674+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0097b0 tx=0x7f7a1c009ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f590e3e220a8890 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.677+0000 7f7a18ff9700 1 -- 172.21.15.26:0/2144710289 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011830 con 0x562cf254b4d0 2023-07-13T22:24:30.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.677+0000 7f7a18ff9700 1 -- 172.21.15.26:0/2144710289 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1c011e50 con 0x562cf254b4d0 2023-07-13T22:24:30.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.677+0000 7f7a18ff9700 1 -- 172.21.15.26:0/2144710289 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00b360 con 0x562cf254b4d0 2023-07-13T22:24:30.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.679+0000 7f7a44475b80 1 -- 172.21.15.26:0/2144710289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 msgr2=0x562cf254b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.679+0000 7f7a44475b80 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0097b0 tx=0x7f7a1c009ac0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 0x7f7a140a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a44475b80 1 -- 172.21.15.26:0/2144710289 shutdown_connections 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a44475b80 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf254d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a44475b80 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254b4d0 0x562cf254b8e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a44475b80 1 --2- 172.21.15.26:0/2144710289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254be20 0x562cf254c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.697+0000 7f7a44475b80 1 -- 172.21.15.26:0/2144710289 >> 172.21.15.26:0/2144710289 conn(0x562cf2540da0 msgr2=0x562cf25461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.698+0000 7f7a44475b80 1 -- 172.21.15.26:0/2144710289 shutdown_connections 2023-07-13T22:24:30.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.698+0000 7f7a44475b80 1 -- 172.21.15.26:0/2144710289 wait complete. 2023-07-13T22:24:30.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.699+0000 7f7a44475b80 1 -- start start 2023-07-13T22:24:30.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.699+0000 7f7a44475b80 1 Processor -- start 2023-07-13T22:24:30.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2555270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 0x562cf25557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf255c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf243cac0 con 0x562cf254b4d0 2023-07-13T22:24:30.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf243c7c0 con 0x562cf254cc70 2023-07-13T22:24:30.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf2320920 con 0x562cf254be20 2023-07-13T22:24:30.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.700+0000 7f7a44475b80 10 client.0 _ll_register_callbacks cb 0x562cf243a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:30.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.701+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 0x562cf25557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.701+0000 7f7a2ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf255c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.701+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 0x562cf25557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51008/0 (socket says 172.21.15.26:51008) 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.701+0000 7f7a29d9b700 1 -- 172.21.15.26:0/3437623106 learned_addr learned my addr 172.21.15.26:0/3437623106 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.702+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3437623106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 msgr2=0x562cf25557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.702+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 0x562cf25557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.702+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3437623106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 msgr2=0x562cf2555270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.702+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2555270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.702+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3437623106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c0093e0 con 0x562cf254cc70 2023-07-13T22:24:30.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.709+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf255c7f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c000c00 tx=0x7f7a1c0105b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4275b31d70e0a80 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.710+0000 7f7a11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.710+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011630 con 0x562cf254cc70 2023-07-13T22:24:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.710+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a1c011790 con 0x562cf254cc70 2023-07-13T22:24:30.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.710+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cf23f64f0 con 0x562cf254cc70 2023-07-13T22:24:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.711+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a1c011950 con 0x562cf254cc70 2023-07-13T22:24:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.711+0000 7f7a44475b80 10 client.27179 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:30.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.711+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562cf22b3c60 con 0x562cf254cc70 2023-07-13T22:24:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cf23f64f0 con 0x562cf254cc70 2023-07-13T22:24:30.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a1c02a070 con 0x562cf254cc70 2023-07-13T22:24:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 10 client.27179 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 10 client.27179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:24:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 20 client.27179 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:30.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf2560ee0 0x562cf25632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 4 client.27179 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:30.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cf2495eb0 con 0x562cf2560ee0 2023-07-13T22:24:30.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.712+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf2560ee0 0x562cf25632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.717+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 0x7f7a140a3800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009770 tx=0x7f7a200047e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.717+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf2560ee0 0x562cf25632f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c2e16c0baf9c289 server_cookie=a3f16b9beddc62cd in_seq=0 out_seq=0 2023-07-13T22:24:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.717+0000 7f7a11ffb700 10 client.27179 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:24:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.718+0000 7f7a11ffb700 1 -- 172.21.15.26:0/3437623106 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562cf2495eb0 con 0x562cf2560ee0 2023-07-13T22:24:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.718+0000 7f7a11ffb700 10 client.27179 handle_command_reply: tid=0 2023-07-13T22:24:30.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.718+0000 7f7a44475b80 1 client.27179 shutdown 2023-07-13T22:24:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.720+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 msgr2=0x562cf255c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.720+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf255c7f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c000c00 tx=0x7f7a1c0105b0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 shutdown_connections 2023-07-13T22:24:30.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf2560ee0 0x562cf25632f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2555270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254cc70 0x562cf255c7f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3437623106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254be20 0x562cf25557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 >> 172.21.15.26:0/3437623106 conn(0x562cf2540da0 msgr2=0x562cf25458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.725+0000 7f7a11ffb700 0 client.27179 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.726+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 shutdown_connections 2023-07-13T22:24:30.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.726+0000 7f7a44475b80 1 -- 172.21.15.26:0/3437623106 wait complete. 2023-07-13T22:24:30.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.727+0000 7f7a44475b80 20 client.27179 trim_cache size 0 max 16384 2023-07-13T22:24:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 Processor -- start 2023-07-13T22:24:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 -- start start 2023-07-13T22:24:30.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2562ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf254fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf254c5a0 con 0x562cf254b4d0 2023-07-13T22:24:30.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf2547500 con 0x562cf254be20 2023-07-13T22:24:30.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.759+0000 7f7a44475b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf2554720 con 0x562cf254cc70 2023-07-13T22:24:30.797 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.761+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.761+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58084/0 (socket says 172.21.15.26:58084) 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.761+0000 7f7a29d9b700 1 -- 172.21.15.26:0/3230965825 learned_addr learned my addr 172.21.15.26:0/3230965825 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.761+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf254fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.762+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3230965825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 msgr2=0x562cf255c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.762+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.762+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3230965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 msgr2=0x562cf2562ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.762+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2562ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.762+0000 7f7a2ad9d700 1 -- 172.21.15.26:0/3230965825 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cf22f1250 con 0x562cf254cc70 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.763+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf254fc10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0096d0 tx=0x7f7a1c009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a5c8a3c198f8d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.763+0000 7f7a197fa700 1 -- 172.21.15.26:0/3230965825 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c01a470 con 0x562cf254cc70 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.763+0000 7f7a197fa700 1 -- 172.21.15.26:0/3230965825 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1c011af0 con 0x562cf254cc70 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.763+0000 7f7a197fa700 1 -- 172.21.15.26:0/3230965825 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c02c940 con 0x562cf254cc70 2023-07-13T22:24:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 -- 172.21.15.26:0/3230965825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 msgr2=0x562cf254fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf254fc10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c0096d0 tx=0x7f7a1c009300 comp rx=0 tx=0).stop 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 -- 172.21.15.26:0/3230965825 shutdown_connections 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf2562ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf254fc10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 --2- 172.21.15.26:0/3230965825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 -- 172.21.15.26:0/3230965825 >> 172.21.15.26:0/3230965825 conn(0x562cf2540da0 msgr2=0x562cf2553100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 -- 172.21.15.26:0/3230965825 shutdown_connections 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.765+0000 7f7a44475b80 1 -- 172.21.15.26:0/3230965825 wait complete. 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.766+0000 7f7a44475b80 1 -- start start 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.766+0000 7f7a44475b80 1 Processor -- start 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf255bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf255a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf255ab60 con 0x562cf254b4d0 2023-07-13T22:24:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf255acd0 con 0x562cf254be20 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cf255ae90 con 0x562cf254cc70 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.767+0000 7f7a44475b80 10 client.0 _ll_register_callbacks cb 0x7f7a1c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58086/0 (socket says 172.21.15.26:58086) 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 -- 172.21.15.26:0/1479735843 learned_addr learned my addr 172.21.15.26:0/1479735843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a2ad9d700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf255a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 -- 172.21.15.26:0/1479735843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 msgr2=0x562cf255a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf255a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 -- 172.21.15.26:0/1479735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 msgr2=0x562cf255bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf255bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.768+0000 7f7a29d9b700 1 -- 172.21.15.26:0/1479735843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cf22b3c60 con 0x562cf254be20 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.771+0000 7f7a29d9b700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7a240977c0 tx=0x7f7a2409aed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4ad32277af21ed server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.771+0000 7f7a11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a240cba60 con 0x562cf254be20 2023-07-13T22:24:30.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7a2409f2f0 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a1c011e50 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a240a3680 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a44475b80 10 client.27185 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.774+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562cf22c92b0 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1c011e50 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7a2409a7b0 con 0x562cf254be20 2023-07-13T22:24:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 10 client.27185 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 10 client.27185 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 20 client.27185 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf25b5ca0 0x562cf25b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 4 client.27185 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.776+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562cf2458050 con 0x562cf25b5ca0 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.777+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf25b5ca0 0x562cf25b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.786+0000 7f7a2a59c700 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf25b5ca0 0x562cf25b80b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86d7806bc104776e server_cookie=66d835735ddafdfa in_seq=0 out_seq=0 2023-07-13T22:24:30.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.786+0000 7f7a11ffb700 10 client.27185 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.786+0000 7f7a11ffb700 1 -- 172.21.15.26:0/1479735843 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562cf2458050 con 0x562cf25b5ca0 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.786+0000 7f7a11ffb700 10 client.27185 handle_command_reply: tid=0 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.786+0000 7f7a44475b80 1 client.27185 shutdown 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 msgr2=0x562cf255c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7a240977c0 tx=0x7f7a2409aed0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 shutdown_connections 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x562cf25b5ca0 0x562cf25b80b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562cf254b4d0 0x562cf255bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562cf254cc70 0x562cf255a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 --2- 172.21.15.26:0/1479735843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562cf254be20 0x562cf255c4c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 >> 172.21.15.26:0/1479735843 conn(0x562cf2540da0 msgr2=0x562cf2553100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a11ffb700 0 client.27185 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:24:30.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.796+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 shutdown_connections 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.797+0000 7f7a44475b80 1 -- 172.21.15.26:0/1479735843 wait complete. 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.797+0000 7f7a44475b80 20 client.27185 trim_cache size 0 max 16384 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.815+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 msgr2=0x7f7a140a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.815+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 0x7f7a140a3800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009770 tx=0x7f7a200047e0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.815+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 msgr2=0x7f7a2c115190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:30.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.815+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b070 tx=0x7f7a240124a0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 shutdown_connections 2023-07-13T22:24:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7a140a1350 0x7f7a140a3800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c070ee0 0x7f7a2c114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7a2c10bd60 0x7f7a2c1161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 --2- 172.21.15.26:0/67035731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2c071830 0x7f7a2c115190 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400b070 tx=0x7f7a240124a0 comp rx=0 tx=0).stop 2023-07-13T22:24:30.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.816+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 >> 172.21.15.26:0/67035731 conn(0x7f7a2c06ca90 msgr2=0x7f7a2c06ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:30.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.824+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 shutdown_connections 2023-07-13T22:24:30.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:30.827+0000 7f7a30bff700 1 -- 172.21.15.26:0/67035731 wait complete. 2023-07-13T22:24:30.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:24:31.032 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:30 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3087546687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:24:31.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:30 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3087546687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:24:31.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:30 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3087546687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:24:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 -- 172.21.15.26:0/1148490954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5600ffae0 msgr2=0x7ff5600ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 --2- 172.21.15.26:0/1148490954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5600ffae0 0x7ff5600ffef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0098f0 tx=0x7ff55c009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 -- 172.21.15.26:0/1148490954 shutdown_connections 2023-07-13T22:24:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 --2- 172.21.15.26:0/1148490954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff560101660 0x7ff560107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 --2- 172.21.15.26:0/1148490954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5600ffae0 0x7ff5600ffef0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 --2- 172.21.15.26:0/1148490954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 0x7ff560101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 -- 172.21.15.26:0/1148490954 >> 172.21.15.26:0/1148490954 conn(0x7ff5600fb0e0 msgr2=0x7ff5600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 -- 172.21.15.26:0/1148490954 shutdown_connections 2023-07-13T22:24:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.337+0000 7ff5685ff700 1 -- 172.21.15.26:0/1148490954 wait complete. 2023-07-13T22:24:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.339+0000 7ff5685ff700 1 Processor -- start 2023-07-13T22:24:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.339+0000 7ff5685ff700 1 -- start start 2023-07-13T22:24:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5600ffae0 0x7ff5601958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 0x7ff560195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019d460 con 0x7ff5600ffae0 2023-07-13T22:24:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019d5a0 con 0x7ff560101660 2023-07-13T22:24:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff5685ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019d730 con 0x7ff560100cd0 2023-07-13T22:24:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff566b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff566b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58090/0 (socket says 172.21.15.26:58090) 2023-07-13T22:24:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff566b9c700 1 -- 172.21.15.26:0/2198148253 learned_addr learned my addr 172.21.15.26:0/2198148253 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.340+0000 7ff565b9a700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 0x7ff560195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff56639b700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5600ffae0 0x7ff5601958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff566b9c700 1 -- 172.21.15.26:0/2198148253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 msgr2=0x7ff560195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff566b9c700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 0x7ff560195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff566b9c700 1 -- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5600ffae0 msgr2=0x7ff5601958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff566b9c700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5600ffae0 0x7ff5601958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.341+0000 7ff566b9c700 1 -- 172.21.15.26:0/2198148253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5500095f0 con 0x7ff560101660 2023-07-13T22:24:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff566b9c700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff55800ef30 tx=0x7ff55800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff5577fe700 1 -- 172.21.15.26:0/2198148253 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558018070 con 0x7ff560101660 2023-07-13T22:24:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff5577fe700 1 -- 172.21.15.26:0/2198148253 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55800f040 con 0x7ff560101660 2023-07-13T22:24:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c009520 con 0x7ff560101660 2023-07-13T22:24:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff5577fe700 1 -- 172.21.15.26:0/2198148253 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55800cd00 con 0x7ff560101660 2023-07-13T22:24:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.342+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56019dd80 con 0x7ff560101660 2023-07-13T22:24:31.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.350+0000 7ff57bf1bb80 1 Processor -- start 2023-07-13T22:24:31.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.350+0000 7ff57bf1bb80 1 -- start start 2023-07-13T22:24:31.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3943c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb39445ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb394468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb39341cb0 con 0x55bb39446480 2023-07-13T22:24:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb39394a40 con 0x55bb3943ce20 2023-07-13T22:24:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.351+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3938ded0 con 0x55bb3943c4d0 2023-07-13T22:24:31.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.352+0000 7ff565b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3943c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.352+0000 7ff565b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3943c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51056/0 (socket says 172.21.15.26:51056) 2023-07-13T22:24:31.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.352+0000 7ff565b9a700 1 -- 172.21.15.26:0/4119762824 learned_addr learned my addr 172.21.15.26:0/4119762824 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:31.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.352+0000 7ff56639b700 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb39445ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.353+0000 7ff565b9a700 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb394468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 -- 172.21.15.26:0/4119762824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 msgr2=0x55bb3943c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3943c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 -- 172.21.15.26:0/4119762824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 msgr2=0x55bb394468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb394468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 -- 172.21.15.26:0/4119762824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb391bbde0 con 0x55bb3943ce20 2023-07-13T22:24:31.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.354+0000 7ff56639b700 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb39445ef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009230 tx=0x7ff55c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c005cf2dfab5811b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.361+0000 7ff5577fe700 1 -- 172.21.15.26:0/2198148253 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff558003a00 con 0x7ff560101660 2023-07-13T22:24:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.363+0000 7ff554bfe700 1 -- 172.21.15.26:0/4119762824 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00f660 con 0x55bb3943ce20 2023-07-13T22:24:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.363+0000 7ff554bfe700 1 -- 172.21.15.26:0/4119762824 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c01d040 con 0x55bb3943ce20 2023-07-13T22:24:31.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.363+0000 7ff554bfe700 1 -- 172.21.15.26:0/4119762824 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0055c0 con 0x55bb3943ce20 2023-07-13T22:24:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.363+0000 7ff5577fe700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 0x7ff54c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff5577fe700 1 -- 172.21.15.26:0/2198148253 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5580cbb50 con 0x7ff560101660 2023-07-13T22:24:31.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4119762824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 msgr2=0x55bb39445ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb39445ef0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009230 tx=0x7ff55c009f70 comp rx=0 tx=0).stop 2023-07-13T22:24:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4119762824 shutdown_connections 2023-07-13T22:24:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb394468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3943c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4119762824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb39445ef0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.364+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4119762824 >> 172.21.15.26:0/4119762824 conn(0x55bb39431da0 msgr2=0x55bb394371d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.365+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4119762824 shutdown_connections 2023-07-13T22:24:31.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.365+0000 7ff565b9a700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 0x7ff54c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.365+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4119762824 wait complete. 2023-07-13T22:24:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.366+0000 7ff57bf1bb80 1 -- start start 2023-07-13T22:24:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.366+0000 7ff57bf1bb80 1 Processor -- start 2023-07-13T22:24:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.369+0000 7ff565b9a700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 0x7ff54c0a3700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff550005fd0 tx=0x7ff550013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3945f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39466eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3932dac0 con 0x55bb39446480 2023-07-13T22:24:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3932d7c0 con 0x55bb3943ce20 2023-07-13T22:24:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb39211920 con 0x55bb3943c4d0 2023-07-13T22:24:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.381+0000 7ff57bf1bb80 10 client.0 _ll_register_callbacks cb 0x55bb3932b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.382+0000 7ff565b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.382+0000 7ff565b9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58114/0 (socket says 172.21.15.26:58114) 2023-07-13T22:24:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.382+0000 7ff565b9a700 1 -- 172.21.15.26:0/1420689580 learned_addr learned my addr 172.21.15.26:0/1420689580 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.383+0000 7ff56639b700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3945f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff565b9a700 1 -- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 msgr2=0x55bb3945f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff565b9a700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3945f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff566b9c700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39466eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff565b9a700 1 -- 172.21.15.26:0/1420689580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 msgr2=0x55bb39466eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff565b9a700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39466eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff565b9a700 1 -- 172.21.15.26:0/1420689580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff560108850 con 0x55bb3943ce20 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.384+0000 7ff566b9c700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39466eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.385+0000 7ff565b9a700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff5500036b0 tx=0x7ff550009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7675d2ae8c8a02 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.385+0000 7ff549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.386+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550021640 con 0x55bb3943ce20 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.386+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff5500217a0 con 0x55bb3943ce20 2023-07-13T22:24:31.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.387+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb392e74f0 con 0x55bb3943ce20 2023-07-13T22:24:31.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.387+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff550031030 con 0x55bb3943ce20 2023-07-13T22:24:31.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.387+0000 7ff57bf1bb80 10 client.27197 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:31.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.387+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bb391a4c60 con 0x55bb3943ce20 2023-07-13T22:24:31.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb392e74f0 con 0x55bb3943ce20 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff55002bb20 con 0x55bb3943ce20 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 10 client.27197 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 10 client.27197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 20 client.27197 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb3946b770 0x55bb3946db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 4 client.27197 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:31.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.388+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb39386eb0 con 0x55bb3946b770 2023-07-13T22:24:31.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.389+0000 7ff56639b700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb3946b770 0x55bb3946db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.390+0000 7ff549ffb700 10 client.27197 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.390+0000 7ff56639b700 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb3946b770 0x55bb3946db80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82356e3e7ade2aac server_cookie=7bb42742f22c6980 in_seq=0 out_seq=0 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.391+0000 7ff549ffb700 1 -- 172.21.15.26:0/1420689580 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb39386eb0 con 0x55bb3946b770 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.391+0000 7ff549ffb700 10 client.27197 handle_command_reply: tid=0 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.391+0000 7ff57bf1bb80 1 client.27197 shutdown 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.392+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 msgr2=0x55bb3945fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.392+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff5500036b0 tx=0x7ff550009310 comp rx=0 tx=0).stop 2023-07-13T22:24:31.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 shutdown_connections 2023-07-13T22:24:31.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb3946b770 0x55bb3946db80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39466eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943c4d0 0x55bb3945f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/1420689580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943ce20 0x55bb3945fe70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 >> 172.21.15.26:0/1420689580 conn(0x55bb39431da0 msgr2=0x55bb394368b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff549ffb700 0 client.27197 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 shutdown_connections 2023-07-13T22:24:31.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.393+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/1420689580 wait complete. 2023-07-13T22:24:31.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.394+0000 7ff57bf1bb80 20 client.27197 trim_cache size 0 max 16384 2023-07-13T22:24:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.410+0000 7ff57bf1bb80 1 Processor -- start 2023-07-13T22:24:31.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.410+0000 7ff57bf1bb80 1 -- start start 2023-07-13T22:24:31.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb394676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39466dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb3944c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3943d460 con 0x55bb39446480 2023-07-13T22:24:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3945ede0 con 0x55bb3943c4d0 2023-07-13T22:24:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.411+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb3943d1f0 con 0x55bb3943ce20 2023-07-13T22:24:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.412+0000 7ff56639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb394676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.412+0000 7ff565b9a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39466dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.412+0000 7ff56639b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb394676e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58122/0 (socket says 172.21.15.26:58122) 2023-07-13T22:24:31.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.412+0000 7ff56639b700 1 -- 172.21.15.26:0/4072520993 learned_addr learned my addr 172.21.15.26:0/4072520993 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:31.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.413+0000 7ff566b9c700 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb3944c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 -- 172.21.15.26:0/4072520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 msgr2=0x55bb39466dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39466dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 -- 172.21.15.26:0/4072520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 msgr2=0x55bb394676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb394676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 -- 172.21.15.26:0/4072520993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb391e2250 con 0x55bb39446480 2023-07-13T22:24:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.414+0000 7ff566b9c700 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb3944c840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff5580c9600 tx=0x7ff5580988f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d3619ffa4aeec server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.416+0000 7ff5553ff700 1 -- 172.21.15.26:0/4072520993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558098f10 con 0x55bb39446480 2023-07-13T22:24:31.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.416+0000 7ff5553ff700 1 -- 172.21.15.26:0/4072520993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55809cf40 con 0x55bb39446480 2023-07-13T22:24:31.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.416+0000 7ff5553ff700 1 -- 172.21.15.26:0/4072520993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558098660 con 0x55bb39446480 2023-07-13T22:24:31.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.417+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4072520993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 msgr2=0x55bb3944c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.417+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb3944c840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff5580c9600 tx=0x7ff5580988f0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4072520993 shutdown_connections 2023-07-13T22:24:31.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb3944c840 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb394676e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/4072520993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39466dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4072520993 >> 172.21.15.26:0/4072520993 conn(0x55bb39431da0 msgr2=0x55bb3945e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4072520993 shutdown_connections 2023-07-13T22:24:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.419+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/4072520993 wait complete. 2023-07-13T22:24:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.420+0000 7ff57bf1bb80 1 -- start start 2023-07-13T22:24:31.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.420+0000 7ff57bf1bb80 1 Processor -- start 2023-07-13T22:24:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb39466cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39463790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb39464370 con 0x55bb39446480 2023-07-13T22:24:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb394bf5e0 con 0x55bb3943c4d0 2023-07-13T22:24:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb394bf750 con 0x55bb3943ce20 2023-07-13T22:24:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.421+0000 7ff57bf1bb80 10 client.0 _ll_register_callbacks cb 0x7ff5500044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48162/0 (socket says 172.21.15.26:48162) 2023-07-13T22:24:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 -- 172.21.15.26:0/36419361 learned_addr learned my addr 172.21.15.26:0/36419361 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 -- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 msgr2=0x55bb39463790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39463790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 -- 172.21.15.26:0/36419361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 msgr2=0x55bb39466cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:31.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb39466cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.422+0000 7ff566b9c700 1 -- 172.21.15.26:0/36419361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56019dd80 con 0x55bb39446480 2023-07-13T22:24:31.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.423+0000 7ff566b9c700 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c6f0 tx=0x7ff5580cbd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dec7d73aaad00c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.425+0000 7ff549ffb700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:24:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.425+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5580a10d0 con 0x55bb39446480 2023-07-13T22:24:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.425+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff558098f10 con 0x55bb39446480 2023-07-13T22:24:31.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.425+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff55c01d040 con 0x55bb39446480 2023-07-13T22:24:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.427+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5580b7310 con 0x55bb39446480 2023-07-13T22:24:31.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.427+0000 7ff57bf1bb80 10 client.16596 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:31.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.427+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55bb391ba2b0 con 0x55bb39446480 2023-07-13T22:24:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.428+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff55c01d040 con 0x55bb39446480 2023-07-13T22:24:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.428+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5580c9ac0 con 0x55bb39446480 2023-07-13T22:24:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 10 client.16596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:24:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 20 client.16596 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:31.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb394c0d90 0x55bb394c3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 4 client.16596 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:24:31.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.429+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb39349050 con 0x55bb394c0d90 2023-07-13T22:24:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.489+0000 7ff56639b700 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb394c0d90 0x55bb394c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:31.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.506+0000 7ff56639b700 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb394c0d90 0x55bb394c3200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38efb8daa0e43e02 server_cookie=263f1bf9598f2b81 in_seq=0 out_seq=0 2023-07-13T22:24:31.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.509+0000 7ff549ffb700 10 client.16596 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:24:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.509+0000 7ff549ffb700 1 -- 172.21.15.26:0/36419361 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb39349050 con 0x55bb394c0d90 2023-07-13T22:24:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.509+0000 7ff549ffb700 10 client.16596 handle_command_reply: tid=0 2023-07-13T22:24:31.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.510+0000 7ff57bf1bb80 1 client.16596 shutdown 2023-07-13T22:24:31.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.515+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 msgr2=0x55bb39463cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.515+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff55800c6f0 tx=0x7ff5580cbd60 comp rx=0 tx=0).stop 2023-07-13T22:24:31.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 shutdown_connections 2023-07-13T22:24:31.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55bb394c0d90 0x55bb394c3200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55bb39446480 0x55bb39463cd0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb3943c4d0 0x55bb39466cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 --2- 172.21.15.26:0/36419361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bb3943ce20 0x55bb39463790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 >> 172.21.15.26:0/36419361 conn(0x55bb39431da0 msgr2=0x55bb3944c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.516+0000 7ff549ffb700 0 client.16596 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:24:31.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.519+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 shutdown_connections 2023-07-13T22:24:31.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.519+0000 7ff57bf1bb80 1 -- 172.21.15.26:0/36419361 wait complete. 2023-07-13T22:24:31.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.520+0000 7ff57bf1bb80 20 client.16596 trim_cache size 0 max 16384 2023-07-13T22:24:31.502 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:24:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 msgr2=0x7ff54c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 0x7ff54c0a3700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff550005fd0 tx=0x7ff550013040 comp rx=0 tx=0).stop 2023-07-13T22:24:31.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 msgr2=0x7ff56019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff55800ef30 tx=0x7ff55800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 shutdown_connections 2023-07-13T22:24:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7ff54c0a1250 0x7ff54c0a3700 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5600ffae0 0x7ff5601958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff560101660 0x7ff56019ce90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 --2- 172.21.15.26:0/2198148253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff560100cd0 0x7ff560195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:31.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.533+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 >> 172.21.15.26:0/2198148253 conn(0x7ff5600fb0e0 msgr2=0x7ff5600fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:31.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.544+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 shutdown_connections 2023-07-13T22:24:31.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:31.544+0000 7ff5685ff700 1 -- 172.21.15.26:0/2198148253 wait complete. 2023-07-13T22:24:31.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:24:31.951 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:31 smithi026 ceph-mon[95647]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 319 MiB/s wr, 103 op/s 2023-07-13T22:24:32.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:31 smithi117 ceph-mon[129074]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 319 MiB/s wr, 103 op/s 2023-07-13T22:24:32.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:31 smithi121 ceph-mon[129187]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 319 MiB/s wr, 103 op/s 2023-07-13T22:24:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.289+0000 7f3e8dfff700 1 -- 172.21.15.26:0/1057903707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e88074550 msgr2=0x7f3e880729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.289+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/1057903707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e88074550 0x7f3e880729a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009990 tx=0x7f3e78009ca0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.289+0000 7f3e8dfff700 1 -- 172.21.15.26:0/1057903707 shutdown_connections 2023-07-13T22:24:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/1057903707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e881079a0 0x7f3e88109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/1057903707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e88074550 0x7f3e880729a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/1057903707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e880733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 -- 172.21.15.26:0/1057903707 >> 172.21.15.26:0/1057903707 conn(0x7f3e880fb0e0 msgr2=0x7f3e880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 -- 172.21.15.26:0/1057903707 shutdown_connections 2023-07-13T22:24:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.290+0000 7f3e8dfff700 1 -- 172.21.15.26:0/1057903707 wait complete. 2023-07-13T22:24:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.291+0000 7f3e8dfff700 1 Processor -- start 2023-07-13T22:24:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 -- start start 2023-07-13T22:24:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e88074550 0x7f3e88103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 0x7f3e880ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88100130 con 0x7f3e88074550 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881002a0 con 0x7f3e881079a0 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.292+0000 7f3e8dfff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88100460 con 0x7f3e88072f70 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51094/0 (socket says 172.21.15.26:51094) 2023-07-13T22:24:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 -- 172.21.15.26:0/192882199 learned_addr learned my addr 172.21.15.26:0/192882199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e87fff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 0x7f3e880ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 -- 172.21.15.26:0/192882199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 msgr2=0x7f3e880ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 0x7f3e880ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 -- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e88074550 msgr2=0x7f3e88103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e88074550 0x7f3e88103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e877fe700 1 -- 172.21.15.26:0/192882199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780095f0 con 0x7f3e88072f70 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.293+0000 7f3e87fff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 0x7f3e880ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:24:32.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.294+0000 7f3e877fe700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800b680 tx=0x7f3e78005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.294+0000 7f3e84ff9700 1 -- 172.21.15.26:0/192882199 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7801d070 con 0x7f3e88072f70 2023-07-13T22:24:32.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.294+0000 7f3e84ff9700 1 -- 172.21.15.26:0/192882199 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7800fbb0 con 0x7f3e88072f70 2023-07-13T22:24:32.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.294+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e881a4060 con 0x7f3e88072f70 2023-07-13T22:24:32.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.294+0000 7f3e84ff9700 1 -- 172.21.15.26:0/192882199 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78017650 con 0x7f3e88072f70 2023-07-13T22:24:32.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.295+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e881a4560 con 0x7f3e88072f70 2023-07-13T22:24:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.300+0000 7f3e84ff9700 1 -- 172.21.15.26:0/192882199 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3e7800fd20 con 0x7f3e88072f70 2023-07-13T22:24:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.302+0000 7f3ea197eb80 1 Processor -- start 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.302+0000 7f3ea197eb80 1 -- start start 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb364d0 0x5612beb368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb36e20 0x5612beb376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea3bcb0 con 0x5612beb364d0 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea8ea40 con 0x5612beb36e20 2023-07-13T22:24:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea87ed0 con 0x5612beb37c70 2023-07-13T22:24:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:51096/0 (socket says 172.21.15.26:51096) 2023-07-13T22:24:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.303+0000 7f3e86ffd700 1 -- 172.21.15.26:0/1842320463 learned_addr learned my addr 172.21.15.26:0/1842320463 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 -- 172.21.15.26:0/1842320463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb36e20 msgr2=0x5612beb376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb36e20 0x5612beb376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 -- 172.21.15.26:0/1842320463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb364d0 msgr2=0x5612beb368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb364d0 0x5612beb368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 -- 172.21.15.26:0/1842320463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be8b5de0 con 0x5612beb37c70 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e84ff9700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 0x7f3e700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009c80 tx=0x7f3e7c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a911a28de54a61a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.304+0000 7f3e84ff9700 1 -- 172.21.15.26:0/192882199 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e780cde60 con 0x7f3e88072f70 2023-07-13T22:24:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.305+0000 7f3e6dffb700 1 -- 172.21.15.26:0/1842320463 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c009800 con 0x5612beb37c70 2023-07-13T22:24:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.305+0000 7f3e6dffb700 1 -- 172.21.15.26:0/1842320463 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7c004cf0 con 0x5612beb37c70 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.305+0000 7f3e6dffb700 1 -- 172.21.15.26:0/1842320463 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c010430 con 0x5612beb37c70 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.305+0000 7f3e87fff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 0x7f3e700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1842320463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 msgr2=0x5612beb380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009c80 tx=0x7f3e7c009f90 comp rx=0 tx=0).stop 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3e87fff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 0x7f3e700a3680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009f20 tx=0x7f3e80009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1842320463 shutdown_connections 2023-07-13T22:24:32.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb364d0 0x5612beb368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb37c70 0x5612beb380e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1842320463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb36e20 0x5612beb376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.306+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1842320463 >> 172.21.15.26:0/1842320463 conn(0x5612beb2bda0 msgr2=0x5612beb311d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.307+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1842320463 shutdown_connections 2023-07-13T22:24:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.307+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1842320463 wait complete. 2023-07-13T22:24:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.307+0000 7f3ea197eb80 1 -- start start 2023-07-13T22:24:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.308+0000 7f3ea197eb80 1 Processor -- start 2023-07-13T22:24:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea27ac0 con 0x5612beb36e20 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612bea277c0 con 0x5612beb37c70 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612be90b920 con 0x5612beb364d0 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3ea197eb80 10 client.0 _ll_register_callbacks cb 0x5612bea25760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.309+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb51720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48212/0 (socket says 172.21.15.26:48212) 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e86ffd700 1 -- 172.21.15.26:0/3693584373 learned_addr learned my addr 172.21.15.26:0/3693584373 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 -- 172.21.15.26:0/3693584373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 msgr2=0x5612beb511e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb511e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 -- 172.21.15.26:0/3693584373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 msgr2=0x5612beb51720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb51720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.310+0000 7f3e87fff700 1 -- 172.21.15.26:0/3693584373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c009960 con 0x5612beb37c70 2023-07-13T22:24:32.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.312+0000 7f3e87fff700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb58760 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009440 tx=0x7f3e80004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5931485600444cf2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.312+0000 7f3e66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:32.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.312+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8001d070 con 0x5612beb37c70 2023-07-13T22:24:32.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.312+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e80006e40 con 0x5612beb37c70 2023-07-13T22:24:32.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.312+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612be9e14f0 con 0x5612beb37c70 2023-07-13T22:24:32.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.313+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e80011070 con 0x5612beb37c70 2023-07-13T22:24:32.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.313+0000 7f3ea197eb80 10 client.27209 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:32.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.313+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612be89ec60 con 0x5612beb37c70 2023-07-13T22:24:32.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.313+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612be9e14f0 con 0x5612beb37c70 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e8002b070 con 0x5612beb37c70 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 10 client.27209 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 10 client.27209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 20 client.27209 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612beb5cf90 0x5612beb5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 4 client.27209 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:32.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.314+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612bea80eb0 con 0x5612beb5cf90 2023-07-13T22:24:32.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.315+0000 7f3e877fe700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612beb5cf90 0x5612beb5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.315+0000 7f3e877fe700 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612beb5cf90 0x5612beb5f3a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=191415bd86ee9ca7 server_cookie=4add92ff86468fbf in_seq=0 out_seq=0 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.316+0000 7f3e66ffd700 10 client.27209 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.316+0000 7f3e66ffd700 1 -- 172.21.15.26:0/3693584373 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612bea80eb0 con 0x5612beb5cf90 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.316+0000 7f3e66ffd700 10 client.27209 handle_command_reply: tid=0 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.316+0000 7f3ea197eb80 1 client.27209 shutdown 2023-07-13T22:24:32.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 msgr2=0x5612beb58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb58760 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009440 tx=0x7f3e80004ad0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 shutdown_connections 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612beb5cf90 0x5612beb5f3a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb58760 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/3693584373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb51720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 >> 172.21.15.26:0/3693584373 conn(0x5612beb2bda0 msgr2=0x5612beb308b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3e66ffd700 0 client.27209 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:24:32.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 shutdown_connections 2023-07-13T22:24:32.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.317+0000 7f3ea197eb80 1 -- 172.21.15.26:0/3693584373 wait complete. 2023-07-13T22:24:32.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.318+0000 7f3ea197eb80 20 client.27209 trim_cache size 0 max 16384 2023-07-13T22:24:32.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.346+0000 7f3ea197eb80 1 Processor -- start 2023-07-13T22:24:32.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 -- start start 2023-07-13T22:24:32.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb371f0 con 0x5612beb36e20 2023-07-13T22:24:32.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb373e0 con 0x5612beb37c70 2023-07-13T22:24:32.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.347+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb32270 con 0x5612beb364d0 2023-07-13T22:24:32.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.350+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.350+0000 7f3e86ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48218/0 (socket says 172.21.15.26:48218) 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.350+0000 7f3e86ffd700 1 -- 172.21.15.26:0/269551358 learned_addr learned my addr 172.21.15.26:0/269551358 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 -- 172.21.15.26:0/269551358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 msgr2=0x5612beb58f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb58f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 -- 172.21.15.26:0/269551358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 msgr2=0x5612beb5efa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb5efa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 -- 172.21.15.26:0/269551358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612be8dc250 con 0x5612beb36e20 2023-07-13T22:24:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.351+0000 7f3e86ffd700 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009230 tx=0x7f3e7c005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4996ced3f16f975d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.354+0000 7f3e6e7fc700 1 -- 172.21.15.26:0/269551358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c005090 con 0x5612beb36e20 2023-07-13T22:24:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.354+0000 7f3e6e7fc700 1 -- 172.21.15.26:0/269551358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7c005370 con 0x5612beb36e20 2023-07-13T22:24:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.354+0000 7f3e6e7fc700 1 -- 172.21.15.26:0/269551358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c0226e0 con 0x5612beb36e20 2023-07-13T22:24:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 -- 172.21.15.26:0/269551358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 msgr2=0x5612beb58720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009230 tx=0x7f3e7c005900 comp rx=0 tx=0).stop 2023-07-13T22:24:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 -- 172.21.15.26:0/269551358 shutdown_connections 2023-07-13T22:24:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb37c70 0x5612beb5efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb364d0 0x5612beb58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/269551358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb36e20 0x5612beb58720 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.355+0000 7f3ea197eb80 1 -- 172.21.15.26:0/269551358 >> 172.21.15.26:0/269551358 conn(0x5612beb2bda0 msgr2=0x5612beb4fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.357+0000 7f3ea197eb80 1 -- 172.21.15.26:0/269551358 shutdown_connections 2023-07-13T22:24:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.357+0000 7f3ea197eb80 1 -- 172.21.15.26:0/269551358 wait complete. 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.357+0000 7f3ea197eb80 1 -- start start 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 Processor -- start 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb364d0 0x5612beb58080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb36e20 0x5612beb585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb551f0 con 0x5612beb37c70 2023-07-13T22:24:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.358+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb55360 con 0x5612beb364d0 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.359+0000 7f3ea197eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612beb55520 con 0x5612beb36e20 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.359+0000 7f3ea197eb80 10 client.0 _ll_register_callbacks cb 0x7f3e8000ce20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.362+0000 7f3e87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.362+0000 7f3e87fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48230/0 (socket says 172.21.15.26:48230) 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.362+0000 7f3e87fff700 1 -- 172.21.15.26:0/1597850055 learned_addr learned my addr 172.21.15.26:0/1597850055 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.363+0000 7f3e87fff700 1 -- 172.21.15.26:0/1597850055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb36e20 msgr2=0x5612beb585c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.363+0000 7f3e87fff700 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb36e20 0x5612beb585c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.363+0000 7f3e87fff700 1 -- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb364d0 msgr2=0x5612beb58080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.363+0000 7f3e87fff700 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb364d0 0x5612beb58080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.363+0000 7f3e87fff700 1 -- 172.21.15.26:0/1597850055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c01c040 con 0x5612beb37c70 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.364+0000 7f3e87fff700 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e80015380 tx=0x7f3e8002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb3034f7c7404ea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.364+0000 7f3e66ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.364+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8001d070 con 0x5612beb37c70 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.364+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3e8002c7b0 con 0x5612beb37c70 2023-07-13T22:24:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.364+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e7c004cf0 con 0x5612beb37c70 2023-07-13T22:24:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.365+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e800095e0 con 0x5612beb37c70 2023-07-13T22:24:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.365+0000 7f3ea197eb80 10 client.16605 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.365+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5612be8b42b0 con 0x5612beb37c70 2023-07-13T22:24:32.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7c004cf0 con 0x5612beb37c70 2023-07-13T22:24:32.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3e80018750 con 0x5612beb37c70 2023-07-13T22:24:32.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 10 client.16605 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:32.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 10 client.16605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:24:32.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 20 client.16605 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612bebc3470 0x5612bebc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 4 client.16605 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.366+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612bea43050 con 0x5612bebc3470 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.367+0000 7f3e877fe700 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612bebc3470 0x5612bebc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.368+0000 7f3e877fe700 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612bebc3470 0x5612bebc5880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67845acea6bf34c server_cookie=f33aba605c5d0d06 in_seq=0 out_seq=0 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.368+0000 7f3e66ffd700 10 client.16605 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:24:32.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.368+0000 7f3e66ffd700 1 -- 172.21.15.26:0/1597850055 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612bea43050 con 0x5612bebc3470 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.368+0000 7f3e66ffd700 10 client.16605 handle_command_reply: tid=0 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.368+0000 7f3ea197eb80 1 client.16605 shutdown 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 msgr2=0x5612beb54b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e80015380 tx=0x7f3e8002c660 comp rx=0 tx=0).stop 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 shutdown_connections 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5612beb37c70 0x5612beb54b20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5612bebc3470 0x5612bebc5880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5612beb364d0 0x5612beb58080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 --2- 172.21.15.26:0/1597850055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5612beb36e20 0x5612beb585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 >> 172.21.15.26:0/1597850055 conn(0x5612beb2bda0 msgr2=0x5612beb5efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3e66ffd700 0 client.16605 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:24:32.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 shutdown_connections 2023-07-13T22:24:32.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.369+0000 7f3ea197eb80 1 -- 172.21.15.26:0/1597850055 wait complete. 2023-07-13T22:24:32.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.370+0000 7f3ea197eb80 20 client.16605 trim_cache size 0 max 16384 2023-07-13T22:24:32.355 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 msgr2=0x7f3e700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 0x7f3e700a3680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009f20 tx=0x7f3e80009510 comp rx=0 tx=0).stop 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 msgr2=0x7f3e88102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800b680 tx=0x7f3e78005ef0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 shutdown_connections 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e881079a0 0x7f3e880ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e88072f70 0x7f3e88102e00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3e700a11d0 0x7f3e700a3680 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 --2- 172.21.15.26:0/192882199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e88074550 0x7f3e88103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.371+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 >> 172.21.15.26:0/192882199 conn(0x7f3e880fb0e0 msgr2=0x7f3e880fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.372+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 shutdown_connections 2023-07-13T22:24:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:32.372+0000 7f3e8dfff700 1 -- 172.21.15.26:0/192882199 wait complete. 2023-07-13T22:24:32.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:24:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.178+0000 7fcd521ff700 1 -- 172.21.15.26:0/3554093866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c10bd60 msgr2=0x7fcd4c10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.178+0000 7fcd521ff700 1 --2- 172.21.15.26:0/3554093866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c10bd60 0x7fcd4c10e1f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009990 tx=0x7fcd40009ca0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 -- 172.21.15.26:0/3554093866 shutdown_connections 2023-07-13T22:24:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 --2- 172.21.15.26:0/3554093866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c10bd60 0x7fcd4c10e1f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 --2- 172.21.15.26:0/3554093866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd4c070ee0 0x7fcd4c0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 --2- 172.21.15.26:0/3554093866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4c071830 0x7fcd4c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 -- 172.21.15.26:0/3554093866 >> 172.21.15.26:0/3554093866 conn(0x7fcd4c06ca90 msgr2=0x7fcd4c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 -- 172.21.15.26:0/3554093866 shutdown_connections 2023-07-13T22:24:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.179+0000 7fcd521ff700 1 -- 172.21.15.26:0/3554093866 wait complete. 2023-07-13T22:24:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.182+0000 7fcd521ff700 1 Processor -- start 2023-07-13T22:24:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.182+0000 7fcd521ff700 1 -- start start 2023-07-13T22:24:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.182+0000 7fcd521ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4c070ee0 0x7fcd4c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.182+0000 7fcd521ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd521ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd4c10bd60 0x7fcd4c1abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd521ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c1ac1f0 con 0x7fcd4c070ee0 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd521ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c10ec40 con 0x7fcd4c071830 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd521ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c10edd0 con 0x7fcd4c10bd60 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39098/0 (socket says 172.21.15.26:39098) 2023-07-13T22:24:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.183+0000 7fcd4affd700 1 -- 172.21.15.26:0/470536445 learned_addr learned my addr 172.21.15.26:0/470536445 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 -- 172.21.15.26:0/470536445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd4c10bd60 msgr2=0x7fcd4c1abc20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:24:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd4c10bd60 0x7fcd4c1abc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 -- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4c070ee0 msgr2=0x7fcd4c1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4c070ee0 0x7fcd4c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 -- 172.21.15.26:0/470536445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400095f0 con 0x7fcd4c071830 2023-07-13T22:24:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.184+0000 7fcd4affd700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b500 tx=0x7fcd3c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.185+0000 7fcd48ff9700 1 -- 172.21.15.26:0/470536445 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c011830 con 0x7fcd4c071830 2023-07-13T22:24:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.185+0000 7fcd48ff9700 1 -- 172.21.15.26:0/470536445 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd3c011e50 con 0x7fcd4c071830 2023-07-13T22:24:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.185+0000 7fcd48ff9700 1 -- 172.21.15.26:0/470536445 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c010e70 con 0x7fcd4c071830 2023-07-13T22:24:33.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.185+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c10f0b0 con 0x7fcd4c071830 2023-07-13T22:24:33.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.185+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c10f570 con 0x7fcd4c071830 2023-07-13T22:24:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.193+0000 7fcd65bbfb80 1 Processor -- start 2023-07-13T22:24:33.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.193+0000 7fcd65bbfb80 1 -- start start 2023-07-13T22:24:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7ce20 0x564f5af7d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7dc70 0x564f5af7e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5ae81cb0 con 0x564f5af7ce20 2023-07-13T22:24:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5aed4a40 con 0x564f5af7dc70 2023-07-13T22:24:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.194+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5aecded0 con 0x564f5af7c4d0 2023-07-13T22:24:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.195+0000 7fcd4bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.197+0000 7fcd4bfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38404/0 (socket says 172.21.15.26:38404) 2023-07-13T22:24:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.197+0000 7fcd4bfff700 1 -- 172.21.15.26:0/2805251960 learned_addr learned my addr 172.21.15.26:0/2805251960 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:33.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.202+0000 7fcd48ff9700 1 -- 172.21.15.26:0/470536445 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcd3c01a400 con 0x7fcd4c071830 2023-07-13T22:24:33.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.204+0000 7fcd48ff9700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 0x7fcd340a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.205+0000 7fcd48ff9700 1 -- 172.21.15.26:0/470536445 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd3c0cc770 con 0x7fcd4c071830 2023-07-13T22:24:33.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.205+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7ce20 0x564f5af7d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.206+0000 7fcd4bfff700 1 -- 172.21.15.26:0/2805251960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7dc70 msgr2=0x564f5af7e0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.206+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7dc70 0x564f5af7e0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.206+0000 7fcd4bfff700 1 -- 172.21.15.26:0/2805251960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7ce20 msgr2=0x564f5af7d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.206+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7ce20 0x564f5af7d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.206+0000 7fcd4bfff700 1 -- 172.21.15.26:0/2805251960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f5acfbde0 con 0x564f5af7c4d0 2023-07-13T22:24:33.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.207+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005910 tx=0x7fcd40005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88bc71fbf506e46 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.207+0000 7fcd31ffb700 1 -- 172.21.15.26:0/2805251960 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4001d070 con 0x564f5af7c4d0 2023-07-13T22:24:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.207+0000 7fcd31ffb700 1 -- 172.21.15.26:0/2805251960 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4000be00 con 0x564f5af7c4d0 2023-07-13T22:24:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.207+0000 7fcd31ffb700 1 -- 172.21.15.26:0/2805251960 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004950 con 0x564f5af7c4d0 2023-07-13T22:24:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.208+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/2805251960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 msgr2=0x564f5af7c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.208+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcd40005910 tx=0x7fcd40005c20 comp rx=0 tx=0).stop 2023-07-13T22:24:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/2805251960 shutdown_connections 2023-07-13T22:24:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7dc70 0x564f5af7e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af7c8e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/2805251960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7ce20 0x564f5af7d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/2805251960 >> 172.21.15.26:0/2805251960 conn(0x564f5af71da0 msgr2=0x564f5af771d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/2805251960 shutdown_connections 2023-07-13T22:24:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.209+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/2805251960 wait complete. 2023-07-13T22:24:33.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.210+0000 7fcd65bbfb80 1 -- start start 2023-07-13T22:24:33.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.211+0000 7fcd65bbfb80 1 Processor -- start 2023-07-13T22:24:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5ae6dac0 con 0x564f5af7dc70 2023-07-13T22:24:33.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5ae6d7c0 con 0x564f5af7ce20 2023-07-13T22:24:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5ad51920 con 0x564f5af7c4d0 2023-07-13T22:24:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.212+0000 7fcd65bbfb80 10 client.0 _ll_register_callbacks cb 0x564f5ae6b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:33.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38410/0 (socket says 172.21.15.26:38410) 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/1208685024 learned_addr learned my addr 172.21.15.26:0/1208685024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/1208685024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 msgr2=0x564f5af866c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af866c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 msgr2=0x564f5af8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.213+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/1208685024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c117640 con 0x564f5af7c4d0 2023-07-13T22:24:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.214+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcd44007a50 tx=0x7fcd44007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546f7568714dc6ea server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.214+0000 7fcd2affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:24:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.214+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004db0 con 0x564f5af7c4d0 2023-07-13T22:24:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.214+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd44003a00 con 0x564f5af7c4d0 2023-07-13T22:24:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.214+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f5ae274f0 con 0x564f5af7c4d0 2023-07-13T22:24:33.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.215+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd44003bc0 con 0x564f5af7c4d0 2023-07-13T22:24:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.215+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 0x7fcd340a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.216+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 0x7fcd340a3800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd400098e0 tx=0x7fcd40005810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.296+0000 7fcd65bbfb80 10 client.27208 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:33.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.296+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f5ace4c60 con 0x564f5af7c4d0 2023-07-13T22:24:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.300+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f5ae274f0 con 0x564f5af7c4d0 2023-07-13T22:24:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.300+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcd44004350 con 0x564f5af7c4d0 2023-07-13T22:24:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 10 client.27208 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:33.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 10 client.27208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:24:33.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 20 client.27208 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:33.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5af92040 0x564f5af94450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 4 client.27208 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:33.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.301+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f5aec6eb0 con 0x564f5af92040 2023-07-13T22:24:33.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.312+0000 7fcd4affd700 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5af92040 0x564f5af94450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.377+0000 7fcd4affd700 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5af92040 0x564f5af94450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=377710b6f5f93617 server_cookie=41fa8c24482438ae in_seq=0 out_seq=0 2023-07-13T22:24:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.377+0000 7fcd2affd700 10 client.27208 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:24:33.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.388+0000 7fcd2affd700 1 -- 172.21.15.26:0/1208685024 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f5aec6eb0 con 0x564f5af92040 2023-07-13T22:24:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.388+0000 7fcd2affd700 10 client.27208 handle_command_reply: tid=0 2023-07-13T22:24:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.388+0000 7fcd65bbfb80 1 client.27208 shutdown 2023-07-13T22:24:33.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 msgr2=0x564f5af86180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcd44007a50 tx=0x7fcd44007e10 comp rx=0 tx=0).stop 2023-07-13T22:24:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 shutdown_connections 2023-07-13T22:24:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5af92040 0x564f5af94450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af86180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/1208685024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af866c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 >> 172.21.15.26:0/1208685024 conn(0x564f5af71da0 msgr2=0x564f5af768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd2affd700 0 client.27208 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.390+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 shutdown_connections 2023-07-13T22:24:33.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.391+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/1208685024 wait complete. 2023-07-13T22:24:33.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.391+0000 7fcd65bbfb80 20 client.27208 trim_cache size 0 max 16384 2023-07-13T22:24:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.419+0000 7fcd65bbfb80 1 Processor -- start 2023-07-13T22:24:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 -- start start 2023-07-13T22:24:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af85830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af8d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af98db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af87300 con 0x564f5af7dc70 2023-07-13T22:24:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af992f0 con 0x564f5af7ce20 2023-07-13T22:24:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.420+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af78a00 con 0x564f5af7c4d0 2023-07-13T22:24:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.421+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af8d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.421+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af8d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39114/0 (socket says 172.21.15.26:39114) 2023-07-13T22:24:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.421+0000 7fcd4affd700 1 -- 172.21.15.26:0/362843737 learned_addr learned my addr 172.21.15.26:0/362843737 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.421+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af85830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.422+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af98db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.423+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/362843737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 msgr2=0x564f5af8d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.423+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af8d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.423+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/362843737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 msgr2=0x564f5af98db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.423+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af98db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.423+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/362843737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f5ad22250 con 0x564f5af7c4d0 2023-07-13T22:24:33.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.424+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af85830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004680 tx=0x7fcd44015470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba4db7089f234fe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.436+0000 7fcd327fc700 1 -- 172.21.15.26:0/362843737 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44006080 con 0x564f5af7c4d0 2023-07-13T22:24:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.436+0000 7fcd327fc700 1 -- 172.21.15.26:0/362843737 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd44015df0 con 0x564f5af7c4d0 2023-07-13T22:24:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.436+0000 7fcd327fc700 1 -- 172.21.15.26:0/362843737 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400fab0 con 0x564f5af7c4d0 2023-07-13T22:24:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/362843737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 msgr2=0x564f5af85830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af85830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcd44004680 tx=0x7fcd44015470 comp rx=0 tx=0).stop 2023-07-13T22:24:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/362843737 shutdown_connections 2023-07-13T22:24:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af98db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af85830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/362843737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af8d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.438+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/362843737 >> 172.21.15.26:0/362843737 conn(0x564f5af71da0 msgr2=0x564f5af771a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.447+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/362843737 shutdown_connections 2023-07-13T22:24:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.447+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/362843737 wait complete. 2023-07-13T22:24:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.448+0000 7fcd65bbfb80 1 -- start start 2023-07-13T22:24:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.449+0000 7fcd65bbfb80 1 Processor -- start 2023-07-13T22:24:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.449+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af8cfa0 con 0x564f5af7dc70 2023-07-13T22:24:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af8d110 con 0x564f5af7ce20 2023-07-13T22:24:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f5af8d2d0 con 0x564f5af7c4d0 2023-07-13T22:24:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.450+0000 7fcd65bbfb80 10 client.0 _ll_register_callbacks cb 0x7fcd44009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.457+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.457+0000 7fcd4affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af89a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39122/0 (socket says 172.21.15.26:39122) 2023-07-13T22:24:33.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.457+0000 7fcd4affd700 1 -- 172.21.15.26:0/3812848590 learned_addr learned my addr 172.21.15.26:0/3812848590 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:33.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.458+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.460+0000 7fcd4bfff700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.469+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/3812848590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 msgr2=0x564f5af89a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.469+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af89a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.469+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 msgr2=0x564f5af8ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:33.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.469+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.469+0000 7fcd4b7fe700 1 -- 172.21.15.26:0/3812848590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f5ace4c60 con 0x564f5af7c4d0 2023-07-13T22:24:33.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.480+0000 7fcd4b7fe700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af89500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x564f5af781b0 tx=0x7fcd4400ff40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b10e46877c8fe2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:33.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.483+0000 7fcd2affd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:24:33.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.485+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44015570 con 0x564f5af7c4d0 2023-07-13T22:24:33.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.485+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcd440156d0 con 0x564f5af7c4d0 2023-07-13T22:24:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.485+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd4000be00 con 0x564f5af7c4d0 2023-07-13T22:24:33.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.487+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd44015890 con 0x564f5af7c4d0 2023-07-13T22:24:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.487+0000 7fcd65bbfb80 10 client.27214 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:33.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.487+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564f5acfa2b0 con 0x564f5af7c4d0 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4000be00 con 0x564f5af7c4d0 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcd4403c070 con 0x564f5af7c4d0 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 10 client.27214 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 10 client.27214 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 20 client.27214 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5afe6b90 0x564f5afe8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:33.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 4 client.27214 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:24:33.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.489+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f5ae89050 con 0x564f5afe6b90 2023-07-13T22:24:33.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.492+0000 7fcd4affd700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5afe6b90 0x564f5afe8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:33.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.494+0000 7fcd4affd700 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5afe6b90 0x564f5afe8f70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f982b97c04c2678 server_cookie=e48eea98b35f9629 in_seq=0 out_seq=0 2023-07-13T22:24:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.494+0000 7fcd2affd700 10 client.27214 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:24:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.495+0000 7fcd2affd700 1 -- 172.21.15.26:0/3812848590 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f5ae89050 con 0x564f5afe6b90 2023-07-13T22:24:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.495+0000 7fcd2affd700 10 client.27214 handle_command_reply: tid=0 2023-07-13T22:24:33.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.495+0000 7fcd65bbfb80 1 client.27214 shutdown 2023-07-13T22:24:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.495+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 msgr2=0x564f5af89500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.495+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af89500 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x564f5af781b0 tx=0x7fcd4400ff40 comp rx=0 tx=0).stop 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 shutdown_connections 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x564f5afe6b90 0x564f5afe8f70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564f5af7dc70 0x564f5af8ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564f5af7c4d0 0x564f5af89500 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 --2- 172.21.15.26:0/3812848590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564f5af7ce20 0x564f5af89a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 >> 172.21.15.26:0/3812848590 conn(0x564f5af71da0 msgr2=0x564f5af8ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.496+0000 7fcd2affd700 0 client.27214 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:24:33.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.497+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 shutdown_connections 2023-07-13T22:24:33.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.497+0000 7fcd65bbfb80 1 -- 172.21.15.26:0/3812848590 wait complete. 2023-07-13T22:24:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.498+0000 7fcd65bbfb80 20 client.27214 trim_cache size 0 max 16384 2023-07-13T22:24:33.480 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:24:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.500+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 msgr2=0x7fcd340a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.500+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 0x7fcd340a3800 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcd400098e0 tx=0x7fcd40005810 comp rx=0 tx=0).stop 2023-07-13T22:24:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.500+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 msgr2=0x7fcd4c1a4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.500+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00b500 tx=0x7fcd3c00b8c0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 shutdown_connections 2023-07-13T22:24:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fcd340a1350 0x7fcd340a3800 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4c070ee0 0x7fcd4c1a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd4c10bd60 0x7fcd4c1abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 --2- 172.21.15.26:0/470536445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd4c071830 0x7fcd4c1a4bb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 >> 172.21.15.26:0/470536445 conn(0x7fcd4c06ca90 msgr2=0x7fcd4c06dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 shutdown_connections 2023-07-13T22:24:33.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:33.501+0000 7fcd521ff700 1 -- 172.21.15.26:0/470536445 wait complete. 2023-07-13T22:24:33.531 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:24:33.865 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:33 smithi026 ceph-mon[95647]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 260 MiB/s wr, 87 op/s 2023-07-13T22:24:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.074+0000 7f7479dff700 1 -- 172.21.15.26:0/3710696019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474071830 msgr2=0x7f747410b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.074+0000 7f7479dff700 1 --2- 172.21.15.26:0/3710696019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474071830 0x7f747410b7d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74640098f0 tx=0x7f7464009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.080+0000 7f7479dff700 1 -- 172.21.15.26:0/3710696019 shutdown_connections 2023-07-13T22:24:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.080+0000 7f7479dff700 1 --2- 172.21.15.26:0/3710696019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f747410bd60 0x7f747410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.080+0000 7f7479dff700 1 --2- 172.21.15.26:0/3710696019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7474070ee0 0x7f74740712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.080+0000 7f7479dff700 1 --2- 172.21.15.26:0/3710696019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474071830 0x7f747410b7d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.080+0000 7f7479dff700 1 -- 172.21.15.26:0/3710696019 >> 172.21.15.26:0/3710696019 conn(0x7f747406ca90 msgr2=0x7f747406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.081+0000 7f7479dff700 1 -- 172.21.15.26:0/3710696019 shutdown_connections 2023-07-13T22:24:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.081+0000 7f7479dff700 1 -- 172.21.15.26:0/3710696019 wait complete. 2023-07-13T22:24:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.082+0000 7f7479dff700 1 Processor -- start 2023-07-13T22:24:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.082+0000 7f7479dff700 1 -- start start 2023-07-13T22:24:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474070ee0 0x7f7474114bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474071830 0x7f74741150f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7474116770 con 0x7f7474070ee0 2023-07-13T22:24:34.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74741168e0 con 0x7f7474071830 2023-07-13T22:24:34.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7479dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7474116a50 con 0x7f747410bd60 2023-07-13T22:24:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.083+0000 7f7473fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38446/0 (socket says 172.21.15.26:38446) 2023-07-13T22:24:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 -- 172.21.15.26:0/1583782457 learned_addr learned my addr 172.21.15.26:0/1583782457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7472ffd700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474071830 0x7f74741150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 -- 172.21.15.26:0/1583782457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474071830 msgr2=0x7f74741150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474071830 0x7f74741150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 -- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474070ee0 msgr2=0x7f7474114bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474070ee0 0x7f7474114bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.084+0000 7f7473fff700 1 -- 172.21.15.26:0/1583782457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009520 con 0x7f747410bd60 2023-07-13T22:24:34.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.085+0000 7f7473fff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f746800e970 tx=0x7f746800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.106+0000 7f7470ff9700 1 -- 172.21.15.26:0/1583782457 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746800cbd0 con 0x7f747410bd60 2023-07-13T22:24:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.106+0000 7f7470ff9700 1 -- 172.21.15.26:0/1583782457 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f746800ee50 con 0x7f747410bd60 2023-07-13T22:24:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.106+0000 7f7470ff9700 1 -- 172.21.15.26:0/1583782457 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468005600 con 0x7f747410bd60 2023-07-13T22:24:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.107+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7474116d30 con 0x7f747410bd60 2023-07-13T22:24:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.107+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74741171c0 con 0x7f747410bd60 2023-07-13T22:24:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.114+0000 7f748d782b80 1 Processor -- start 2023-07-13T22:24:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 -- start start 2023-07-13T22:24:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x5593346406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334544cb0 con 0x55933463fe20 2023-07-13T22:24:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334597a40 con 0x55933463f4d0 2023-07-13T22:24:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.115+0000 7f748d782b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334590ed0 con 0x559334640c70 2023-07-13T22:24:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39158/0 (socket says 172.21.15.26:39158) 2023-07-13T22:24:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 -- 172.21.15.26:0/2489099455 learned_addr learned my addr 172.21.15.26:0/2489099455 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 -- 172.21.15.26:0/2489099455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 msgr2=0x5593346410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 -- 172.21.15.26:0/2489099455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 msgr2=0x5593346406e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:24:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x5593346406e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.120+0000 7f7472ffd700 1 -- 172.21.15.26:0/2489099455 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593343bede0 con 0x55933463f4d0 2023-07-13T22:24:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.125+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f746400a010 tx=0x7f7464019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701d6616f3bf02e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.126+0000 7f7461ffb700 1 -- 172.21.15.26:0/2489099455 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400f790 con 0x55933463f4d0 2023-07-13T22:24:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.126+0000 7f7461ffb700 1 -- 172.21.15.26:0/2489099455 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f746401d070 con 0x55933463f4d0 2023-07-13T22:24:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.126+0000 7f7461ffb700 1 -- 172.21.15.26:0/2489099455 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74640055d0 con 0x55933463f4d0 2023-07-13T22:24:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.126+0000 7f7470ff9700 1 -- 172.21.15.26:0/1583782457 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7468004830 con 0x7f747410bd60 2023-07-13T22:24:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.129+0000 7f7470ff9700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 0x7f745c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.129+0000 7f7470ff9700 1 -- 172.21.15.26:0/1583782457 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7468014070 con 0x7f747410bd60 2023-07-13T22:24:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.131+0000 7f748d782b80 1 -- 172.21.15.26:0/2489099455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 msgr2=0x55933463f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.131+0000 7f748d782b80 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f746400a010 tx=0x7f7464019040 comp rx=0 tx=0).stop 2023-07-13T22:24:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.132+0000 7f748d782b80 1 -- 172.21.15.26:0/2489099455 shutdown_connections 2023-07-13T22:24:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.132+0000 7f748d782b80 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.132+0000 7f748d782b80 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933463f8e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.132+0000 7f748d782b80 1 --2- 172.21.15.26:0/2489099455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x5593346406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.132+0000 7f748d782b80 1 -- 172.21.15.26:0/2489099455 >> 172.21.15.26:0/2489099455 conn(0x559334634da0 msgr2=0x55933463a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.135+0000 7f74737fe700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 0x7f745c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.136+0000 7f74737fe700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 0x7f745c0a3800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f746c0094f0 tx=0x7f746c009460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.140+0000 7f748d782b80 1 -- 172.21.15.26:0/2489099455 shutdown_connections 2023-07-13T22:24:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.144+0000 7f748d782b80 1 -- 172.21.15.26:0/2489099455 wait complete. 2023-07-13T22:24:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.144+0000 7f748d782b80 1 -- start start 2023-07-13T22:24:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.145+0000 7f748d782b80 1 Processor -- start 2023-07-13T22:24:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334649140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334530ac0 con 0x55933463f4d0 2023-07-13T22:24:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593345307c0 con 0x55933463fe20 2023-07-13T22:24:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334414920 con 0x559334640c70 2023-07-13T22:24:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.146+0000 7f748d782b80 10 client.0 _ll_register_callbacks cb 0x55933452e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.147+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.147+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39174/0 (socket says 172.21.15.26:39174) 2023-07-13T22:24:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.147+0000 7f7472ffd700 1 -- 172.21.15.26:0/2966790223 learned_addr learned my addr 172.21.15.26:0/2966790223 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.147+0000 7f7473fff700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.153+0000 7f7472ffd700 1 -- 172.21.15.26:0/2966790223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 msgr2=0x5593346506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.153+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.153+0000 7f7472ffd700 1 -- 172.21.15.26:0/2966790223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 msgr2=0x559334649140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.153+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334649140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.153+0000 7f7472ffd700 1 -- 172.21.15.26:0/2966790223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7474117640 con 0x55933463fe20 2023-07-13T22:24:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.154+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7464011040 tx=0x7f746400b2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ae10be2ae30efb server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.154+0000 7f7456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.155+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464005550 con 0x55933463fe20 2023-07-13T22:24:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.155+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f74640056b0 con 0x55933463fe20 2023-07-13T22:24:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.155+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593344ea4f0 con 0x55933463fe20 2023-07-13T22:24:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.156+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f746400b560 con 0x55933463fe20 2023-07-13T22:24:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.156+0000 7f748d782b80 10 client.27233 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.156+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5593343a7c60 con 0x55933463fe20 2023-07-13T22:24:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.156+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593344ea4f0 con 0x55933463fe20 2023-07-13T22:24:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7464015020 con 0x55933463fe20 2023-07-13T22:24:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 10 client.27233 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 10 client.27233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 20 client.27233 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559334654a20 0x559334656e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 4 client.27233 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.157+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559334589eb0 con 0x559334654a20 2023-07-13T22:24:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.158+0000 7f74737fe700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559334654a20 0x559334656e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.159+0000 7f74737fe700 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559334654a20 0x559334656e30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdec2d14c1599713 server_cookie=9c4e7de5d91f825c in_seq=0 out_seq=0 2023-07-13T22:24:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.159+0000 7f7456ffd700 10 client.27233 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.159+0000 7f7456ffd700 1 -- 172.21.15.26:0/2966790223 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559334589eb0 con 0x559334654a20 2023-07-13T22:24:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.159+0000 7f7456ffd700 10 client.27233 handle_command_reply: tid=0 2023-07-13T22:24:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.159+0000 7f748d782b80 1 client.27233 shutdown 2023-07-13T22:24:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 msgr2=0x559334649680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7464011040 tx=0x7f746400b2f0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 shutdown_connections 2023-07-13T22:24:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334649140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x559334654a20 0x559334656e30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x5593346506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 --2- 172.21.15.26:0/2966790223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463fe20 0x559334649680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.161+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 >> 172.21.15.26:0/2966790223 conn(0x559334634da0 msgr2=0x5593346398b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.162+0000 7f7456ffd700 0 client.27233 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.162+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 shutdown_connections 2023-07-13T22:24:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.162+0000 7f748d782b80 1 -- 172.21.15.26:0/2966790223 wait complete. 2023-07-13T22:24:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.163+0000 7f748d782b80 20 client.27233 trim_cache size 0 max 16384 2023-07-13T22:24:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.191+0000 7f748d782b80 1 Processor -- start 2023-07-13T22:24:34.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 -- start start 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933463fe20 0x5593346505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559334640c70 0x559334646fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933463bd00 con 0x55933463f4d0 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x559334640530 con 0x559334640c70 2023-07-13T22:24:34.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.192+0000 7f748d782b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933464a2c0 con 0x55933463fe20 2023-07-13T22:24:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.194+0000 7f74737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.194+0000 7f74737fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37394/0 (socket says 172.21.15.26:37394) 2023-07-13T22:24:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.194+0000 7f74737fe700 1 -- 172.21.15.26:0/2868684856 learned_addr learned my addr 172.21.15.26:0/2868684856 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.195+0000 7f74737fe700 1 -- 172.21.15.26:0/2868684856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933463fe20 msgr2=0x5593346505e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.195+0000 7f74737fe700 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933463fe20 0x5593346505e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.195+0000 7f74737fe700 1 -- 172.21.15.26:0/2868684856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559334640c70 msgr2=0x559334646fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.195+0000 7f74737fe700 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559334640c70 0x559334646fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.195+0000 7f74737fe700 1 -- 172.21.15.26:0/2868684856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593343e5250 con 0x55933463f4d0 2023-07-13T22:24:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.196+0000 7f74737fe700 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f746c023a60 tx=0x7f746c00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3165b0f35130c77 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.198+0000 7f74627fc700 1 -- 172.21.15.26:0/2868684856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c01c040 con 0x55933463f4d0 2023-07-13T22:24:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.198+0000 7f74627fc700 1 -- 172.21.15.26:0/2868684856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f746c01e040 con 0x55933463f4d0 2023-07-13T22:24:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.198+0000 7f74627fc700 1 -- 172.21.15.26:0/2868684856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c014600 con 0x55933463f4d0 2023-07-13T22:24:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.199+0000 7f748d782b80 1 -- 172.21.15.26:0/2868684856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 msgr2=0x559334650fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.199+0000 7f748d782b80 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f746c023a60 tx=0x7f746c00b050 comp rx=0 tx=0).stop 2023-07-13T22:24:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 -- 172.21.15.26:0/2868684856 shutdown_connections 2023-07-13T22:24:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463f4d0 0x559334650fb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559334640c70 0x559334646fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 --2- 172.21.15.26:0/2868684856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55933463fe20 0x5593346505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 -- 172.21.15.26:0/2868684856 >> 172.21.15.26:0/2868684856 conn(0x559334634da0 msgr2=0x559334647e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 -- 172.21.15.26:0/2868684856 shutdown_connections 2023-07-13T22:24:34.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.201+0000 7f748d782b80 1 -- 172.21.15.26:0/2868684856 wait complete. 2023-07-13T22:24:34.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.202+0000 7f748d782b80 1 -- start start 2023-07-13T22:24:34.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.202+0000 7f748d782b80 1 Processor -- start 2023-07-13T22:24:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933464ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x55933464c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933464d0b0 con 0x55933463fe20 2023-07-13T22:24:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933464d1f0 con 0x55933463f4d0 2023-07-13T22:24:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55933464d3b0 con 0x559334640c70 2023-07-13T22:24:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.203+0000 7f748d782b80 10 client.0 _ll_register_callbacks cb 0x7f74680091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:34.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:33 smithi117 ceph-mon[129074]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 260 MiB/s wr, 87 op/s 2023-07-13T22:24:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.205+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.205+0000 7f7472ffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37406/0 (socket says 172.21.15.26:37406) 2023-07-13T22:24:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.205+0000 7f7472ffd700 1 -- 172.21.15.26:0/2824838622 learned_addr learned my addr 172.21.15.26:0/2824838622 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.206+0000 7f7472ffd700 1 -- 172.21.15.26:0/2824838622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 msgr2=0x55933464c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.206+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x55933464c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.206+0000 7f7472ffd700 1 -- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 msgr2=0x55933464ff30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.206+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933464ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.206+0000 7f7472ffd700 1 -- 172.21.15.26:0/2824838622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c007e20 con 0x55933463fe20 2023-07-13T22:24:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.208+0000 7f7472ffd700 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7464009810 tx=0x7f7464004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cee3795434ffe3b server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:34.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.212+0000 7f7456ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:24:34.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.212+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464017400 con 0x55933463fe20 2023-07-13T22:24:34.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.212+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7464000e40 con 0x55933463fe20 2023-07-13T22:24:34.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.212+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f746401d070 con 0x55933463fe20 2023-07-13T22:24:34.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.213+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f746400b770 con 0x55933463fe20 2023-07-13T22:24:34.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.213+0000 7f748d782b80 10 client.16623 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:34.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.213+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5593343bd2b0 con 0x55933463fe20 2023-07-13T22:24:34.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.214+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f746401d070 con 0x55933463fe20 2023-07-13T22:24:34.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.214+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f7464015050 con 0x55933463fe20 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 10 client.16623 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 20 client.16623 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5593346a9a30 0x5593346abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 4 client.16623 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:24:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.215+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55933454c050 con 0x5593346a9a30 2023-07-13T22:24:34.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.216+0000 7f74737fe700 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5593346a9a30 0x5593346abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:34.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.217+0000 7f74737fe700 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5593346a9a30 0x5593346abe10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fd8f1c6987a94a server_cookie=832746a3828267d6 in_seq=0 out_seq=0 2023-07-13T22:24:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.217+0000 7f7456ffd700 10 client.16623 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.218+0000 7f7456ffd700 1 -- 172.21.15.26:0/2824838622 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55933454c050 con 0x5593346a9a30 2023-07-13T22:24:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.218+0000 7f7456ffd700 10 client.16623 handle_command_reply: tid=0 2023-07-13T22:24:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.218+0000 7f748d782b80 1 client.16623 shutdown 2023-07-13T22:24:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.218+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 msgr2=0x559334650470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.218+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7464009810 tx=0x7f7464004900 comp rx=0 tx=0).stop 2023-07-13T22:24:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 shutdown_connections 2023-07-13T22:24:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x5593346a9a30 0x5593346abe10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559334640c70 0x55933464c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55933463f4d0 0x55933464ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 --2- 172.21.15.26:0/2824838622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55933463fe20 0x559334650470 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 >> 172.21.15.26:0/2824838622 conn(0x559334634da0 msgr2=0x559334646fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.219+0000 7f7456ffd700 0 client.16623 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.220+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 shutdown_connections 2023-07-13T22:24:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.220+0000 7f748d782b80 1 -- 172.21.15.26:0/2824838622 wait complete. 2023-07-13T22:24:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.226+0000 7f748d782b80 20 client.16623 trim_cache size 0 max 16384 2023-07-13T22:24:34.215 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:24:34.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.237+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 msgr2=0x7f745c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.237+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 0x7f745c0a3800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f746c0094f0 tx=0x7f746c009460 comp rx=0 tx=0).stop 2023-07-13T22:24:34.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.237+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 msgr2=0x7f7474116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:34.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.237+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f746800e970 tx=0x7f746800ed30 comp rx=0 tx=0).stop 2023-07-13T22:24:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 shutdown_connections 2023-07-13T22:24:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f745c0a1350 0x7f745c0a3800 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7474070ee0 0x7f7474114bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f747410bd60 0x7f7474116170 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 --2- 172.21.15.26:0/1583782457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474071830 0x7f74741150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 >> 172.21.15.26:0/1583782457 conn(0x7f747406ca90 msgr2=0x7f747406dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 shutdown_connections 2023-07-13T22:24:34.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:34.238+0000 7f7479dff700 1 -- 172.21.15.26:0/1583782457 wait complete. 2023-07-13T22:24:34.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:33 smithi121 ceph-mon[129187]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 260 MiB/s wr, 87 op/s 2023-07-13T22:24:35.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:34 smithi117 ceph-mon[129074]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:35.210 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:34 smithi026 ceph-mon[95647]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:35.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:34 smithi121 ceph-mon[129187]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:35.307 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:24:35.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:24:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.856+0000 7f5c38fff700 1 -- 172.21.15.26:0/831227215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 msgr2=0x7f5c34102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.856+0000 7f5c38fff700 1 --2- 172.21.15.26:0/831227215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c34102120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098f0 tx=0x7f5c24009c00 comp rx=0 tx=0).stop 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 -- 172.21.15.26:0/831227215 shutdown_connections 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 --2- 172.21.15.26:0/831227215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c34103890 0x7f5c3410a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 --2- 172.21.15.26:0/831227215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c34102120 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 --2- 172.21.15.26:0/831227215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34102f00 0x7f5c34103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 -- 172.21.15.26:0/831227215 >> 172.21.15.26:0/831227215 conn(0x7f5c340fd350 msgr2=0x7f5c340ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 -- 172.21.15.26:0/831227215 shutdown_connections 2023-07-13T22:24:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.857+0000 7f5c38fff700 1 -- 172.21.15.26:0/831227215 wait complete. 2023-07-13T22:24:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.858+0000 7f5c38fff700 1 Processor -- start 2023-07-13T22:24:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.858+0000 7f5c38fff700 1 -- start start 2023-07-13T22:24:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c34102f00 0x7f5c34193b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34103890 0x7f5c3419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419b1d0 con 0x7f5c34102f00 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419b340 con 0x7f5c34103890 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c38fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419b4b0 con 0x7f5c34101d10 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c32d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34103890 0x7f5c3419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38488/0 (socket says 172.21.15.26:38488) 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 -- 172.21.15.26:0/2209504376 learned_addr learned my addr 172.21.15.26:0/2209504376 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 -- 172.21.15.26:0/2209504376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34103890 msgr2=0x7f5c3419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34103890 0x7f5c3419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 -- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c34102f00 msgr2=0x7f5c34193b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c34102f00 0x7f5c34193b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.859+0000 7f5c3259c700 1 -- 172.21.15.26:0/2209504376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009520 con 0x7f5c34101d10 2023-07-13T22:24:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c2400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c237fe700 1 -- 172.21.15.26:0/2209504376 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2401c070 con 0x7f5c34101d10 2023-07-13T22:24:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c237fe700 1 -- 172.21.15.26:0/2209504376 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2400f820 con 0x7f5c34101d10 2023-07-13T22:24:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c237fe700 1 -- 172.21.15.26:0/2209504376 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003b20 con 0x7f5c34101d10 2023-07-13T22:24:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3419b730 con 0x7f5c34101d10 2023-07-13T22:24:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.860+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3419bbf0 con 0x7f5c34101d10 2023-07-13T22:24:35.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.864+0000 7f5c237fe700 1 -- 172.21.15.26:0/2209504376 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5c24020030 con 0x7f5c34101d10 2023-07-13T22:24:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.866+0000 7f5c237fe700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 0x7f5c1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.866+0000 7f5c237fe700 1 -- 172.21.15.26:0/2209504376 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c240cd940 con 0x7f5c34101d10 2023-07-13T22:24:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.867+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 0x7f5c1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.867+0000 7f5c4c8d1b80 1 Processor -- start 2023-07-13T22:24:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.868+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 0x7f5c1c0a3680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005950 tx=0x7f5c280058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.868+0000 7f5c4c8d1b80 1 -- start start 2023-07-13T22:24:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b4f7d30 con 0x55c21b5f3cf0 2023-07-13T22:24:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b54aac0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b543f50 con 0x55c21b5f2550 2023-07-13T22:24:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38500/0 (socket says 172.21.15.26:38500) 2023-07-13T22:24:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 -- 172.21.15.26:0/2620419946 learned_addr learned my addr 172.21.15.26:0/2620419946 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c32d9d700 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 -- 172.21.15.26:0/2620419946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 msgr2=0x55c21b5f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 -- 172.21.15.26:0/2620419946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 msgr2=0x55c21b5f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.869+0000 7f5c3259c700 1 -- 172.21.15.26:0/2620419946 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c21b371e60 con 0x55c21b5f2550 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.870+0000 7f5c3259c700 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c240a2c60 tx=0x7f5c2409ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7bab1ed0505eb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.870+0000 7f5c20ff9700 1 -- 172.21.15.26:0/2620419946 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240b2070 con 0x55c21b5f2550 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.870+0000 7f5c20ff9700 1 -- 172.21.15.26:0/2620419946 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2409add0 con 0x55c21b5f2550 2023-07-13T22:24:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.870+0000 7f5c20ff9700 1 -- 172.21.15.26:0/2620419946 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003c80 con 0x55c21b5f2550 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/2620419946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 msgr2=0x55c21b5f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c240a2c60 tx=0x7f5c2409ac70 comp rx=0 tx=0).stop 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/2620419946 shutdown_connections 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5f2960 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/2620419946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/2620419946 >> 172.21.15.26:0/2620419946 conn(0x55c21b5e7e20 msgr2=0x55c21b5ed250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/2620419946 shutdown_connections 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.871+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/2620419946 wait complete. 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.872+0000 7f5c4c8d1b80 1 -- start start 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.872+0000 7f5c4c8d1b80 1 Processor -- start 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b4e3b40 con 0x55c21b5f2550 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b4e3840 con 0x55c21b5f3cf0 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b3c79a0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c4c8d1b80 10 client.0 _ll_register_callbacks cb 0x55c21b4e17e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c32d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c32d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39228/0 (socket says 172.21.15.26:39228) 2023-07-13T22:24:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.873+0000 7f5c32d9d700 1 -- 172.21.15.26:0/1633794807 learned_addr learned my addr 172.21.15.26:0/1633794807 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 -- 172.21.15.26:0/1633794807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 msgr2=0x55c21b5fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 -- 172.21.15.26:0/1633794807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 msgr2=0x55c21b5fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 -- 172.21.15.26:0/1633794807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3419bbf0 con 0x55c21b5f3cf0 2023-07-13T22:24:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c3259c700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:24:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c32d9d700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c011c90 tx=0x7f5c2c007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7db6d3c6e5652c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.874+0000 7f5c19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00f990 con 0x55c21b5f3cf0 2023-07-13T22:24:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c21b49d570 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c2c00faf0 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c2c00fcb0 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c4c8d1b80 10 client.27245 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.875+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c21b35ace0 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c21b49d570 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c2c015550 con 0x55c21b5f3cf0 2023-07-13T22:24:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 10 client.27245 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 20 client.27245 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b607d10 0x55c21b60a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 4 client.27245 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c21b53cf30 con 0x55c21b607d10 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.876+0000 7f5c3259c700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b607d10 0x55c21b60a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.877+0000 7f5c19ffb700 10 client.27245 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.877+0000 7f5c3259c700 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b607d10 0x55c21b60a120 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390b8cc16661447 server_cookie=d43dbfc8730073a7 in_seq=0 out_seq=0 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.877+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1633794807 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c21b53cf30 con 0x55c21b607d10 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.877+0000 7f5c19ffb700 10 client.27245 handle_command_reply: tid=0 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.877+0000 7f5c4c8d1b80 1 client.27245 shutdown 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 msgr2=0x55c21b603410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c011c90 tx=0x7f5c2c007ed0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 shutdown_connections 2023-07-13T22:24:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b607d10 0x55c21b60a120 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b603410 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1633794807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 >> 172.21.15.26:0/1633794807 conn(0x55c21b5e7e20 msgr2=0x55c21b5ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c19ffb700 0 client.27245 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 shutdown_connections 2023-07-13T22:24:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1633794807 wait complete. 2023-07-13T22:24:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.878+0000 7f5c4c8d1b80 20 client.27245 trim_cache size 0 max 16384 2023-07-13T22:24:35.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.911+0000 7f5c4c8d1b80 1 Processor -- start 2023-07-13T22:24:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.920+0000 7f5c4c8d1b80 1 -- start start 2023-07-13T22:24:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b6033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b53cf30 con 0x55c21b5f2550 2023-07-13T22:24:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b49d570 con 0x55c21b5f3cf0 2023-07-13T22:24:35.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.921+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b5f35f0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37466/0 (socket says 172.21.15.26:37466) 2023-07-13T22:24:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 -- 172.21.15.26:0/4220519667 learned_addr learned my addr 172.21.15.26:0/4220519667 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 -- 172.21.15.26:0/4220519667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 msgr2=0x55c21b6033d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:24:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b6033d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 -- 172.21.15.26:0/4220519667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 msgr2=0x55c21b5f71c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f71c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.922+0000 7f5c3259c700 1 -- 172.21.15.26:0/4220519667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c21b5f4390 con 0x55c21b5f2550 2023-07-13T22:24:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.923+0000 7f5c3259c700 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5c2409aad0 tx=0x7f5c24019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e470a8bb2eec3383 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.925+0000 7f5c217fa700 1 -- 172.21.15.26:0/4220519667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c240b2040 con 0x55c21b5f2550 2023-07-13T22:24:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.925+0000 7f5c217fa700 1 -- 172.21.15.26:0/4220519667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c240b9130 con 0x55c21b5f2550 2023-07-13T22:24:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.925+0000 7f5c217fa700 1 -- 172.21.15.26:0/4220519667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003ea0 con 0x55c21b5f2550 2023-07-13T22:24:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.926+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/4220519667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 msgr2=0x55c21b5f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.926+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5c2409aad0 tx=0x7f5c24019040 comp rx=0 tx=0).stop 2023-07-13T22:24:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.942+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/4220519667 shutdown_connections 2023-07-13T22:24:35.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.942+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f2550 0x55c21b5f6c80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.942+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f3cf0 0x55c21b5f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.942+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/4220519667 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2ea0 0x55c21b6033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.942+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/4220519667 >> 172.21.15.26:0/4220519667 conn(0x55c21b5e7e20 msgr2=0x55c21b603be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.945+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/4220519667 shutdown_connections 2023-07-13T22:24:35.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.945+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/4220519667 wait complete. 2023-07-13T22:24:35.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.946+0000 7f5c4c8d1b80 1 -- start start 2023-07-13T22:24:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.946+0000 7f5c4c8d1b80 1 Processor -- start 2023-07-13T22:24:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b602e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b603390 con 0x55c21b5f3cf0 2023-07-13T22:24:35.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b603500 con 0x55c21b5f2ea0 2023-07-13T22:24:35.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c21b6036c0 con 0x55c21b5f2550 2023-07-13T22:24:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.949+0000 7f5c4c8d1b80 10 client.0 _ll_register_callbacks cb 0x7f5c2c0115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.950+0000 7f5c3259c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.950+0000 7f5c31d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39236/0 (socket says 172.21.15.26:39236) 2023-07-13T22:24:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 -- 172.21.15.26:0/1078790139 learned_addr learned my addr 172.21.15.26:0/1078790139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c32d9d700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b602e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 -- 172.21.15.26:0/1078790139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 msgr2=0x55c21b5fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 -- 172.21.15.26:0/1078790139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 msgr2=0x55c21b602e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.952+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b602e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.953+0000 7f5c31d9b700 1 -- 172.21.15.26:0/1078790139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c21b371e60 con 0x55c21b5f2ea0 2023-07-13T22:24:35.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.953+0000 7f5c31d9b700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c2801e470 tx=0x7f5c28004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2823ed247cbce686 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:35.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.965+0000 7f5c19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:24:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.965+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28010430 con 0x55c21b5f2ea0 2023-07-13T22:24:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.965+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c28010590 con 0x55c21b5f2ea0 2023-07-13T22:24:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.965+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c2409add0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.972+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c28010750 con 0x55c21b5f2ea0 2023-07-13T22:24:35.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.973+0000 7f5c4c8d1b80 10 client.27248 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:35.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.973+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c21b3982d0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2409add0 con 0x55c21b5f2ea0 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5c2802e070 con 0x55c21b5f2ea0 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 10 client.27248 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 20 client.27248 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b66e3e0 0x55c21b6707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 4 client.27248 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:24:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.974+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c21b4ff0d0 con 0x55c21b66e3e0 2023-07-13T22:24:35.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.977+0000 7f5c3259c700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b66e3e0 0x55c21b6707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:35.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.979+0000 7f5c3259c700 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b66e3e0 0x55c21b6707f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13182d49642a08fc server_cookie=717b05ec41997883 in_seq=0 out_seq=0 2023-07-13T22:24:35.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.979+0000 7f5c19ffb700 10 client.27248 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.981+0000 7f5c19ffb700 1 -- 172.21.15.26:0/1078790139 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c21b4ff0d0 con 0x55c21b66e3e0 2023-07-13T22:24:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.981+0000 7f5c19ffb700 10 client.27248 handle_command_reply: tid=0 2023-07-13T22:24:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.981+0000 7f5c4c8d1b80 1 client.27248 shutdown 2023-07-13T22:24:35.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 msgr2=0x55c21b5fdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c2801e470 tx=0x7f5c28004750 comp rx=0 tx=0).stop 2023-07-13T22:24:35.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 shutdown_connections 2023-07-13T22:24:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c21b5f3cf0 0x55c21b602e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c21b66e3e0 0x55c21b6707f0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21b5f2550 0x55c21b5fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 --2- 172.21.15.26:0/1078790139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c21b5f2ea0 0x55c21b5fdf80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.983+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 >> 172.21.15.26:0/1078790139 conn(0x55c21b5e7e20 msgr2=0x55c21b5f6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.984+0000 7f5c19ffb700 0 client.27248 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.984+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 shutdown_connections 2023-07-13T22:24:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.984+0000 7f5c4c8d1b80 1 -- 172.21.15.26:0/1078790139 wait complete. 2023-07-13T22:24:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.986+0000 7f5c4c8d1b80 20 client.27248 trim_cache size 0 max 16384 2023-07-13T22:24:35.975 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:24:35.975 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:24:35.976 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "9feaf624-1608-45f0-b6f1-47bb65e7e54c", 2023-07-13T22:24:35.976 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:24:35.976 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:24:35.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.998+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 msgr2=0x7f5c1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.998+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 0x7f5c1c0a3680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005950 tx=0x7f5c280058e0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.998+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 msgr2=0x7f5c341935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:35.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.998+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009fd0 tx=0x7f5c2400b790 comp rx=0 tx=0).stop 2023-07-13T22:24:35.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 shutdown_connections 2023-07-13T22:24:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c34103890 0x7f5c3419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5c34101d10 0x7f5c341935e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5c1c0a11d0 0x7f5c1c0a3680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 --2- 172.21.15.26:0/2209504376 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c34102f00 0x7f5c34193b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:35.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:35.999+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 >> 172.21.15.26:0/2209504376 conn(0x7f5c340fd350 msgr2=0x7f5c340ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.000+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 shutdown_connections 2023-07-13T22:24:35.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.001+0000 7f5c38fff700 1 -- 172.21.15.26:0/2209504376 wait complete. 2023-07-13T22:24:36.050 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:24:36.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.723+0000 7f5f769ff700 1 -- 172.21.15.26:0/363382975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f7010bd60 msgr2=0x7f5f7010e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.723+0000 7f5f769ff700 1 --2- 172.21.15.26:0/363382975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f7010bd60 0x7f5f7010e1f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800b0a0 tx=0x7f5f6800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.731+0000 7f5f769ff700 1 -- 172.21.15.26:0/363382975 shutdown_connections 2023-07-13T22:24:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.731+0000 7f5f769ff700 1 --2- 172.21.15.26:0/363382975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f70070ee0 0x7f5f700712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.731+0000 7f5f769ff700 1 --2- 172.21.15.26:0/363382975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f7010bd60 0x7f5f7010e1f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.731+0000 7f5f769ff700 1 --2- 172.21.15.26:0/363382975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70071830 0x7f5f7010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.731+0000 7f5f769ff700 1 -- 172.21.15.26:0/363382975 >> 172.21.15.26:0/363382975 conn(0x7f5f7006ca90 msgr2=0x7f5f7006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:36.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.751+0000 7f5f769ff700 1 -- 172.21.15.26:0/363382975 shutdown_connections 2023-07-13T22:24:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.751+0000 7f5f769ff700 1 -- 172.21.15.26:0/363382975 wait complete. 2023-07-13T22:24:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.756+0000 7f5f769ff700 1 Processor -- start 2023-07-13T22:24:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.758+0000 7f5f769ff700 1 -- start start 2023-07-13T22:24:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.760+0000 7f5f769ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.760+0000 7f5f769ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f70071830 0x7f5f70113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.761+0000 7f5f769ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f7010bd60 0x7f5f701b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.761+0000 7f5f769ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70117040 con 0x7f5f7010bd60 2023-07-13T22:24:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.761+0000 7f5f769ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701171b0 con 0x7f5f70071830 2023-07-13T22:24:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.761+0000 7f5f769ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701b4e00 con 0x7f5f70070ee0 2023-07-13T22:24:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.766+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.766+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38526/0 (socket says 172.21.15.26:38526) 2023-07-13T22:24:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.766+0000 7f5f6ffff700 1 -- 172.21.15.26:0/4164246930 learned_addr learned my addr 172.21.15.26:0/4164246930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.770+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f70071830 0x7f5f70113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.783+0000 7f5f6ffff700 1 -- 172.21.15.26:0/4164246930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f70071830 msgr2=0x7f5f70113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.783+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f70071830 0x7f5f70113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.783+0000 7f5f6ffff700 1 -- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f7010bd60 msgr2=0x7f5f701b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.783+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f7010bd60 0x7f5f701b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.783+0000 7f5f6ffff700 1 -- 172.21.15.26:0/4164246930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f68009d00 con 0x7f5f70070ee0 2023-07-13T22:24:36.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.784+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b530 tx=0x7f5f6000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.803+0000 7f5f6d7fa700 1 -- 172.21.15.26:0/4164246930 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60011830 con 0x7f5f70070ee0 2023-07-13T22:24:36.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.803+0000 7f5f6d7fa700 1 -- 172.21.15.26:0/4164246930 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f60011e50 con 0x7f5f70070ee0 2023-07-13T22:24:36.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.804+0000 7f5f6d7fa700 1 -- 172.21.15.26:0/4164246930 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60010e70 con 0x7f5f70070ee0 2023-07-13T22:24:36.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.808+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f701b4fa0 con 0x7f5f70070ee0 2023-07-13T22:24:36.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.808+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f701b5480 con 0x7f5f70070ee0 2023-07-13T22:24:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.815+0000 7f5f8a245b80 1 Processor -- start 2023-07-13T22:24:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.815+0000 7f5f8a245b80 1 -- start start 2023-07-13T22:24:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67ee18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 0x560c67ee26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 0x560c67ee30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67de6cb0 con 0x560c67ee14d0 2023-07-13T22:24:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67e39a40 con 0x560c67ee1e20 2023-07-13T22:24:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.816+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67e32ed0 con 0x560c67ee2c70 2023-07-13T22:24:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.839+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 0x560c67ee26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.839+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 0x560c67ee26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39258/0 (socket says 172.21.15.26:39258) 2023-07-13T22:24:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.839+0000 7f5f6ffff700 1 -- 172.21.15.26:0/2165708549 learned_addr learned my addr 172.21.15.26:0/2165708549 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:36.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.839+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67ee18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.843+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 0x560c67ee30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.845+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/2165708549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 msgr2=0x560c67ee26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.845+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 0x560c67ee26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.845+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/2165708549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 msgr2=0x560c67ee18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.845+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67ee18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.845+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/2165708549 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c67c60de0 con 0x560c67ee2c70 2023-07-13T22:24:36.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.846+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 0x560c67ee30e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009bd0 tx=0x7f5f64009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45586133c54ac11d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.849+0000 7f5f6d7fa700 1 -- 172.21.15.26:0/4164246930 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f60010510 con 0x7f5f70070ee0 2023-07-13T22:24:36.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.851+0000 7f5f6d7fa700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 0x7f5f580a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.851+0000 7f5f567fc700 1 -- 172.21.15.26:0/2165708549 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64011bf0 con 0x560c67ee2c70 2023-07-13T22:24:36.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.852+0000 7f5f567fc700 1 -- 172.21.15.26:0/2165708549 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f64011d50 con 0x560c67ee2c70 2023-07-13T22:24:36.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.852+0000 7f5f567fc700 1 -- 172.21.15.26:0/2165708549 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400f7c0 con 0x560c67ee2c70 2023-07-13T22:24:36.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.852+0000 7f5f6d7fa700 1 -- 172.21.15.26:0/4164246930 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f600ce210 con 0x7f5f70070ee0 2023-07-13T22:24:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 -- 172.21.15.26:0/2165708549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 msgr2=0x560c67ee30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 0x560c67ee30e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009bd0 tx=0x7f5f64009ee0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 -- 172.21.15.26:0/2165708549 shutdown_connections 2023-07-13T22:24:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67ee18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee2c70 0x560c67ee30e0 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009bd0 tx=0x7f5f64009ee0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/2165708549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee1e20 0x560c67ee26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.854+0000 7f5f8a245b80 1 -- 172.21.15.26:0/2165708549 >> 172.21.15.26:0/2165708549 conn(0x560c67ed6da0 msgr2=0x560c67edc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:36.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.872+0000 7f5f8a245b80 1 -- 172.21.15.26:0/2165708549 shutdown_connections 2023-07-13T22:24:36.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.882+0000 7f5f8a245b80 1 -- 172.21.15.26:0/2165708549 wait complete. 2023-07-13T22:24:36.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.882+0000 7f5f8a245b80 1 -- start start 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.883+0000 7f5f8a245b80 1 Processor -- start 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67eeb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebaa0 0x560c67ef0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 0x560c67ef3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67dd2ac0 con 0x560c67ee14d0 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67dd27c0 con 0x560c67eebaa0 2023-07-13T22:24:36.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67cb6920 con 0x560c67eebf10 2023-07-13T22:24:36.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.884+0000 7f5f8a245b80 10 client.0 _ll_register_callbacks cb 0x560c67dd0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:36.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.887+0000 7f5f74f9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 0x560c67ef3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.887+0000 7f5f74f9c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 0x560c67ef3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:38540/0 (socket says 172.21.15.26:38540) 2023-07-13T22:24:36.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.887+0000 7f5f74f9c700 1 -- 172.21.15.26:0/779483199 learned_addr learned my addr 172.21.15.26:0/779483199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:36.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.888+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 0x7f5f580a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.893+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67eeb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.896+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebaa0 0x560c67ef0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.897+0000 7f5f6ffff700 1 -- 172.21.15.26:0/779483199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 msgr2=0x560c67ef3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.897+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 0x560c67ef3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.897+0000 7f5f6ffff700 1 -- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebaa0 msgr2=0x560c67ef0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.897+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebaa0 0x560c67ef0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.897+0000 7f5f6ffff700 1 -- 172.21.15.26:0/779483199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f64009830 con 0x560c67ee14d0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.898+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67eeb560 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f60097ad0 tx=0x7f5f60097990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d393fa45e1f4b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.898+0000 7f5f4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.898+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6009b110 con 0x560c67ee14d0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.899+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f6009b5d0 con 0x560c67ee14d0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.899+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c67d8c4f0 con 0x560c67ee14d0 2023-07-13T22:24:36.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.899+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f6009af30 con 0x560c67ee14d0 2023-07-13T22:24:36.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.899+0000 7f5f8a245b80 10 client.16641 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:36.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.899+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c67c49c60 con 0x560c67ee14d0 2023-07-13T22:24:36.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.900+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c67d8c4f0 con 0x560c67ee14d0 2023-07-13T22:24:36.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.900+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f6009b830 con 0x560c67ee14d0 2023-07-13T22:24:36.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.900+0000 7f5f8a245b80 10 client.16641 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:36.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.900+0000 7f5f8a245b80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.901+0000 7f5f8a245b80 20 client.16641 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.901+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67ef7aa0 0x560c67ef9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.901+0000 7f5f8a245b80 4 client.16641 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:24:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.901+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c67e2beb0 con 0x560c67ef7aa0 2023-07-13T22:24:36.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.902+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67ef7aa0 0x560c67ef9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.905+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67ef7aa0 0x560c67ef9eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0438cb6648a8030 server_cookie=ccdcb724a98800ad in_seq=0 out_seq=0 2023-07-13T22:24:36.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.905+0000 7f5f4f7fe700 10 client.16641 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.912+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/779483199 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c67e2beb0 con 0x560c67ef7aa0 2023-07-13T22:24:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.912+0000 7f5f4f7fe700 10 client.16641 handle_command_reply: tid=0 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.912+0000 7f5f8a245b80 1 client.16641 shutdown 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.912+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 msgr2=0x560c67eeb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.912+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67eeb560 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f60097ad0 tx=0x7f5f60097990 comp rx=0 tx=0).stop 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 shutdown_connections 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67ee14d0 0x560c67eeb560 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67ef7aa0 0x560c67ef9eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebf10 0x560c67ef3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/779483199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebaa0 0x560c67ef0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.921+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 >> 172.21.15.26:0/779483199 conn(0x560c67ed6da0 msgr2=0x560c67edc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.928+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 0x7f5f580a3800 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x560c67eec610 tx=0x7f5f6800b700 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.928+0000 7f5f4f7fe700 0 client.16641 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.938+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 shutdown_connections 2023-07-13T22:24:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.941+0000 7f5f8a245b80 1 -- 172.21.15.26:0/779483199 wait complete. 2023-07-13T22:24:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.941+0000 7f5f8a245b80 20 client.16641 trim_cache size 0 max 16384 2023-07-13T22:24:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.978+0000 7f5f8a245b80 1 Processor -- start 2023-07-13T22:24:36.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.981+0000 7f5f8a245b80 1 -- start start 2023-07-13T22:24:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee14d0 0x560c67ee93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebf10 0x560c67eeb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67edd4c0 con 0x560c67eebaa0 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67eeaa10 con 0x560c67eebf10 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.982+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67ee21f0 con 0x560c67ee14d0 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37534/0 (socket says 172.21.15.26:37534) 2023-07-13T22:24:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/316830485 learned_addr learned my addr 172.21.15.26:0/316830485 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/316830485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee14d0 msgr2=0x560c67ee93d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee14d0 0x560c67ee93d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/316830485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebf10 msgr2=0x560c67eeb310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:24:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebf10 0x560c67eeb310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.984+0000 7f5f6f7fe700 1 -- 172.21.15.26:0/316830485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c67c87250 con 0x560c67eebaa0 2023-07-13T22:24:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.985+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x560c67ee2610 tx=0x7f5f64014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420fb12ea0354d4d server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.986+0000 7f5f56ffd700 1 -- 172.21.15.26:0/316830485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64029040 con 0x560c67eebaa0 2023-07-13T22:24:36.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.986+0000 7f5f56ffd700 1 -- 172.21.15.26:0/316830485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f6402b860 con 0x560c67eebaa0 2023-07-13T22:24:36.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.986+0000 7f5f56ffd700 1 -- 172.21.15.26:0/316830485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64005510 con 0x560c67eebaa0 2023-07-13T22:24:36.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.987+0000 7f5f8a245b80 1 -- 172.21.15.26:0/316830485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 msgr2=0x560c67ef0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.987+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x560c67ee2610 tx=0x7f5f64014040 comp rx=0 tx=0).stop 2023-07-13T22:24:36.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.989+0000 7f5f8a245b80 1 -- 172.21.15.26:0/316830485 shutdown_connections 2023-07-13T22:24:36.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.989+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67eebf10 0x560c67eeb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.989+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67ee14d0 0x560c67ee93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.990+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/316830485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebaa0 0x560c67ef0a90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.990+0000 7f5f8a245b80 1 -- 172.21.15.26:0/316830485 >> 172.21.15.26:0/316830485 conn(0x560c67ed6da0 msgr2=0x560c67efe850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.990+0000 7f5f8a245b80 1 -- 172.21.15.26:0/316830485 shutdown_connections 2023-07-13T22:24:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.990+0000 7f5f8a245b80 1 -- 172.21.15.26:0/316830485 wait complete. 2023-07-13T22:24:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.991+0000 7f5f8a245b80 1 -- start start 2023-07-13T22:24:36.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.991+0000 7f5f8a245b80 1 Processor -- start 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 0x560c67ef0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebaa0 0x560c67eef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 0x560c67eef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67eefda0 con 0x560c67eebf10 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67eefee0 con 0x560c67ee14d0 2023-07-13T22:24:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c67f4b990 con 0x560c67eebaa0 2023-07-13T22:24:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.992+0000 7f5f8a245b80 10 client.0 _ll_register_callbacks cb 0x7f5f60009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:24:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.997+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 0x560c67ef0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.997+0000 7f5f6ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 0x560c67ef0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:39280/0 (socket says 172.21.15.26:39280) 2023-07-13T22:24:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.997+0000 7f5f6ffff700 1 -- 172.21.15.26:0/1127063611 learned_addr learned my addr 172.21.15.26:0/1127063611 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:24:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.997+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 0x560c67eef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:36.997+0000 7f5f6f7fe700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebaa0 0x560c67eef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.003+0000 7f5f74f9c700 1 -- 172.21.15.26:0/1127063611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebaa0 msgr2=0x560c67eef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.003+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebaa0 0x560c67eef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.003+0000 7f5f74f9c700 1 -- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 msgr2=0x560c67ef0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:24:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.003+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 0x560c67ef0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.003+0000 7f5f74f9c700 1 -- 172.21.15.26:0/1127063611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c67c60de0 con 0x560c67eebf10 2023-07-13T22:24:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.004+0000 7f5f74f9c700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 0x560c67eef6c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009fd0 tx=0x7f5f680099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e33a374e7cb8e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:24:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.004+0000 7f5f4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:24:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800e070 con 0x560c67eebf10 2023-07-13T22:24:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5f68005640 con 0x560c67eebf10 2023-07-13T22:24:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f64011d50 con 0x560c67eebf10 2023-07-13T22:24:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f68005800 con 0x560c67eebf10 2023-07-13T22:24:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f8a245b80 10 client.16647 fetch_fsmap learned FSMap version 29 2023-07-13T22:24:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.005+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c67c5f2b0 con 0x560c67eebf10 2023-07-13T22:24:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.006+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f64011d50 con 0x560c67eebf10 2023-07-13T22:24:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.006+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5f6801fca0 con 0x560c67eebf10 2023-07-13T22:24:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:24:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:24:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 20 client.16647 populate_metadata read hostname 'smithi026' 2023-07-13T22:24:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67f4d2e0 0x560c67f4f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:24:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 4 client.16647 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:24:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.007+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c67dee050 con 0x560c67f4d2e0 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.008+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67f4d2e0 0x560c67f4f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f6ffff700 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67f4d2e0 0x560c67f4f6f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7692ee65f678dd5b server_cookie=cb9d25b945bdf3d2 in_seq=0 out_seq=0 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f4f7fe700 10 client.16647 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f4f7fe700 1 -- 172.21.15.26:0/1127063611 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560c67dee050 con 0x560c67f4d2e0 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f4f7fe700 10 client.16647 handle_command_reply: tid=0 2023-07-13T22:24:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f8a245b80 1 client.16647 shutdown 2023-07-13T22:24:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 msgr2=0x560c67eef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.009+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 0x560c67eef6c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009fd0 tx=0x7f5f680099e0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 shutdown_connections 2023-07-13T22:24:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560c67eebf10 0x560c67eef6c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560c67f4d2e0 0x560c67f4f6f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c67ee14d0 0x560c67ef0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 --2- 172.21.15.26:0/1127063611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c67eebaa0 0x560c67eef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 >> 172.21.15.26:0/1127063611 conn(0x560c67ed6da0 msgr2=0x560c67eeb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.010+0000 7f5f4f7fe700 0 client.16647 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:24:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.032+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 shutdown_connections 2023-07-13T22:24:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.032+0000 7f5f8a245b80 1 -- 172.21.15.26:0/1127063611 wait complete. 2023-07-13T22:24:37.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.033+0000 7f5f8a245b80 20 client.16647 trim_cache size 0 max 16384 2023-07-13T22:24:37.015 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:24:37.015 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: "9feaf624-1608-45f0-b6f1-47bb65e7e54c": { 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "9feaf624-1608-45f0-b6f1-47bb65e7e54c", 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:24:37.016 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:24:37.017 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:24:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.040+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 msgr2=0x7f5f580a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.040+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 0x7f5f580a3800 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x560c67eec610 tx=0x7f5f6800b700 comp rx=0 tx=0).stop 2023-07-13T22:24:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.040+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 msgr2=0x7f5f70113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:24:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.040+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b530 tx=0x7f5f6000b8f0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 shutdown_connections 2023-07-13T22:24:37.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5f580a1350 0x7f5f580a3800 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f7010bd60 0x7f5f701b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f70070ee0 0x7f5f70113a20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 --2- 172.21.15.26:0/4164246930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f70071830 0x7f5f70113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:24:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 >> 172.21.15.26:0/4164246930 conn(0x7f5f7006ca90 msgr2=0x7f5f7006dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:24:37.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 shutdown_connections 2023-07-13T22:24:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:24:37.042+0000 7f5f769ff700 1 -- 172.21.15.26:0/4164246930 wait complete. 2023-07-13T22:24:37.081 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:36 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:24:37.081 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:36 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:37.082 INFO:tasks.cephfs.filesystem:scrub status for tag:9feaf624-1608-45f0-b6f1-47bb65e7e54c - {'path': '/', 'tag': '9feaf624-1608-45f0-b6f1-47bb65e7e54c', 'options': 'recursive,force'} 2023-07-13T22:24:37.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:36 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:24:37.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:36 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:37.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:36 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:24:37.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:36 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:37 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:24:38.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:37 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:37 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:24:38.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:37 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:37 smithi117 ceph-mon[129074]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 90 op/s 2023-07-13T22:24:38.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:37 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:24:38.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:37 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:37 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:24:38.197 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:37 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:37 smithi026 ceph-mon[95647]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 90 op/s 2023-07-13T22:24:38.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:37 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:24:38.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:37 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:37 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:24:38.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:37 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:24:38.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:37 smithi121 ceph-mon[129187]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 279 MiB/s wr, 90 op/s 2023-07-13T22:24:40.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:39 smithi117 ceph-mon[129074]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 312 MiB/s wr, 102 op/s 2023-07-13T22:24:40.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:39 smithi026 ceph-mon[95647]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 312 MiB/s wr, 102 op/s 2023-07-13T22:24:40.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:39 smithi121 ceph-mon[129187]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 312 MiB/s wr, 102 op/s 2023-07-13T22:24:42.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:41 smithi117 ceph-mon[129074]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 93 op/s 2023-07-13T22:24:42.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:41 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:24:42.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:41 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:24:42.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:41 smithi026 ceph-mon[95647]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 93 op/s 2023-07-13T22:24:42.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:41 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:24:42.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:41 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:24:42.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:41 smithi121 ceph-mon[129187]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 295 MiB/s wr, 93 op/s 2023-07-13T22:24:42.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:41 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:24:42.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:41 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:24:44.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:43 smithi117 ceph-mon[129074]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:24:44.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:43 smithi026 ceph-mon[95647]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:24:44.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:43 smithi121 ceph-mon[129187]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 275 MiB/s wr, 89 op/s 2023-07-13T22:24:45.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:44 smithi121 ceph-mon[129187]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 100 op/s 2023-07-13T22:24:45.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:44 smithi117 ceph-mon[129074]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 100 op/s 2023-07-13T22:24:45.456 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:44 smithi026 ceph-mon[95647]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 317 MiB/s wr, 100 op/s 2023-07-13T22:24:48.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:47 smithi117 ceph-mon[129074]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:24:48.205 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:47 smithi026 ceph-mon[95647]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:24:48.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:47 smithi121 ceph-mon[129187]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 284 MiB/s wr, 91 op/s 2023-07-13T22:24:50.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:49 smithi117 ceph-mon[129074]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 329 MiB/s wr, 108 op/s 2023-07-13T22:24:50.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:49 smithi026 ceph-mon[95647]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 329 MiB/s wr, 108 op/s 2023-07-13T22:24:50.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:49 smithi121 ceph-mon[129187]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 329 MiB/s wr, 108 op/s 2023-07-13T22:24:52.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:51 smithi117 ceph-mon[129074]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:52.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:51 smithi026 ceph-mon[95647]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:52.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:51 smithi121 ceph-mon[129187]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 313 MiB/s wr, 100 op/s 2023-07-13T22:24:54.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:53 smithi117 ceph-mon[129074]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-07-13T22:24:54.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:53 smithi026 ceph-mon[95647]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-07-13T22:24:54.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:53 smithi121 ceph-mon[129187]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 292 MiB/s wr, 96 op/s 2023-07-13T22:24:55.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:54 smithi121 ceph-mon[129187]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 329 MiB/s wr, 106 op/s 2023-07-13T22:24:55.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:54 smithi117 ceph-mon[129074]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 329 MiB/s wr, 106 op/s 2023-07-13T22:24:55.450 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:54 smithi026 ceph-mon[95647]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 329 MiB/s wr, 106 op/s 2023-07-13T22:24:58.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:57 smithi117 ceph-mon[129074]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 96 op/s 2023-07-13T22:24:58.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:57 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:24:58.203 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:57 smithi026 ceph-mon[95647]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 96 op/s 2023-07-13T22:24:58.204 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:57 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:24:58.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:57 smithi121 ceph-mon[129187]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 299 MiB/s wr, 96 op/s 2023-07-13T22:24:58.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:57 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:25:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:59 smithi117 ceph-mon[129074]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 340 MiB/s wr, 111 op/s 2023-07-13T22:25:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:24:59 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.211 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:59 smithi026 ceph-mon[95647]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 340 MiB/s wr, 111 op/s 2023-07-13T22:25:00.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.212 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:24:59 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:59 smithi121 ceph-mon[129187]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 340 MiB/s wr, 111 op/s 2023-07-13T22:25:00.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:00.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:24:59 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:02.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:01 smithi117 ceph-mon[129074]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 99 op/s 2023-07-13T22:25:02.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:01 smithi026 ceph-mon[95647]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 99 op/s 2023-07-13T22:25:02.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:01 smithi121 ceph-mon[129187]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 311 MiB/s wr, 99 op/s 2023-07-13T22:25:04.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:03 smithi117 ceph-mon[129074]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 95 op/s 2023-07-13T22:25:04.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:03 smithi026 ceph-mon[95647]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 95 op/s 2023-07-13T22:25:04.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:03 smithi121 ceph-mon[129187]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 289 MiB/s wr, 95 op/s 2023-07-13T22:25:06.198 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:05 smithi026 ceph-mon[95647]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:25:06.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:06.199 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:05 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:06.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:05 smithi121 ceph-mon[129187]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:25:06.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:06.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:05 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:06.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:05 smithi117 ceph-mon[129074]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 327 MiB/s wr, 106 op/s 2023-07-13T22:25:06.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:06.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:05 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:07.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:25:07.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.889+0000 7fb3d1dff700 1 -- 172.21.15.26:0/875805935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 msgr2=0x7fb3cc10e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.889+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/875805935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc10e1f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b0a0 tx=0x7fb3c400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.890+0000 7fb3d1dff700 1 -- 172.21.15.26:0/875805935 shutdown_connections 2023-07-13T22:25:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.890+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/875805935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc10e1f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.890+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/875805935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.890+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/875805935 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc071830 0x7fb3cc10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.890+0000 7fb3d1dff700 1 -- 172.21.15.26:0/875805935 >> 172.21.15.26:0/875805935 conn(0x7fb3cc06ca90 msgr2=0x7fb3cc06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.891+0000 7fb3d1dff700 1 -- 172.21.15.26:0/875805935 shutdown_connections 2023-07-13T22:25:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.901+0000 7fb3d1dff700 1 -- 172.21.15.26:0/875805935 wait complete. 2023-07-13T22:25:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.902+0000 7fb3d1dff700 1 Processor -- start 2023-07-13T22:25:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 -- start start 2023-07-13T22:25:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc071830 0x7fb3cc1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc1abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc1ac300 con 0x7fb3cc071830 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc10ec40 con 0x7fb3cc070ee0 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.903+0000 7fb3d1dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3cc10edd0 con 0x7fb3cc10bd60 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.904+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.904+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc1a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55956/0 (socket says 172.21.15.26:55956) 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.904+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/921659962 learned_addr learned my addr 172.21.15.26:0/921659962 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc1abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 -- 172.21.15.26:0/921659962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 msgr2=0x7fb3cc1a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc1a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 -- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc071830 msgr2=0x7fb3cc1a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc071830 0x7fb3cc1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.905+0000 7fb3cbfff700 1 -- 172.21.15.26:0/921659962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4009d00 con 0x7fb3cc10bd60 2023-07-13T22:25:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.911+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc1abd30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b070 tx=0x7fb3c4004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.924+0000 7fb3c8ff9700 1 -- 172.21.15.26:0/921659962 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400e040 con 0x7fb3cc10bd60 2023-07-13T22:25:07.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.925+0000 7fb3c8ff9700 1 -- 172.21.15.26:0/921659962 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c401bb20 con 0x7fb3cc10bd60 2023-07-13T22:25:07.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.925+0000 7fb3c8ff9700 1 -- 172.21.15.26:0/921659962 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005690 con 0x7fb3cc10bd60 2023-07-13T22:25:07.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.927+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc10f050 con 0x7fb3cc10bd60 2023-07-13T22:25:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.927+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3cc10f510 con 0x7fb3cc10bd60 2023-07-13T22:25:07.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.934+0000 7fb3e56f2b80 1 Processor -- start 2023-07-13T22:25:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 -- start start 2023-07-13T22:25:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4b4d0 0x55c946a4b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4cc70 0x55c946a4d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946950cb0 con 0x55c946a4b4d0 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9469a3a40 con 0x55c946a4cc70 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.935+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94699ced0 con 0x55c946a4be20 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.938+0000 7fb3caffd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4b4d0 0x55c946a4b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.938+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.939+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47090/0 (socket says 172.21.15.26:47090) 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.939+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/4261413105 learned_addr learned my addr 172.21.15.26:0/4261413105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3caffd700 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4cc70 0x55c946a4d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/4261413105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4cc70 msgr2=0x55c946a4d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4cc70 0x55c946a4d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/4261413105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4b4d0 msgr2=0x55c946a4b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4b4d0 0x55c946a4b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.940+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/4261413105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9467cade0 con 0x55c946a4be20 2023-07-13T22:25:07.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.944+0000 7fb3c8ff9700 1 -- 172.21.15.26:0/921659962 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3c401bc90 con 0x7fb3cc10bd60 2023-07-13T22:25:07.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.950+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bea0 tx=0x7fb3c00106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f2e22f59f6c604 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.956+0000 7fb3b1ffb700 1 -- 172.21.15.26:0/4261413105 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0011440 con 0x55c946a4be20 2023-07-13T22:25:07.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.956+0000 7fb3b1ffb700 1 -- 172.21.15.26:0/4261413105 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c0015070 con 0x55c946a4be20 2023-07-13T22:25:07.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.956+0000 7fb3b1ffb700 1 -- 172.21.15.26:0/4261413105 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0019900 con 0x55c946a4be20 2023-07-13T22:25:07.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.957+0000 7fb3c8ff9700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 0x7fb3b40a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.958+0000 7fb3c8ff9700 1 -- 172.21.15.26:0/921659962 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3c40cdf10 con 0x7fb3cc10bd60 2023-07-13T22:25:07.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.960+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/4261413105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 msgr2=0x55c946a4c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:07.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.960+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bea0 tx=0x7fb3c00106b0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/4261413105 shutdown_connections 2023-07-13T22:25:07.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4b4d0 0x55c946a4b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4cc70 0x55c946a4d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/4261413105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a4c6e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/4261413105 >> 172.21.15.26:0/4261413105 conn(0x55c946a40da0 msgr2=0x55c946a461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:07.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/4261413105 shutdown_connections 2023-07-13T22:25:07.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.961+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/4261413105 wait complete. 2023-07-13T22:25:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.963+0000 7fb3e56f2b80 1 -- start start 2023-07-13T22:25:07.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.963+0000 7fb3e56f2b80 1 Processor -- start 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a5c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94693cac0 con 0x55c946a4be20 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c94693c7c0 con 0x55c946a4b4d0 2023-07-13T22:25:07.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946820920 con 0x55c946a4cc70 2023-07-13T22:25:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.964+0000 7fb3e56f2b80 10 client.0 _ll_register_callbacks cb 0x55c94693a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.965+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55982/0 (socket says 172.21.15.26:55982) 2023-07-13T22:25:07.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2405131881 learned_addr learned my addr 172.21.15.26:0/2405131881 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a5c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2405131881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 msgr2=0x55c946a5c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a5c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2405131881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 msgr2=0x55c946a556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:07.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.966+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2405131881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c000bad0 con 0x55c946a4b4d0 2023-07-13T22:25:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.967+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000c00 tx=0x7fb3c0010f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821084fa8cd4f0de server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.968+0000 7fb3aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.968+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0005470 con 0x55c946a4b4d0 2023-07-13T22:25:07.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.968+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3c00055d0 con 0x55c946a4b4d0 2023-07-13T22:25:07.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.968+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9468f64f0 con 0x55c946a4b4d0 2023-07-13T22:25:07.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.979+0000 7fb3caffd700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 0x7fb3b40a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.979+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c0004750 con 0x55c946a4b4d0 2023-07-13T22:25:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.988+0000 7fb3caffd700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 0x7fb3b40a3800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005d90 tx=0x7fb3bc005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:07.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.999+0000 7fb3e56f2b80 10 client.27275 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:07.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:07.999+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9467b3c60 con 0x55c946a4b4d0 2023-07-13T22:25:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.010+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9468f64f0 con 0x55c946a4b4d0 2023-07-13T22:25:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.011+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3c0021070 con 0x55c946a4b4d0 2023-07-13T22:25:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 10 client.27275 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 10 client.27275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 20 client.27275 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946a60e00 0x55c946a63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 4 client.27275 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.012+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c946995eb0 con 0x55c946a60e00 2023-07-13T22:25:08.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.040+0000 7fb3caffd700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946a60e00 0x55c946a63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.050+0000 7fb3caffd700 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946a60e00 0x55c946a63210 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89ffccf3ca00d0d server_cookie=423caffbb399dbbd in_seq=0 out_seq=0 2023-07-13T22:25:08.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.050+0000 7fb3aaffd700 10 client.27275 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.050+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2405131881 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c946995eb0 con 0x55c946a60e00 2023-07-13T22:25:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.050+0000 7fb3aaffd700 10 client.27275 handle_command_reply: tid=0 2023-07-13T22:25:08.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.051+0000 7fb3e56f2b80 1 client.27275 shutdown 2023-07-13T22:25:08.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.052+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 msgr2=0x55c946a551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.052+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0000c00 tx=0x7fb3c0010f70 comp rx=0 tx=0).stop 2023-07-13T22:25:08.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 shutdown_connections 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946a60e00 0x55c946a63210 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a5c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a551b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2405131881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 >> 172.21.15.26:0/2405131881 conn(0x55c946a40da0 msgr2=0x55c946a458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.070+0000 7fb3aaffd700 0 client.27275 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:08.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.071+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 shutdown_connections 2023-07-13T22:25:08.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.072+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2405131881 wait complete. 2023-07-13T22:25:08.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.075+0000 7fb3e56f2b80 20 client.27275 trim_cache size 0 max 16384 2023-07-13T22:25:08.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.131+0000 7fb3e56f2b80 1 Processor -- start 2023-07-13T22:25:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.131+0000 7fb3e56f2b80 1 -- start start 2023-07-13T22:25:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a5c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a45fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a54d80 con 0x55c946a4be20 2023-07-13T22:25:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a4c1f0 con 0x55c946a4b4d0 2023-07-13T22:25:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.132+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a40970 con 0x55c946a4cc70 2023-07-13T22:25:08.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.133+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.133+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a67b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:55998/0 (socket says 172.21.15.26:55998) 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.133+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/1235477545 learned_addr learned my addr 172.21.15.26:0/1235477545 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.133+0000 7fb3caffd700 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a5c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.134+0000 7fb3caffd700 1 -- 172.21.15.26:0/1235477545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 msgr2=0x55c946a45fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.134+0000 7fb3caffd700 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a45fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.134+0000 7fb3caffd700 1 -- 172.21.15.26:0/1235477545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 msgr2=0x55c946a67b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.134+0000 7fb3caffd700 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a67b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.134+0000 7fb3caffd700 1 -- 172.21.15.26:0/1235477545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9467f1250 con 0x55c946a4be20 2023-07-13T22:25:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.135+0000 7fb3caffd700 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a5c650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55c946a63af0 tx=0x7fb3bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c227a3d5e84ef3db server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.136+0000 7fb3b27fc700 1 -- 172.21.15.26:0/1235477545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00c8a0 con 0x55c946a4be20 2023-07-13T22:25:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.136+0000 7fb3b27fc700 1 -- 172.21.15.26:0/1235477545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3bc024040 con 0x55c946a4be20 2023-07-13T22:25:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.136+0000 7fb3b27fc700 1 -- 172.21.15.26:0/1235477545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc004ca0 con 0x55c946a4be20 2023-07-13T22:25:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.137+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/1235477545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 msgr2=0x55c946a5c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.137+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a5c650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55c946a63af0 tx=0x7fb3bc013000 comp rx=0 tx=0).stop 2023-07-13T22:25:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.137+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/1235477545 shutdown_connections 2023-07-13T22:25:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.137+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4cc70 0x55c946a45fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.137+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a67b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.138+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/1235477545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4be20 0x55c946a5c650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.138+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/1235477545 >> 172.21.15.26:0/1235477545 conn(0x55c946a40da0 msgr2=0x55c946a53ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.138+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/1235477545 shutdown_connections 2023-07-13T22:25:08.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.138+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/1235477545 wait complete. 2023-07-13T22:25:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.139+0000 7fb3e56f2b80 1 -- start start 2023-07-13T22:25:08.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.139+0000 7fb3e56f2b80 1 Processor -- start 2023-07-13T22:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a58a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4cc70 0x55c946a5ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a5bf90 con 0x55c946a4cc70 2023-07-13T22:25:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a5c100 con 0x55c946a4b4d0 2023-07-13T22:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c946a5c2c0 con 0x55c946a4be20 2023-07-13T22:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.141+0000 7fb3e56f2b80 10 client.0 _ll_register_callbacks cb 0x7fb3c000bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.142+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.142+0000 7fb3cb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56010/0 (socket says 172.21.15.26:56010) 2023-07-13T22:25:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.142+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2454133831 learned_addr learned my addr 172.21.15.26:0/2454133831 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:08.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.143+0000 7fb3cbfff700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4cc70 0x55c946a5ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2454133831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 msgr2=0x55c946a58a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:25:08.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a58a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2454133831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4cc70 msgr2=0x55c946a5ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:08.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4cc70 0x55c946a5ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 -- 172.21.15.26:0/2454133831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc010940 con 0x55c946a4b4d0 2023-07-13T22:25:08.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3cb7fe700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bdc0 tx=0x7fb3c0005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c31a5364de5664c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:08.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.144+0000 7fb3aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.145+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000b580 con 0x55c946a4b4d0 2023-07-13T22:25:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.145+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb3c0005680 con 0x55c946a4b4d0 2023-07-13T22:25:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.145+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c0015070 con 0x55c946a4b4d0 2023-07-13T22:25:08.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.145+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c00193d0 con 0x55c946a4b4d0 2023-07-13T22:25:08.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.146+0000 7fb3e56f2b80 10 client.27281 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:08.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.146+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c9467c92b0 con 0x55c946a4b4d0 2023-07-13T22:25:08.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.147+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3c0015070 con 0x55c946a4b4d0 2023-07-13T22:25:08.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.147+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3c0021070 con 0x55c946a4b4d0 2023-07-13T22:25:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 10 client.27281 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 20 client.27281 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946ab5a40 0x55c946ab7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 4 client.27281 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c946958050 con 0x55c946ab5a40 2023-07-13T22:25:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.148+0000 7fb3caffd700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946ab5a40 0x55c946ab7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:08.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.150+0000 7fb3caffd700 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946ab5a40 0x55c946ab7e50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc855f00be2ece73 server_cookie=bbf2900ff3d8a29a in_seq=0 out_seq=0 2023-07-13T22:25:08.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.151+0000 7fb3aaffd700 10 client.27281 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.151+0000 7fb3aaffd700 1 -- 172.21.15.26:0/2454133831 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c946958050 con 0x55c946ab5a40 2023-07-13T22:25:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.151+0000 7fb3aaffd700 10 client.27281 handle_command_reply: tid=0 2023-07-13T22:25:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.151+0000 7fb3e56f2b80 1 client.27281 shutdown 2023-07-13T22:25:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.153+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 msgr2=0x55c946a584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.153+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000bdc0 tx=0x7fb3c0005c20 comp rx=0 tx=0).stop 2023-07-13T22:25:08.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 shutdown_connections 2023-07-13T22:25:08.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55c946a4cc70 0x55c946a5ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55c946ab5a40 0x55c946ab7e50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c946a4b4d0 0x55c946a584f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 --2- 172.21.15.26:0/2454133831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c946a4be20 0x55c946a58a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 >> 172.21.15.26:0/2454133831 conn(0x55c946a40da0 msgr2=0x55c946a62e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.154+0000 7fb3aaffd700 0 client.27281 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.157+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 shutdown_connections 2023-07-13T22:25:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.158+0000 7fb3e56f2b80 1 -- 172.21.15.26:0/2454133831 wait complete. 2023-07-13T22:25:08.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.159+0000 7fb3e56f2b80 20 client.27281 trim_cache size 0 max 16384 2023-07-13T22:25:08.143 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:08.143 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:25:08.143 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:25:08.144 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:08.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 msgr2=0x7fb3b40a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 0x7fb3b40a3800 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005d90 tx=0x7fb3bc005ee0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 msgr2=0x7fb3cc1abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc1abd30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b070 tx=0x7fb3c4004c80 comp rx=0 tx=0).stop 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 shutdown_connections 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb3cc10bd60 0x7fb3cc1abd30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3cc070ee0 0x7fb3cc1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb3b40a1350 0x7fb3b40a3800 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 --2- 172.21.15.26:0/921659962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3cc071830 0x7fb3cc1a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.162+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 >> 172.21.15.26:0/921659962 conn(0x7fb3cc06ca90 msgr2=0x7fb3cc06dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.164+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 shutdown_connections 2023-07-13T22:25:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:08.164+0000 7fb3d1dff700 1 -- 172.21.15.26:0/921659962 wait complete. 2023-07-13T22:25:08.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:07 smithi121 ceph-mon[129187]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:25:08.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:25:08.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:25:08.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:07 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:08.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:25:08.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:08.307 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:07 smithi026 ceph-mon[95647]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:25:08.307 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:25:08.308 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:25:08.308 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:07 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:08.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:07 smithi117 ceph-mon[129074]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 278 MiB/s wr, 90 op/s 2023-07-13T22:25:08.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:25:08.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:25:08.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:07 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:25:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 -- 172.21.15.26:0/557579269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 msgr2=0x7f237c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 --2- 172.21.15.26:0/557579269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c0733e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f23780098f0 tx=0x7f2378009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 -- 172.21.15.26:0/557579269 shutdown_connections 2023-07-13T22:25:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 --2- 172.21.15.26:0/557579269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 --2- 172.21.15.26:0/557579269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 --2- 172.21.15.26:0/557579269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c0733e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.069+0000 7f2383fff700 1 -- 172.21.15.26:0/557579269 >> 172.21.15.26:0/557579269 conn(0x7f237c0fb0e0 msgr2=0x7f237c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.070+0000 7f2383fff700 1 -- 172.21.15.26:0/557579269 shutdown_connections 2023-07-13T22:25:09.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.070+0000 7f2383fff700 1 -- 172.21.15.26:0/557579269 wait complete. 2023-07-13T22:25:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.071+0000 7f2383fff700 1 Processor -- start 2023-07-13T22:25:09.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 -- start start 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c1a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a3a50 con 0x7f237c074550 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a3b90 con 0x7f237c072f70 2023-07-13T22:25:09.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.072+0000 7f2383fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1a3d20 con 0x7f237c10c0d0 2023-07-13T22:25:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.073+0000 7f238159a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.073+0000 7f238159a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53358/0 (socket says 172.21.15.26:53358) 2023-07-13T22:25:09.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.073+0000 7f238159a700 1 -- 172.21.15.26:0/1387194732 learned_addr learned my addr 172.21.15.26:0/1387194732 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.073+0000 7f2381d9b700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.073+0000 7f238259c700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c1a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 -- 172.21.15.26:0/1387194732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 msgr2=0x7f237c1a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c1a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 -- 172.21.15.26:0/1387194732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 msgr2=0x7f237c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 -- 172.21.15.26:0/1387194732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378009520 con 0x7f237c074550 2023-07-13T22:25:09.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238259c700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.074+0000 7f238159a700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2378009fd0 tx=0x7f237800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.075+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237801c070 con 0x7f237c074550 2023-07-13T22:25:09.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.075+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c1a3fa0 con 0x7f237c074550 2023-07-13T22:25:09.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.075+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c1a4360 con 0x7f237c074550 2023-07-13T22:25:09.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.075+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237800f820 con 0x7f237c074550 2023-07-13T22:25:09.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.075+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378003a00 con 0x7f237c074550 2023-07-13T22:25:09.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.077+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2360005320 con 0x7f237c074550 2023-07-13T22:25:09.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.082+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2378020030 con 0x7f237c074550 2023-07-13T22:25:09.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.084+0000 7f2372ffd700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 0x7f23680a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:09.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.085+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23780cdc00 con 0x7f237c074550 2023-07-13T22:25:09.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.085+0000 7f2381d9b700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 0x7f23680a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:09.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.086+0000 7f2381d9b700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 0x7f23680a3650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f236c005950 tx=0x7f236c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:09.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.097+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2378097480 con 0x7f237c074550 2023-07-13T22:25:09.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.789+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2360005190 con 0x7f237c074550 2023-07-13T22:25:09.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.790+0000 7f2372ffd700 1 -- 172.21.15.26:0/1387194732 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10891 (secure 0 0 0) 0x7f237809aaa0 con 0x7f237c074550 2023-07-13T22:25:09.772 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:25:09.773 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:17:26.934870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:25:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.794+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 msgr2=0x7f23680a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.794+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 0x7f23680a3650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f236c005950 tx=0x7f236c00b410 comp rx=0 tx=0).stop 2023-07-13T22:25:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.794+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 msgr2=0x7f237c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.794+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2378009fd0 tx=0x7f237800b790 comp rx=0 tx=0).stop 2023-07-13T22:25:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.794+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 shutdown_connections 2023-07-13T22:25:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.795+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f237c10c0d0 0x7f237c1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.795+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f237c072f70 0x7f237c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.795+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f23680a11a0 0x7f23680a3650 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f236c005950 tx=0x7f236c00b410 comp rx=0 tx=0).stop 2023-07-13T22:25:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.795+0000 7f2383fff700 1 --2- 172.21.15.26:0/1387194732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237c074550 0x7f237c19c410 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.795+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 >> 172.21.15.26:0/1387194732 conn(0x7f237c0fb0e0 msgr2=0x7f237c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.800+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 shutdown_connections 2023-07-13T22:25:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:09.800+0000 7f2383fff700 1 -- 172.21.15.26:0/1387194732 wait complete. 2023-07-13T22:25:09.792 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 29 2023-07-13T22:25:09.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:25:10.065 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:09 smithi026 ceph-mon[95647]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 337 MiB/s wr, 112 op/s 2023-07-13T22:25:10.066 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:09 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1387194732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:10.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:09 smithi121 ceph-mon[129187]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 337 MiB/s wr, 112 op/s 2023-07-13T22:25:10.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:09 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1387194732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:10.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.430+0000 7fb208b9d700 1 -- 172.21.15.26:0/2907433624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2040fe8d0 msgr2=0x7fb2040fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.430+0000 7fb208b9d700 1 --2- 172.21.15.26:0/2907433624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2040fe8d0 0x7fb2040fece0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009990 tx=0x7fb1f4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.430+0000 7fb208b9d700 1 -- 172.21.15.26:0/2907433624 shutdown_connections 2023-07-13T22:25:10.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.430+0000 7fb208b9d700 1 --2- 172.21.15.26:0/2907433624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb204100450 0x7fb204106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.430+0000 7fb208b9d700 1 --2- 172.21.15.26:0/2907433624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2040fe8d0 0x7fb2040fece0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.431+0000 7fb208b9d700 1 --2- 172.21.15.26:0/2907433624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.431+0000 7fb208b9d700 1 -- 172.21.15.26:0/2907433624 >> 172.21.15.26:0/2907433624 conn(0x7fb2040f9ed0 msgr2=0x7fb2040fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.431+0000 7fb208b9d700 1 -- 172.21.15.26:0/2907433624 shutdown_connections 2023-07-13T22:25:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.431+0000 7fb208b9d700 1 -- 172.21.15.26:0/2907433624 wait complete. 2023-07-13T22:25:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.433+0000 7fb208b9d700 1 Processor -- start 2023-07-13T22:25:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.433+0000 7fb208b9d700 1 -- start start 2023-07-13T22:25:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.434+0000 7fb208b9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 0x7fb204193460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.434+0000 7fb208b9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.434+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 0x7fb204193460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.434+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 0x7fb204193460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47162/0 (socket says 172.21.15.26:47162) 2023-07-13T22:25:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53368/0 (socket says 172.21.15.26:53368) 2023-07-13T22:25:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb208b9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb204100450 0x7fb2041abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb208b9d700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204196a80 con 0x7fb2040ffac0 2023-07-13T22:25:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb208b9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb204196bf0 con 0x7fb204100450 2023-07-13T22:25:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.435+0000 7fb208b9d700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2041ac430 con 0x7fb2040fe8d0 2023-07-13T22:25:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.436+0000 7fb2037fe700 1 -- 172.21.15.26:0/1909356195 learned_addr learned my addr 172.21.15.26:0/1909356195 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.437+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1909356195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 msgr2=0x7fb204193460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.437+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 0x7fb204193460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.437+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1909356195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb204100450 msgr2=0x7fb2041abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.437+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb204100450 0x7fb2041abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.437+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1909356195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40095f0 con 0x7fb2040ffac0 2023-07-13T22:25:10.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.438+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00b450 tx=0x7fb1ec00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.440+0000 7fb2017fa700 1 -- 172.21.15.26:0/1909356195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec011830 con 0x7fb2040ffac0 2023-07-13T22:25:10.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.440+0000 7fb2017fa700 1 -- 172.21.15.26:0/1909356195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1ec011e50 con 0x7fb2040ffac0 2023-07-13T22:25:10.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.440+0000 7fb2017fa700 1 -- 172.21.15.26:0/1909356195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec010e40 con 0x7fb2040ffac0 2023-07-13T22:25:10.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.440+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2041ac5d0 con 0x7fb2040ffac0 2023-07-13T22:25:10.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.440+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2041acb10 con 0x7fb2040ffac0 2023-07-13T22:25:10.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.449+0000 7fb2017fa700 1 -- 172.21.15.26:0/1909356195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb1ec011990 con 0x7fb2040ffac0 2023-07-13T22:25:10.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.452+0000 7fb2017fa700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 0x7fb1f00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.452+0000 7fb2017fa700 1 -- 172.21.15.26:0/1909356195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1ec0cc250 con 0x7fb2040ffac0 2023-07-13T22:25:10.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:09 smithi117 ceph-mon[129074]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 337 MiB/s wr, 112 op/s 2023-07-13T22:25:10.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:09 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1387194732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.458+0000 7fb21d7bfb80 1 Processor -- start 2023-07-13T22:25:10.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 -- start start 2023-07-13T22:25:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 0x563e47fc4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc5030 0x563e47fc58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fc62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47eb8cb0 con 0x563e47fc5030 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47f0ba40 con 0x563e47fc46e0 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.459+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47f04ed0 con 0x563e47fc5e70 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.466+0000 7fb203fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fc62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.467+0000 7fb203fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fc62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47172/0 (socket says 172.21.15.26:47172) 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.467+0000 7fb203fff700 1 -- 172.21.15.26:0/64454618 learned_addr learned my addr 172.21.15.26:0/64454618 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:10.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.467+0000 7fb2037fe700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 0x7fb1f00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.467+0000 7fb2037fe700 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 0x563e47fc4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 -- 172.21.15.26:0/64454618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 msgr2=0x563e47fc62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fc62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 -- 172.21.15.26:0/64454618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc5030 msgr2=0x563e47fc58e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc5030 0x563e47fc58e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 -- 172.21.15.26:0/64454618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e47d32de0 con 0x563e47fc46e0 2023-07-13T22:25:10.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.468+0000 7fb2037fe700 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 0x563e47fc4af0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400b5c0 tx=0x7fb1f4005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e892d1005b69799 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.475+0000 7fb2037fe700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 0x7fb1f00a3890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb1f402d000 tx=0x7fb1f4004030 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.478+0000 7fb1f9ffb700 1 -- 172.21.15.26:0/64454618 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f402d030 con 0x563e47fc46e0 2023-07-13T22:25:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.478+0000 7fb1f9ffb700 1 -- 172.21.15.26:0/64454618 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f4004500 con 0x563e47fc46e0 2023-07-13T22:25:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.478+0000 7fb1f9ffb700 1 -- 172.21.15.26:0/64454618 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4003af0 con 0x563e47fc46e0 2023-07-13T22:25:10.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.479+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/64454618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 msgr2=0x563e47fc4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.479+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 0x563e47fc4af0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400b5c0 tx=0x7fb1f4005600 comp rx=0 tx=0).stop 2023-07-13T22:25:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/64454618 shutdown_connections 2023-07-13T22:25:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fc62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc46e0 0x563e47fc4af0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/64454618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc5030 0x563e47fc58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/64454618 >> 172.21.15.26:0/64454618 conn(0x563e47fa8ff0 msgr2=0x563e47fae460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.480+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/64454618 shutdown_connections 2023-07-13T22:25:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.481+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/64454618 wait complete. 2023-07-13T22:25:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.481+0000 7fb21d7bfb80 1 -- start start 2023-07-13T22:25:10.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.482+0000 7fb21d7bfb80 1 Processor -- start 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47ea4ac0 con 0x563e47fc46e0 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47ea47c0 con 0x563e47fc5e70 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47d88920 con 0x563e47fc5030 2023-07-13T22:25:10.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb21d7bfb80 10 client.0 _ll_register_callbacks cb 0x563e47ea2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47188/0 (socket says 172.21.15.26:47188) 2023-07-13T22:25:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.483+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1289348948 learned_addr learned my addr 172.21.15.26:0/1289348948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:10.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 msgr2=0x563e47fd59b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd59b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1289348948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 msgr2=0x563e47fce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 -- 172.21.15.26:0/1289348948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb204107640 con 0x563e47fc5030 2023-07-13T22:25:10.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.484+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec099310 tx=0x7fb1ec0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bb4f6be5c938f1c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.487+0000 7fb1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:10.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.487+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec09d640 con 0x563e47fc5030 2023-07-13T22:25:10.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.487+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1ec0a1a60 con 0x563e47fc5030 2023-07-13T22:25:10.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.487+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e47e5e4f0 con 0x563e47fc5030 2023-07-13T22:25:10.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.488+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1ec0b90b0 con 0x563e47fc5030 2023-07-13T22:25:10.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.489+0000 7fb21d7bfb80 10 client.27283 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:10.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.489+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563e47d1bc60 con 0x563e47fc5030 2023-07-13T22:25:10.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.489+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e47e5e4f0 con 0x563e47fc5030 2023-07-13T22:25:10.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.490+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb1ec09d150 con 0x563e47fc5030 2023-07-13T22:25:10.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.490+0000 7fb21d7bfb80 10 client.27283 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:10.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.490+0000 7fb21d7bfb80 10 client.27283 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:25:10.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.490+0000 7fb21d7bfb80 20 client.27283 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:10.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.490+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e47fda270 0x563e47fdc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.491+0000 7fb21d7bfb80 4 client.27283 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:10.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.491+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e47efdeb0 con 0x563e47fda270 2023-07-13T22:25:10.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.491+0000 7fb2037fe700 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e47fda270 0x563e47fdc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.492+0000 7fb2037fe700 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e47fda270 0x563e47fdc680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6c7c2306e21776 server_cookie=85e47eab20e07350 in_seq=0 out_seq=0 2023-07-13T22:25:10.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.492+0000 7fb1e2ffd700 10 client.27283 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:10.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.492+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/1289348948 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e47efdeb0 con 0x563e47fda270 2023-07-13T22:25:10.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.492+0000 7fb1e2ffd700 10 client.27283 handle_command_reply: tid=0 2023-07-13T22:25:10.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 client.27283 shutdown 2023-07-13T22:25:10.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 msgr2=0x563e47fce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec099310 tx=0x7fb1ec0093b0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 shutdown_connections 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e47fda270 0x563e47fdc680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/1289348948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5030 0x563e47fce970 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec099310 tx=0x7fb1ec0093b0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 >> 172.21.15.26:0/1289348948 conn(0x563e47fa8ff0 msgr2=0x563e47fadb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.493+0000 7fb1e2ffd700 0 client.27283 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.494+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 shutdown_connections 2023-07-13T22:25:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.494+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/1289348948 wait complete. 2023-07-13T22:25:10.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.494+0000 7fb21d7bfb80 20 client.27283 trim_cache size 0 max 16384 2023-07-13T22:25:10.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.529+0000 7fb21d7bfb80 1 Processor -- start 2023-07-13T22:25:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.529+0000 7fb21d7bfb80 1 -- start start 2023-07-13T22:25:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fd6660 0x563e47fc8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47e5e4f0 con 0x563e47fc46e0 2023-07-13T22:25:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47fc5770 con 0x563e47fc5e70 2023-07-13T22:25:10.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.530+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47fd76a0 con 0x563e47fd6660 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.531+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.531+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53394/0 (socket says 172.21.15.26:53394) 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.531+0000 7fb2037fe700 1 -- 172.21.15.26:0/3740347890 learned_addr learned my addr 172.21.15.26:0/3740347890 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb2037fe700 1 -- 172.21.15.26:0/3740347890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fd6660 msgr2=0x563e47fc8dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:25:10.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb2037fe700 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fd6660 0x563e47fc8dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb2037fe700 1 -- 172.21.15.26:0/3740347890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 msgr2=0x563e47fd58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb2037fe700 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.532+0000 7fb2037fe700 1 -- 172.21.15.26:0/3740347890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e47d59250 con 0x563e47fc46e0 2023-07-13T22:25:10.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.533+0000 7fb2037fe700 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563e47fc4480 tx=0x7fb1f401a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539480c826d4ee24 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.535+0000 7fb1fa7fc700 1 -- 172.21.15.26:0/3740347890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4004150 con 0x563e47fc46e0 2023-07-13T22:25:10.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.535+0000 7fb1fa7fc700 1 -- 172.21.15.26:0/3740347890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f4005040 con 0x563e47fc46e0 2023-07-13T22:25:10.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.535+0000 7fb1fa7fc700 1 -- 172.21.15.26:0/3740347890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f403f030 con 0x563e47fc46e0 2023-07-13T22:25:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/3740347890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 msgr2=0x563e47fd6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563e47fc4480 tx=0x7fb1f401a070 comp rx=0 tx=0).stop 2023-07-13T22:25:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/3740347890 shutdown_connections 2023-07-13T22:25:10.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fc46e0 0x563e47fd6120 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x563e47fc4480 tx=0x7fb1f401a070 comp rx=0 tx=0).stop 2023-07-13T22:25:10.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fd6660 0x563e47fc8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/3740347890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fc5e70 0x563e47fd58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/3740347890 >> 172.21.15.26:0/3740347890 conn(0x563e47fa8ff0 msgr2=0x563e47fcd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/3740347890 shutdown_connections 2023-07-13T22:25:10.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.537+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/3740347890 wait complete. 2023-07-13T22:25:10.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.538+0000 7fb21d7bfb80 1 -- start start 2023-07-13T22:25:10.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.538+0000 7fb21d7bfb80 1 Processor -- start 2023-07-13T22:25:10.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 0x563e47fd2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fd2550 0x563e4802e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47fd5ca0 con 0x563e47fd6660 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47fd29c0 con 0x563e47fd2550 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e47fd2b30 con 0x563e47fc5e70 2023-07-13T22:25:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.539+0000 7fb21d7bfb80 10 client.0 _ll_register_callbacks cb 0x7fb1ec099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:10.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 0x563e47fd2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb2037fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47208/0 (socket says 172.21.15.26:47208) 2023-07-13T22:25:10.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb1fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 0x563e47fd2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53402/0 (socket says 172.21.15.26:53402) 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb2037fe700 1 -- 172.21.15.26:0/501016869 learned_addr learned my addr 172.21.15.26:0/501016869 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb2037fe700 1 -- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fd2550 msgr2=0x563e4802e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.540+0000 7fb2037fe700 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fd2550 0x563e4802e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.541+0000 7fb2037fe700 1 -- 172.21.15.26:0/501016869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 msgr2=0x563e47fd2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.541+0000 7fb2037fe700 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 0x563e47fd2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.541+0000 7fb2037fe700 1 -- 172.21.15.26:0/501016869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f4037040 con 0x563e47fc5e70 2023-07-13T22:25:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.541+0000 7fb2037fe700 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb1f403fbb0 tx=0x7fb1f4005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70b997ca8e543be8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.541+0000 7fb1e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.542+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4005230 con 0x563e47fc5e70 2023-07-13T22:25:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.542+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1f402e070 con 0x563e47fc5e70 2023-07-13T22:25:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.542+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1f4004500 con 0x563e47fc5e70 2023-07-13T22:25:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.543+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1f4003890 con 0x563e47fc5e70 2023-07-13T22:25:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.543+0000 7fb21d7bfb80 10 client.27286 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.543+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563e47d312b0 con 0x563e47fc5e70 2023-07-13T22:25:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.543+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f4004500 con 0x563e47fc5e70 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.543+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb1f4051050 con 0x563e47fc5e70 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 10 client.27286 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 10 client.27286 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 20 client.27286 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e4802f7c0 0x563e48031bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 4 client.27286 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.544+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e47ec0050 con 0x563e4802f7c0 2023-07-13T22:25:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.545+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e4802f7c0 0x563e48031bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.545+0000 7fb1fbfff700 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e4802f7c0 0x563e48031bd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5845f4dad0863973 server_cookie=e7b5a061eb964224 in_seq=0 out_seq=0 2023-07-13T22:25:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb1e2ffd700 10 client.27286 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb1e2ffd700 1 -- 172.21.15.26:0/501016869 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e47ec0050 con 0x563e4802f7c0 2023-07-13T22:25:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb1e2ffd700 10 client.27286 handle_command_reply: tid=0 2023-07-13T22:25:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb21d7bfb80 1 client.27286 shutdown 2023-07-13T22:25:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 msgr2=0x563e47fd54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.546+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb1f403fbb0 tx=0x7fb1f4005980 comp rx=0 tx=0).stop 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 shutdown_connections 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x563e4802f7c0 0x563e48031bd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563e47fd2550 0x563e4802e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e47fc5e70 0x563e47fd54c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 --2- 172.21.15.26:0/501016869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563e47fd6660 0x563e47fd2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 >> 172.21.15.26:0/501016869 conn(0x563e47fa8ff0 msgr2=0x563e47fc8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb1e2ffd700 0 client.27286 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 shutdown_connections 2023-07-13T22:25:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.547+0000 7fb21d7bfb80 1 -- 172.21.15.26:0/501016869 wait complete. 2023-07-13T22:25:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.548+0000 7fb21d7bfb80 20 client.27286 trim_cache size 0 max 16384 2023-07-13T22:25:10.532 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.549+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 msgr2=0x7fb1f00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.549+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 0x7fb1f00a3890 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb1f402d000 tx=0x7fb1f4004030 comp rx=0 tx=0).stop 2023-07-13T22:25:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.549+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 msgr2=0x7fb2041939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.549+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00b450 tx=0x7fb1ec00b810 comp rx=0 tx=0).stop 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 shutdown_connections 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb204100450 0x7fb2041abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2040fe8d0 0x7fb204193460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fb1f00a13e0 0x7fb1f00a3890 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 --2- 172.21.15.26:0/1909356195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2040ffac0 0x7fb2041939a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 >> 172.21.15.26:0/1909356195 conn(0x7fb2040f9ed0 msgr2=0x7fb2040fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 shutdown_connections 2023-07-13T22:25:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:10.550+0000 7fb208b9d700 1 -- 172.21.15.26:0/1909356195 wait complete. 2023-07-13T22:25:10.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:25:10.658 INFO:tasks.workunit.client.0.smithi026.stdout:stopping iogen 2023-07-13T22:25:10.659 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo 'stopping iogen' 2023-07-13T22:25:10.659 INFO:tasks.workunit.client.0.smithi026.stderr:+ ./iogen -k 2023-07-13T22:25:10.713 INFO:tasks.workunit.client.0.smithi026.stdout:OK 2023-07-13T22:25:10.714 INFO:tasks.workunit.client.0.smithi026.stderr:+ echo OK 2023-07-13T22:25:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 -- 172.21.15.26:0/3132036233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0077210 msgr2=0x7facc0079690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 --2- 172.21.15.26:0/3132036233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0077210 0x7facc0079690 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7facb800b0a0 tx=0x7facb800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 -- 172.21.15.26:0/3132036233 shutdown_connections 2023-07-13T22:25:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 --2- 172.21.15.26:0/3132036233 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0077210 0x7facc0079690 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 --2- 172.21.15.26:0/3132036233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0071200 0x7facc0071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 --2- 172.21.15.26:0/3132036233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facc0074850 0x7facc0076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.082+0000 7facc49ff700 1 -- 172.21.15.26:0/3132036233 >> 172.21.15.26:0/3132036233 conn(0x7facc006ca90 msgr2=0x7facc006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.084+0000 7facc49ff700 1 -- 172.21.15.26:0/3132036233 shutdown_connections 2023-07-13T22:25:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.084+0000 7facc49ff700 1 -- 172.21.15.26:0/3132036233 wait complete. 2023-07-13T22:25:11.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.085+0000 7facc49ff700 1 Processor -- start 2023-07-13T22:25:11.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 -- start start 2023-07-13T22:25:11.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0071200 0x7facc0082340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facc0074850 0x7facc0082880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc012fd20 con 0x7facc0074850 2023-07-13T22:25:11.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc012fe90 con 0x7facc0077210 2023-07-13T22:25:11.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.086+0000 7facc49ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc0130000 con 0x7facc0071200 2023-07-13T22:25:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.087+0000 7facbed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.087+0000 7facbed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56116/0 (socket says 172.21.15.26:56116) 2023-07-13T22:25:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.087+0000 7facbed9d700 1 -- 172.21.15.26:0/1922468796 learned_addr learned my addr 172.21.15.26:0/1922468796 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.087+0000 7facbe59c700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0071200 0x7facc0082340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 -- 172.21.15.26:0/1922468796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0071200 msgr2=0x7facc0082340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0071200 0x7facc0082340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 -- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facc0074850 msgr2=0x7facc0082880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facc0074850 0x7facc0082880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 -- 172.21.15.26:0/1922468796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009d00 con 0x7facc0077210 2023-07-13T22:25:11.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.088+0000 7facbed9d700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7facb8006070 tx=0x7facb80093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.089+0000 7facaf7fe700 1 -- 172.21.15.26:0/1922468796 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800e040 con 0x7facc0077210 2023-07-13T22:25:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.089+0000 7facaf7fe700 1 -- 172.21.15.26:0/1922468796 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb8003c00 con 0x7facc0077210 2023-07-13T22:25:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.089+0000 7facaf7fe700 1 -- 172.21.15.26:0/1922468796 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80054d0 con 0x7facc0077210 2023-07-13T22:25:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.090+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc0130280 con 0x7facc0077210 2023-07-13T22:25:11.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.090+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01306e0 con 0x7facc0077210 2023-07-13T22:25:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.097+0000 7facd82c3b80 1 Processor -- start 2023-07-13T22:25:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.097+0000 7facaf7fe700 1 -- 172.21.15.26:0/1922468796 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7facb8005630 con 0x7facc0077210 2023-07-13T22:25:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 -- start start 2023-07-13T22:25:11.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b27296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b262dcb0 con 0x55e8b27284d0 2023-07-13T22:25:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2680a40 con 0x55e8b2729c70 2023-07-13T22:25:11.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.098+0000 7facd82c3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2679ed0 con 0x55e8b2728e20 2023-07-13T22:25:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.099+0000 7facbdd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.099+0000 7facbdd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56124/0 (socket says 172.21.15.26:56124) 2023-07-13T22:25:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.099+0000 7facbdd9b700 1 -- 172.21.15.26:0/3668350615 learned_addr learned my addr 172.21.15.26:0/3668350615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbe59c700 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b27296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbdd9b700 1 -- 172.21.15.26:0/3668350615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 msgr2=0x55e8b27296e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbdd9b700 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b27296e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbdd9b700 1 -- 172.21.15.26:0/3668350615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 msgr2=0x55e8b27288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbdd9b700 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.100+0000 7facbdd9b700 1 -- 172.21.15.26:0/3668350615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b24a7de0 con 0x55e8b2729c70 2023-07-13T22:25:11.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.101+0000 7facbdd9b700 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7facb400bd80 tx=0x7facb400d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec2a2347d4735a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.101+0000 7facacff9700 1 -- 172.21.15.26:0/3668350615 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4010070 con 0x55e8b2729c70 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.101+0000 7facacff9700 1 -- 172.21.15.26:0/3668350615 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb40044e0 con 0x55e8b2729c70 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.101+0000 7facacff9700 1 -- 172.21.15.26:0/3668350615 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4004e80 con 0x55e8b2729c70 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 -- 172.21.15.26:0/3668350615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 msgr2=0x55e8b272a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7facb400bd80 tx=0x7facb400d330 comp rx=0 tx=0).stop 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 -- 172.21.15.26:0/3668350615 shutdown_connections 2023-07-13T22:25:11.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b272a0e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 --2- 172.21.15.26:0/3668350615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b27296e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 -- 172.21.15.26:0/3668350615 >> 172.21.15.26:0/3668350615 conn(0x55e8b271dda0 msgr2=0x55e8b27231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 -- 172.21.15.26:0/3668350615 shutdown_connections 2023-07-13T22:25:11.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.103+0000 7facd82c3b80 1 -- 172.21.15.26:0/3668350615 wait complete. 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.104+0000 7facd82c3b80 1 -- start start 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.104+0000 7facaf7fe700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 0x7faca80a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.104+0000 7facaf7fe700 1 -- 172.21.15.26:0/1922468796 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7facb80cdc70 con 0x7facc0077210 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.104+0000 7facd82c3b80 1 Processor -- start 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.105+0000 7facbdd9b700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 0x7faca80a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.105+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.105+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b2732700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.105+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2739760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facd82c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2619ac0 con 0x55e8b27284d0 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facd82c3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b26197c0 con 0x55e8b2729c70 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facd82c3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b24fd920 con 0x55e8b2728e20 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbdd9b700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 0x7faca80a3730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facb400f010 tx=0x7facb4004860 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facd82c3b80 10 client.0 _ll_register_callbacks cb 0x55e8b2617760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbdd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b2732700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2739760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbdd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b2732700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47238/0 (socket says 172.21.15.26:47238) 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbdd9b700 1 -- 172.21.15.26:0/2227678123 learned_addr learned my addr 172.21.15.26:0/2227678123 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbe59c700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbed9d700 1 -- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 msgr2=0x55e8b2732700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbed9d700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b2732700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbed9d700 1 -- 172.21.15.26:0/2227678123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 msgr2=0x55e8b27321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.106+0000 7facbed9d700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7facbed9d700 1 -- 172.21.15.26:0/2227678123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb400ba30 con 0x55e8b2729c70 2023-07-13T22:25:11.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7facbed9d700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2739760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7facb80cb720 tx=0x7facb809ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36d4780292fdfd3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7faca5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80b2070 con 0x55e8b2729c70 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7facb809b0c0 con 0x55e8b2729c70 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8b25d34f0 con 0x55e8b2729c70 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb809f0f0 con 0x55e8b2729c70 2023-07-13T22:25:11.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7facd82c3b80 10 client.27308 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.107+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e8b2490c60 con 0x55e8b2729c70 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e8b25d34f0 con 0x55e8b2729c70 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7facb809b3b0 con 0x55e8b2729c70 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 10 client.27308 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 10 client.27308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 20 client.27308 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b273dcb0 0x55e8b27400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 4 client.27308 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8b2672eb0 con 0x55e8b273dcb0 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.108+0000 7facbe59c700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b273dcb0 0x55e8b27400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.109+0000 7facbe59c700 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b273dcb0 0x55e8b27400c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79a78c63e74785f0 server_cookie=6ee6e5c236526a14 in_seq=0 out_seq=0 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.109+0000 7faca5ffb700 10 client.27308 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.109+0000 7faca5ffb700 1 -- 172.21.15.26:0/2227678123 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8b2672eb0 con 0x55e8b273dcb0 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.109+0000 7faca5ffb700 10 client.27308 handle_command_reply: tid=0 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 client.27308 shutdown 2023-07-13T22:25:11.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 msgr2=0x55e8b2739760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2739760 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7facb80cb720 tx=0x7facb809ac00 comp rx=0 tx=0).stop 2023-07-13T22:25:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 shutdown_connections 2023-07-13T22:25:11.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b273dcb0 0x55e8b27400c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b27284d0 0x55e8b27321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7faca5ffb700 0 client.27308 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2739760 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2227678123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2728e20 0x55e8b2732700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 >> 172.21.15.26:0/2227678123 conn(0x55e8b271dda0 msgr2=0x55e8b27228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 shutdown_connections 2023-07-13T22:25:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 1 -- 172.21.15.26:0/2227678123 wait complete. 2023-07-13T22:25:11.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.110+0000 7facd82c3b80 20 client.27308 trim_cache size 0 max 16384 2023-07-13T22:25:11.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.137+0000 7facd82c3b80 1 Processor -- start 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.137+0000 7facd82c3b80 1 -- start start 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 0x55e8b272a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2739720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2731d50 con 0x55e8b2728e20 2023-07-13T22:25:11.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b272fe50 con 0x55e8b27284d0 2023-07-13T22:25:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facd82c3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2724ca0 con 0x55e8b2729c70 2023-07-13T22:25:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.138+0000 7facbed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.139+0000 7facbed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47240/0 (socket says 172.21.15.26:47240) 2023-07-13T22:25:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.139+0000 7facbed9d700 1 -- 172.21.15.26:0/2416572729 learned_addr learned my addr 172.21.15.26:0/2416572729 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.139+0000 7facbdd9b700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2739720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbe59c700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 0x55e8b272a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 -- 172.21.15.26:0/2416572729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 msgr2=0x55e8b272a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 0x55e8b272a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 -- 172.21.15.26:0/2416572729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 msgr2=0x55e8b2739720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2739720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 -- 172.21.15.26:0/2416572729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8b24ce250 con 0x55e8b2729c70 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbe59c700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 0x55e8b272a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbdd9b700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2739720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facbed9d700 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fd0 tx=0x7facb800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df67348063bedab server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.140+0000 7facad7fa700 1 -- 172.21.15.26:0/2416572729 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80b2070 con 0x55e8b2729c70 2023-07-13T22:25:11.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.141+0000 7facad7fa700 1 -- 172.21.15.26:0/2416572729 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb809a680 con 0x55e8b2729c70 2023-07-13T22:25:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.141+0000 7facad7fa700 1 -- 172.21.15.26:0/2416572729 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb809b320 con 0x55e8b2729c70 2023-07-13T22:25:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.142+0000 7facd82c3b80 1 -- 172.21.15.26:0/2416572729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 msgr2=0x55e8b2730e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.142+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7facb8009fd0 tx=0x7facb800b3b0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.176+0000 7facd82c3b80 1 -- 172.21.15.26:0/2416572729 shutdown_connections 2023-07-13T22:25:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.176+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b2729c70 0x55e8b2730e50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.176+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b27284d0 0x55e8b272a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.176+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2416572729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2739720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.176+0000 7facd82c3b80 1 -- 172.21.15.26:0/2416572729 >> 172.21.15.26:0/2416572729 conn(0x55e8b271dda0 msgr2=0x55e8b2744a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.177+0000 7facd82c3b80 1 -- 172.21.15.26:0/2416572729 shutdown_connections 2023-07-13T22:25:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.177+0000 7facd82c3b80 1 -- 172.21.15.26:0/2416572729 wait complete. 2023-07-13T22:25:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.178+0000 7facd82c3b80 1 -- start start 2023-07-13T22:25:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.178+0000 7facd82c3b80 1 Processor -- start 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.179+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.179+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2735c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.179+0000 7facd82c3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2738c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.179+0000 7facd82c3b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b2739170 con 0x55e8b2728e20 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.179+0000 7facd82c3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b27392e0 con 0x55e8b2729c70 2023-07-13T22:25:11.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.180+0000 7facd82c3b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8b27394a0 con 0x55e8b27284d0 2023-07-13T22:25:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.180+0000 7facd82c3b80 10 client.0 _ll_register_callbacks cb 0x7facb8097400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.180+0000 7facbe59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.180+0000 7facbe59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47256/0 (socket says 172.21.15.26:47256) 2023-07-13T22:25:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 -- 172.21.15.26:0/2131517275 learned_addr learned my addr 172.21.15.26:0/2131517275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbed9d700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2738c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 -- 172.21.15.26:0/2131517275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 msgr2=0x55e8b2738c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2738c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 -- 172.21.15.26:0/2131517275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 msgr2=0x55e8b2735c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2735c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.181+0000 7facbe59c700 1 -- 172.21.15.26:0/2131517275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8b2490c60 con 0x55e8b27284d0 2023-07-13T22:25:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.191+0000 7facbe59c700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55e8b2740970 tx=0x7facb001dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37f238e31f55a7a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.194+0000 7faca5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.194+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0017040 con 0x55e8b27284d0 2023-07-13T22:25:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.194+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7facb00129b0 con 0x55e8b27284d0 2023-07-13T22:25:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.202+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facb40044e0 con 0x55e8b27284d0 2023-07-13T22:25:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.203+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7facb001d620 con 0x55e8b27284d0 2023-07-13T22:25:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.203+0000 7facd82c3b80 10 client.27304 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.204+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e8b24a62b0 con 0x55e8b27284d0 2023-07-13T22:25:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.204+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facb40044e0 con 0x55e8b27284d0 2023-07-13T22:25:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.204+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7facb0026420 con 0x55e8b27284d0 2023-07-13T22:25:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 10 client.27304 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 10 client.27304 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 20 client.27304 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b2792c60 0x55e8b2795040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 4 client.27304 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.205+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8b2635050 con 0x55e8b2792c60 2023-07-13T22:25:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.206+0000 7facbdd9b700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b2792c60 0x55e8b2795040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7facbdd9b700 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b2792c60 0x55e8b2795040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e533547b8514d83 server_cookie=a75c99be437363c8 in_seq=0 out_seq=0 2023-07-13T22:25:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7faca5ffb700 10 client.27304 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7faca5ffb700 1 -- 172.21.15.26:0/2131517275 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8b2635050 con 0x55e8b2792c60 2023-07-13T22:25:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7faca5ffb700 10 client.27304 handle_command_reply: tid=0 2023-07-13T22:25:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7facd82c3b80 1 client.27304 shutdown 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 msgr2=0x55e8b27356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.212+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55e8b2740970 tx=0x7facb001dd50 comp rx=0 tx=0).stop 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 shutdown_connections 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55e8b2792c60 0x55e8b2795040 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8b2729c70 0x55e8b2738c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e8b27284d0 0x55e8b27356d0 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55e8b2740970 tx=0x7facb001dd50 comp rx=0 tx=0).stop 2023-07-13T22:25:11.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 --2- 172.21.15.26:0/2131517275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e8b2728e20 0x55e8b2735c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 >> 172.21.15.26:0/2131517275 conn(0x55e8b271dda0 msgr2=0x55e8b2730e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.213+0000 7faca5ffb700 0 client.27304 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.215+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 shutdown_connections 2023-07-13T22:25:11.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.215+0000 7facd82c3b80 1 -- 172.21.15.26:0/2131517275 wait complete. 2023-07-13T22:25:11.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.217+0000 7facd82c3b80 20 client.27304 trim_cache size 0 max 16384 2023-07-13T22:25:11.200 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.238+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 msgr2=0x7faca80a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.238+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 0x7faca80a3730 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7facb400f010 tx=0x7facb4004860 comp rx=0 tx=0).stop 2023-07-13T22:25:11.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.238+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 msgr2=0x7facc012f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.238+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7facb8006070 tx=0x7facb80093b0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 shutdown_connections 2023-07-13T22:25:11.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0077210 0x7facc012f720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facc0071200 0x7facc0082340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7faca80a1280 0x7faca80a3730 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 --2- 172.21.15.26:0/1922468796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7facc0074850 0x7facc0082880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.239+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 >> 172.21.15.26:0/1922468796 conn(0x7facc006ca90 msgr2=0x7facc0076060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.269+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 shutdown_connections 2023-07-13T22:25:11.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.269+0000 7facc49ff700 1 -- 172.21.15.26:0/1922468796 wait complete. 2023-07-13T22:25:11.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:25:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 -- 172.21.15.26:0/1829438648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00ff470 msgr2=0x7f8bf0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/1829438648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00ff470 0x7f8bf0103d20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009990 tx=0x7f8be4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 -- 172.21.15.26:0/1829438648 shutdown_connections 2023-07-13T22:25:11.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/1829438648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf00fd990 0x7f8bf00fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/1829438648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00ff470 0x7f8bf0103d20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/1829438648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bf00feac0 0x7f8bf00fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.916+0000 7f8bf53ff700 1 -- 172.21.15.26:0/1829438648 >> 172.21.15.26:0/1829438648 conn(0x7f8bf00f9340 msgr2=0x7f8bf00fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.917+0000 7f8bf53ff700 1 -- 172.21.15.26:0/1829438648 shutdown_connections 2023-07-13T22:25:11.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.917+0000 7f8bf53ff700 1 -- 172.21.15.26:0/1829438648 wait complete. 2023-07-13T22:25:11.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.918+0000 7f8bf53ff700 1 Processor -- start 2023-07-13T22:25:11.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 -- start start 2023-07-13T22:25:11.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bf00fd990 0x7f8bf010a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf00ff470 0x7f8bf01ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf010da00 con 0x7f8bf00ff470 2023-07-13T22:25:11.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf010db40 con 0x7f8bf00feac0 2023-07-13T22:25:11.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.919+0000 7f8bf53ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf0069f30 con 0x7f8bf00fd990 2023-07-13T22:25:11.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56158/0 (socket says 172.21.15.26:56158) 2023-07-13T22:25:11.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 -- 172.21.15.26:0/3798573712 learned_addr learned my addr 172.21.15.26:0/3798573712 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 -- 172.21.15.26:0/3798573712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bf00fd990 msgr2=0x7f8bf010a3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:25:11.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bf00fd990 0x7f8bf010a3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 -- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf00ff470 msgr2=0x7f8bf01ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf00ff470 0x7f8bf01ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.922+0000 7f8bee7fc700 1 -- 172.21.15.26:0/3798573712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40095f0 con 0x7f8bf00feac0 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.923+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ef10 tx=0x7f8be800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.924+0000 7f8bd7fff700 1 -- 172.21.15.26:0/3798573712 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800cad0 con 0x7f8bf00feac0 2023-07-13T22:25:11.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.924+0000 7f8bd7fff700 1 -- 172.21.15.26:0/3798573712 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8be800f040 con 0x7f8bf00feac0 2023-07-13T22:25:11.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.924+0000 7f8bd7fff700 1 -- 172.21.15.26:0/3798573712 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80146e0 con 0x7f8bf00feac0 2023-07-13T22:25:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.924+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf006a1b0 con 0x7f8bf00feac0 2023-07-13T22:25:11.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.924+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf006a670 con 0x7f8bf00feac0 2023-07-13T22:25:11.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.934+0000 7f8c08d2cb80 1 Processor -- start 2023-07-13T22:25:11.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.934+0000 7f8c08d2cb80 1 -- start start 2023-07-13T22:25:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5eec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5eee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5df1cb0 con 0x55d5d5eec4d0 2023-07-13T22:25:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5e44a40 con 0x55d5d5eedc70 2023-07-13T22:25:11.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.935+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5e3ded0 con 0x55d5d5eece20 2023-07-13T22:25:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.939+0000 7f8beeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.939+0000 7f8beeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47282/0 (socket says 172.21.15.26:47282) 2023-07-13T22:25:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.939+0000 7f8beeffd700 1 -- 172.21.15.26:0/1960543732 learned_addr learned my addr 172.21.15.26:0/1960543732 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.940+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5eee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.940+0000 7f8bd7fff700 1 -- 172.21.15.26:0/3798573712 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8be8007500 con 0x7f8bf00feac0 2023-07-13T22:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 -- 172.21.15.26:0/1960543732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 msgr2=0x55d5d5eee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5eee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 -- 172.21.15.26:0/1960543732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 msgr2=0x55d5d5eec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5eec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 -- 172.21.15.26:0/1960543732 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d5c6bde0 con 0x55d5d5eece20 2023-07-13T22:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.943+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009830 tx=0x7f8be0009b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb7e315145b9c392 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.944+0000 7f8bd57fa700 1 -- 172.21.15.26:0/1960543732 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0011830 con 0x55d5d5eece20 2023-07-13T22:25:11.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.944+0000 7f8bd57fa700 1 -- 172.21.15.26:0/1960543732 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8be0011e50 con 0x55d5d5eece20 2023-07-13T22:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.944+0000 7f8bd57fa700 1 -- 172.21.15.26:0/1960543732 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0010e70 con 0x55d5d5eece20 2023-07-13T22:25:11.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.945+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1960543732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 msgr2=0x55d5d5eed6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.945+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009830 tx=0x7f8be0009b40 comp rx=0 tx=0).stop 2023-07-13T22:25:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.946+0000 7f8bd7fff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 0x7f8bd80a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.946+0000 7f8bd7fff700 1 -- 172.21.15.26:0/3798573712 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8be80ccb90 con 0x7f8bf00feac0 2023-07-13T22:25:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1960543732 shutdown_connections 2023-07-13T22:25:11.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5eec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5eee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1960543732 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5eed6e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1960543732 >> 172.21.15.26:0/1960543732 conn(0x55d5d5ee1da0 msgr2=0x55d5d5ee71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1960543732 shutdown_connections 2023-07-13T22:25:11.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.947+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 0x7f8bd80a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.948+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1960543732 wait complete. 2023-07-13T22:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.948+0000 7f8c08d2cb80 1 -- start start 2023-07-13T22:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.949+0000 7f8c08d2cb80 1 Processor -- start 2023-07-13T22:25:11.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5ef6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5dddac0 con 0x55d5d5eec4d0 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5ddd7c0 con 0x55d5d5eedc70 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5cc1920 con 0x55d5d5eece20 2023-07-13T22:25:11.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.950+0000 7f8c08d2cb80 10 client.0 _ll_register_callbacks cb 0x55d5d5ddb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:11.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.951+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.951+0000 7f8bef7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56174/0 (socket says 172.21.15.26:56174) 2023-07-13T22:25:11.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.951+0000 7f8bef7fe700 1 -- 172.21.15.26:0/1635697374 learned_addr learned my addr 172.21.15.26:0/1635697374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.951+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5ef6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.952+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.954+0000 7f8bef7fe700 1 -- 172.21.15.26:0/1635697374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 msgr2=0x55d5d5ef6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.954+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5ef6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.954+0000 7f8bef7fe700 1 -- 172.21.15.26:0/1635697374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 msgr2=0x55d5d5ef61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.954+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.954+0000 7f8bef7fe700 1 -- 172.21.15.26:0/1635697374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be0009460 con 0x55d5d5eedc70 2023-07-13T22:25:11.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.955+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8be400bba0 tx=0x7f8be400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c039b9b09ab8542f server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.955+0000 7f8bd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.956+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be400f660 con 0x55d5d5eedc70 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.956+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8be4003d60 con 0x55d5d5eedc70 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.956+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5d5d974f0 con 0x55d5d5eedc70 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.957+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8be4033480 con 0x55d5d5eedc70 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.957+0000 7f8c08d2cb80 10 client.27323 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:11.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.957+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5d5c54c60 con 0x55d5d5eedc70 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.957+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 0x7f8bd80a3800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8be4003f20 tx=0x7f8be4026000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.957+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5d5d974f0 con 0x55d5d5eedc70 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8be4026640 con 0x55d5d5eedc70 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 10 client.27323 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 10 client.27323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 20 client.27323 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:11.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f02110 0x55d5d5f04520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 4 client.27323 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.958+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5d5e36eb0 con 0x55d5d5f02110 2023-07-13T22:25:11.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.960+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f02110 0x55d5d5f04520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.961+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f02110 0x55d5d5f04520 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18e77413a34e24fb server_cookie=98b79aa91939980b in_seq=0 out_seq=0 2023-07-13T22:25:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.961+0000 7f8bd2ff5700 10 client.27323 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.963+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1635697374 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5d5e36eb0 con 0x55d5d5f02110 2023-07-13T22:25:11.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.963+0000 7f8bd2ff5700 10 client.27323 handle_command_reply: tid=0 2023-07-13T22:25:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.963+0000 7f8c08d2cb80 1 client.27323 shutdown 2023-07-13T22:25:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.964+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 msgr2=0x55d5d5efd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:11.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.964+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8be400bba0 tx=0x7f8be400bc80 comp rx=0 tx=0).stop 2023-07-13T22:25:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 shutdown_connections 2023-07-13T22:25:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f02110 0x55d5d5f04520 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efd770 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1635697374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eece20 0x55d5d5ef6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 >> 172.21.15.26:0/1635697374 conn(0x55d5d5ee1da0 msgr2=0x55d5d5ee68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8bd2ff5700 0 client.27323 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.965+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 shutdown_connections 2023-07-13T22:25:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.966+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1635697374 wait complete. 2023-07-13T22:25:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.966+0000 7f8c08d2cb80 20 client.27323 trim_cache size 0 max 16384 2023-07-13T22:25:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.997+0000 7f8c08d2cb80 1 Processor -- start 2023-07-13T22:25:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 -- start start 2023-07-13T22:25:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eedc70 0x55d5d5ef4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5ee8680 con 0x55d5d5eec4d0 2023-07-13T22:25:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5ef56a0 con 0x55d5d5eece20 2023-07-13T22:25:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.998+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5eed1f0 con 0x55d5d5eedc70 2023-07-13T22:25:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.999+0000 7f8bee7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.999+0000 7f8bee7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56182/0 (socket says 172.21.15.26:56182) 2023-07-13T22:25:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.999+0000 7f8bee7fc700 1 -- 172.21.15.26:0/2159514957 learned_addr learned my addr 172.21.15.26:0/2159514957 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:11.999+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eedc70 0x55d5d5ef4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.005+0000 7f8bee7fc700 1 -- 172.21.15.26:0/2159514957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eedc70 msgr2=0x55d5d5ef4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.005+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eedc70 0x55d5d5ef4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.005+0000 7f8bee7fc700 1 -- 172.21.15.26:0/2159514957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 msgr2=0x55d5d5ef58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.005+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.005+0000 7f8bee7fc700 1 -- 172.21.15.26:0/2159514957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5d5c92250 con 0x55d5d5eece20 2023-07-13T22:25:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.017+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8be8099c50 tx=0x7f8be80ca640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3c6f01d44981a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.017+0000 7f8bd5ffb700 1 -- 172.21.15.26:0/2159514957 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8096290 con 0x55d5d5eece20 2023-07-13T22:25:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.018+0000 7f8bd5ffb700 1 -- 172.21.15.26:0/2159514957 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8be8099d20 con 0x55d5d5eece20 2023-07-13T22:25:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.018+0000 7f8bd5ffb700 1 -- 172.21.15.26:0/2159514957 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80a23a0 con 0x55d5d5eece20 2023-07-13T22:25:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.019+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/2159514957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 msgr2=0x55d5d5efd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.019+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8be8099c50 tx=0x7f8be80ca640 comp rx=0 tx=0).stop 2023-07-13T22:25:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.027+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/2159514957 shutdown_connections 2023-07-13T22:25:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.027+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.027+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eedc70 0x55d5d5ef4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.027+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/2159514957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eece20 0x55d5d5efd690 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.027+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/2159514957 >> 172.21.15.26:0/2159514957 conn(0x55d5d5ee1da0 msgr2=0x55d5d5f08ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.032+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/2159514957 shutdown_connections 2023-07-13T22:25:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.032+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/2159514957 wait complete. 2023-07-13T22:25:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.033+0000 7f8c08d2cb80 1 -- start start 2023-07-13T22:25:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.033+0000 7f8c08d2cb80 1 Processor -- start 2023-07-13T22:25:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eece20 0x55d5d5ef9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5efd3e0 con 0x55d5d5eece20 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5efd550 con 0x55d5d5eedc70 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5d5efd710 con 0x55d5d5eec4d0 2023-07-13T22:25:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.034+0000 7f8c08d2cb80 10 client.0 _ll_register_callbacks cb 0x7f8be80097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.035+0000 7f8beeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.035+0000 7f8beeffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47304/0 (socket says 172.21.15.26:47304) 2023-07-13T22:25:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.035+0000 7f8beeffd700 1 -- 172.21.15.26:0/1196835126 learned_addr learned my addr 172.21.15.26:0/1196835126 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.035+0000 7f8bef7fe700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 -- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 msgr2=0x55d5d5efcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 -- 172.21.15.26:0/1196835126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eece20 msgr2=0x55d5d5ef9e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eece20 0x55d5d5ef9e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 -- 172.21.15.26:0/1196835126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf006a670 con 0x55d5d5eec4d0 2023-07-13T22:25:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.036+0000 7f8beeffd700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009750 tx=0x7f8be0006c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f876d740a51381 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.041+0000 7f8bd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.041+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00235a0 con 0x55d5d5eec4d0 2023-07-13T22:25:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.041+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f8be0020040 con 0x55d5d5eec4d0 2023-07-13T22:25:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.041+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8be0011e50 con 0x55d5d5eec4d0 2023-07-13T22:25:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.042+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8be0010690 con 0x55d5d5eec4d0 2023-07-13T22:25:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.042+0000 7f8c08d2cb80 10 client.27319 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.042+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55d5d5c6a2b0 con 0x55d5d5eec4d0 2023-07-13T22:25:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.043+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8be0011e50 con 0x55d5d5eec4d0 2023-07-13T22:25:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.043+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8be0011630 con 0x55d5d5eec4d0 2023-07-13T22:25:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 10 client.27319 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 10 client.27319 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:25:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 20 client.27319 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f57460 0x55d5d5f59870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 4 client.27319 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.045+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5d5df9050 con 0x55d5d5f57460 2023-07-13T22:25:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.046+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f57460 0x55d5d5f59870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.064+0000 7f8bee7fc700 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f57460 0x55d5d5f59870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f6a846f96920771 server_cookie=4425af32fb8e943 in_seq=0 out_seq=0 2023-07-13T22:25:12.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.064+0000 7f8bd2ff5700 10 client.27319 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.065+0000 7f8bd2ff5700 1 -- 172.21.15.26:0/1196835126 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5d5df9050 con 0x55d5d5f57460 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.065+0000 7f8bd2ff5700 10 client.27319 handle_command_reply: tid=0 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.065+0000 7f8c08d2cb80 1 client.27319 shutdown 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.066+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 msgr2=0x55d5d5ef9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.066+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009750 tx=0x7f8be0006c60 comp rx=0 tx=0).stop 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 shutdown_connections 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x55d5d5f57460 0x55d5d5f59870 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5d5eedc70 0x55d5d5efcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d5d5eec4d0 0x55d5d5ef9940 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 --2- 172.21.15.26:0/1196835126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d5d5eece20 0x55d5d5ef9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 >> 172.21.15.26:0/1196835126 conn(0x55d5d5ee1da0 msgr2=0x55d5d5efdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.067+0000 7f8bd2ff5700 0 client.27319 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.077+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 shutdown_connections 2023-07-13T22:25:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.078+0000 7f8c08d2cb80 1 -- 172.21.15.26:0/1196835126 wait complete. 2023-07-13T22:25:12.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.087+0000 7f8c08d2cb80 20 client.27319 trim_cache size 0 max 16384 2023-07-13T22:25:12.074 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:12.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.094+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 msgr2=0x7f8bd80a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.094+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 0x7f8bd80a3800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8be4003f20 tx=0x7f8be4026000 comp rx=0 tx=0).stop 2023-07-13T22:25:12.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.094+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 msgr2=0x7f8bf010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ef10 tx=0x7f8be800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 shutdown_connections 2023-07-13T22:25:12.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f8bd80a1350 0x7f8bd80a3800 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf00ff470 0x7f8bf01ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bf00fd990 0x7f8bf010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 --2- 172.21.15.26:0/3798573712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bf00feac0 0x7f8bf010a920 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.095+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 >> 172.21.15.26:0/3798573712 conn(0x7f8bf00f9340 msgr2=0x7f8bf00f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.101+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 shutdown_connections 2023-07-13T22:25:12.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.101+0000 7f8bf53ff700 1 -- 172.21.15.26:0/3798573712 wait complete. 2023-07-13T22:25:12.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:25:12.277 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:12 smithi026 ceph-mon[95647]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:25:12.436 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:12 smithi117 ceph-mon[129074]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:25:12.486 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:12 smithi121 ceph-mon[129187]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 301 MiB/s wr, 97 op/s 2023-07-13T22:25:12.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 -- 172.21.15.26:0/3292602344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 msgr2=0x7f4f9010b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3292602344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 0x7f4f9010b7d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009990 tx=0x7f4f80009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 -- 172.21.15.26:0/3292602344 shutdown_connections 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3292602344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f90070ee0 0x7f4f900712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3292602344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f9010bd60 0x7f4f9010e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3292602344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 0x7f4f9010b7d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 -- 172.21.15.26:0/3292602344 >> 172.21.15.26:0/3292602344 conn(0x7f4f9006ca90 msgr2=0x7f4f9006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.664+0000 7f4f97dff700 1 -- 172.21.15.26:0/3292602344 shutdown_connections 2023-07-13T22:25:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.665+0000 7f4f97dff700 1 -- 172.21.15.26:0/3292602344 wait complete. 2023-07-13T22:25:12.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.666+0000 7f4f97dff700 1 Processor -- start 2023-07-13T22:25:12.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 -- start start 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 0x7f4f901a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f9010bd60 0x7f4f901abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901ac250 con 0x7f4f9010bd60 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901ac390 con 0x7f4f90071830 2023-07-13T22:25:12.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.667+0000 7f4f97dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901ac520 con 0x7f4f90070ee0 2023-07-13T22:25:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.668+0000 7f4f95b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.668+0000 7f4f95b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47328/0 (socket says 172.21.15.26:47328) 2023-07-13T22:25:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.668+0000 7f4f95b9b700 1 -- 172.21.15.26:0/3145225378 learned_addr learned my addr 172.21.15.26:0/3145225378 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 -- 172.21.15.26:0/3145225378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 msgr2=0x7f4f901a4c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-13T22:25:12.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 0x7f4f901a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 -- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f9010bd60 msgr2=0x7f4f901abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f9010bd60 0x7f4f901abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 -- 172.21.15.26:0/3145225378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800095f0 con 0x7f4f90070ee0 2023-07-13T22:25:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.669+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b530 tx=0x7f4f8c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.670+0000 7f4f86ffd700 1 -- 172.21.15.26:0/3145225378 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011830 con 0x7f4f90070ee0 2023-07-13T22:25:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.670+0000 7f4f86ffd700 1 -- 172.21.15.26:0/3145225378 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c011e50 con 0x7f4f90070ee0 2023-07-13T22:25:12.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.670+0000 7f4f86ffd700 1 -- 172.21.15.26:0/3145225378 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00f2e0 con 0x7f4f90070ee0 2023-07-13T22:25:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.671+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f901ac800 con 0x7f4f90070ee0 2023-07-13T22:25:12.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.671+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901acd10 con 0x7f4f90070ee0 2023-07-13T22:25:12.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.678+0000 7f4f86ffd700 1 -- 172.21.15.26:0/3145225378 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f8c010580 con 0x7f4f90070ee0 2023-07-13T22:25:12.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.679+0000 7f4fab746b80 1 Processor -- start 2023-07-13T22:25:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 -- start start 2023-07-13T22:25:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a7686808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a768585cb0 con 0x55a7686804d0 2023-07-13T22:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7685d8a40 con 0x55a768681c70 2023-07-13T22:25:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.680+0000 7f4fab746b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7685d1ed0 con 0x55a768680e20 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.681+0000 7f4f9539a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56224/0 (socket says 172.21.15.26:56224) 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 -- 172.21.15.26:0/2303852404 learned_addr learned my addr 172.21.15.26:0/2303852404 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 -- 172.21.15.26:0/2303852404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 msgr2=0x55a7686816e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686816e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 -- 172.21.15.26:0/2303852404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 msgr2=0x55a7686808e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a7686808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.682+0000 7f4f9539a700 1 -- 172.21.15.26:0/2303852404 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7683ffde0 con 0x55a768681c70 2023-07-13T22:25:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.683+0000 7f4f86ffd700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 0x7f4f7c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.683+0000 7f4f86ffd700 1 -- 172.21.15.26:0/3145225378 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f8c0cc3a0 con 0x7f4f90070ee0 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.683+0000 7f4f9539a700 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009510 tx=0x7f4f80009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1be77cd3874429 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.683+0000 7f4f7bfff700 1 -- 172.21.15.26:0/2303852404 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001d070 con 0x55a768681c70 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.684+0000 7f4f7bfff700 1 -- 172.21.15.26:0/2303852404 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8000bd90 con 0x55a768681c70 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.684+0000 7f4f7bfff700 1 -- 172.21.15.26:0/2303852404 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000fea0 con 0x55a768681c70 2023-07-13T22:25:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.685+0000 7f4f9639c700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 0x7f4f7c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.685+0000 7f4fab746b80 1 -- 172.21.15.26:0/2303852404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 msgr2=0x55a7686820e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.685+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009510 tx=0x7f4f80009540 comp rx=0 tx=0).stop 2023-07-13T22:25:12.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.686+0000 7f4fab746b80 1 -- 172.21.15.26:0/2303852404 shutdown_connections 2023-07-13T22:25:12.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.686+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a7686808e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.686+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686820e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.686+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2303852404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686816e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.686+0000 7f4fab746b80 1 -- 172.21.15.26:0/2303852404 >> 172.21.15.26:0/2303852404 conn(0x55a768675da0 msgr2=0x55a76867b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.688+0000 7f4f9639c700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 0x7f4f7c0a3800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e330 tx=0x7f4f8800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.688+0000 7f4fab746b80 1 -- 172.21.15.26:0/2303852404 shutdown_connections 2023-07-13T22:25:12.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.688+0000 7f4fab746b80 1 -- 172.21.15.26:0/2303852404 wait complete. 2023-07-13T22:25:12.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.693+0000 7f4fab746b80 1 -- start start 2023-07-13T22:25:12.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.695+0000 7f4fab746b80 1 Processor -- start 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768680e20 0x55a76869b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a7686a2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a768571ac0 con 0x55a768681c70 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7685717c0 con 0x55a768680e20 2023-07-13T22:25:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a768455920 con 0x55a7686804d0 2023-07-13T22:25:12.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4fab746b80 10 client.0 _ll_register_callbacks cb 0x55a76856f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:12.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4f95b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4f95b9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:50420/0 (socket says 172.21.15.26:47344) 2023-07-13T22:25:12.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4f95b9b700 1 -- 172.21.15.26:0/1080208361 learned_addr learned my addr 172.21.15.26:0/1080208361 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.697+0000 7f4f9539a700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768680e20 0x55a76869b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 -- 172.21.15.26:0/1080208361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768680e20 msgr2=0x55a76869b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768680e20 0x55a76869b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 -- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 msgr2=0x55a7686a2610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a7686a2610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 -- 172.21.15.26:0/1080208361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f90117640 con 0x55a7686804d0 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b500 tx=0x7f4f8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87ac08bc40668f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c010380 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f8c099760 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.698+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a76852b4f0 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8c09d790 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 10 client.27331 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a7683e8c60 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c0b3080 con 0x55a7686804d0 2023-07-13T22:25:12.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f8c099ae0 con 0x55a7686804d0 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 10 client.27331 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 10 client.27331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 20 client.27331 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686a6f40 0x55a7686a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 4 client.27331 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:12.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.699+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7685caeb0 con 0x55a7686a6f40 2023-07-13T22:25:12.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.701+0000 7f4f9539a700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686a6f40 0x55a7686a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.701+0000 7f4f9539a700 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686a6f40 0x55a7686a9350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cda3ef2854c83ea2 server_cookie=e66c30d578471cb5 in_seq=0 out_seq=0 2023-07-13T22:25:12.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.701+0000 7f4f797fa700 10 client.27331 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.703+0000 7f4f797fa700 1 -- 172.21.15.26:0/1080208361 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7685caeb0 con 0x55a7686a6f40 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.703+0000 7f4f797fa700 10 client.27331 handle_command_reply: tid=0 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.703+0000 7f4fab746b80 1 client.27331 shutdown 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.703+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 msgr2=0x55a76869b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.703+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b500 tx=0x7f4f8c004750 comp rx=0 tx=0).stop 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 shutdown_connections 2023-07-13T22:25:12.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686a6f40 0x55a7686a9350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a7686a2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a7686804d0 0x55a76869b090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 --2- 172.21.15.26:0/1080208361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768680e20 0x55a76869b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4f797fa700 0 client.27331 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 >> 172.21.15.26:0/1080208361 conn(0x55a768675da0 msgr2=0x55a76867a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 shutdown_connections 2023-07-13T22:25:12.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 1 -- 172.21.15.26:0/1080208361 wait complete. 2023-07-13T22:25:12.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.704+0000 7f4fab746b80 20 client.27331 trim_cache size 0 max 16384 2023-07-13T22:25:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 Processor -- start 2023-07-13T22:25:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 -- start start 2023-07-13T22:25:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a76869a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7686ae1b0 con 0x55a7686804d0 2023-07-13T22:25:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7686a6bc0 con 0x55a768681c70 2023-07-13T22:25:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.734+0000 7f4fab746b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a768680af0 con 0x55a768680e20 2023-07-13T22:25:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.735+0000 7f4f9639c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.735+0000 7f4f9639c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:56250/0 (socket says 172.21.15.26:56250) 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.735+0000 7f4f9639c700 1 -- 172.21.15.26:0/2924736388 learned_addr learned my addr 172.21.15.26:0/2924736388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a76869a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 -- 172.21.15.26:0/2924736388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 msgr2=0x55a7686a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 -- 172.21.15.26:0/2924736388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 msgr2=0x55a76869a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a76869a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 -- 172.21.15.26:0/2924736388 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a768426250 con 0x55a768681c70 2023-07-13T22:25:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.736+0000 7f4f9639c700 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009160 tx=0x7f4f88004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb0651904bacb58 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.737+0000 7f4f84bff700 1 -- 172.21.15.26:0/2924736388 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88008c50 con 0x55a768681c70 2023-07-13T22:25:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.737+0000 7f4f84bff700 1 -- 172.21.15.26:0/2924736388 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8800f040 con 0x55a768681c70 2023-07-13T22:25:12.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.737+0000 7f4f84bff700 1 -- 172.21.15.26:0/2924736388 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88015650 con 0x55a768681c70 2023-07-13T22:25:12.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 -- 172.21.15.26:0/2924736388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 msgr2=0x55a7686adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009160 tx=0x7f4f88004ad0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 -- 172.21.15.26:0/2924736388 shutdown_connections 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a7686804d0 0x55a76869a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a768681c70 0x55a7686adc70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 --2- 172.21.15.26:0/2924736388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.738+0000 7f4fab746b80 1 -- 172.21.15.26:0/2924736388 >> 172.21.15.26:0/2924736388 conn(0x55a768675da0 msgr2=0x55a768699dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.739+0000 7f4fab746b80 1 -- 172.21.15.26:0/2924736388 shutdown_connections 2023-07-13T22:25:12.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.739+0000 7f4fab746b80 1 -- 172.21.15.26:0/2924736388 wait complete. 2023-07-13T22:25:12.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.739+0000 7f4fab746b80 1 -- start start 2023-07-13T22:25:12.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.740+0000 7f4fab746b80 1 Processor -- start 2023-07-13T22:25:12.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.740+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7686804d0 0x55a7686a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.740+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4fab746b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a76869eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4fab746b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76869f230 con 0x55a768681c70 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4fab746b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76869f3a0 con 0x55a7686804d0 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4fab746b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a76869f560 con 0x55a768680e20 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4fab746b80 10 client.0 _ll_register_callbacks cb 0x55a76856f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4f9539a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4f9539a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:47368/0 (socket says 172.21.15.26:47368) 2023-07-13T22:25:12.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4f9539a700 1 -- 172.21.15.26:0/551142616 learned_addr learned my addr 172.21.15.26:0/551142616 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:12.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.741+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7686804d0 0x55a7686a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9639c700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a76869eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 -- 172.21.15.26:0/551142616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7686804d0 msgr2=0x55a7686a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7686804d0 0x55a7686a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 -- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 msgr2=0x55a76869eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a76869eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 -- 172.21.15.26:0/551142616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8800ee40 con 0x55a768680e20 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f9539a700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098e0 tx=0x7f4f8000f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d77a03dfbc68e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:12.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8001d040 con 0x55a768680e20 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4f8000b850 con 0x55a768680e20 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.742+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f8000bd90 con 0x55a768680e20 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.743+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8000fa30 con 0x55a768680e20 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.743+0000 7f4fab746b80 10 client.27334 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:12.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.743+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55a7683fe2b0 con 0x55a768680e20 2023-07-13T22:25:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.743+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8000bd90 con 0x55a768680e20 2023-07-13T22:25:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4f8003f070 con 0x55a768680e20 2023-07-13T22:25:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 10 client.27334 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 10 client.27334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:25:12.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 20 client.27334 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686fbfc0 0x55a7686fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 4 client.27334 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a76858d050 con 0x55a7686fbfc0 2023-07-13T22:25:12.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.745+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686fbfc0 0x55a7686fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:12.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.746+0000 7f4f95b9b700 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686fbfc0 0x55a7686fe3d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98358f1d67b9deb3 server_cookie=7d31581af05b1177 in_seq=0 out_seq=0 2023-07-13T22:25:12.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.748+0000 7f4f797fa700 10 client.27334 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.748+0000 7f4f797fa700 1 -- 172.21.15.26:0/551142616 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a76858d050 con 0x55a7686fbfc0 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.748+0000 7f4f797fa700 10 client.27334 handle_command_reply: tid=0 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.748+0000 7f4fab746b80 1 client.27334 shutdown 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 msgr2=0x55a7686a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098e0 tx=0x7f4f8000f510 comp rx=0 tx=0).stop 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 shutdown_connections 2023-07-13T22:25:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x55a7686fbfc0 0x55a7686fe3d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a768681c70 0x55a76869eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7686804d0 0x55a7686a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 --2- 172.21.15.26:0/551142616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a768680e20 0x55a7686a25c0 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098e0 tx=0x7f4f8000f510 comp rx=0 tx=0).stop 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.749+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 >> 172.21.15.26:0/551142616 conn(0x55a768675da0 msgr2=0x55a7686880d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.756+0000 7f4f797fa700 0 client.27334 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.756+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 shutdown_connections 2023-07-13T22:25:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.756+0000 7f4fab746b80 1 -- 172.21.15.26:0/551142616 wait complete. 2023-07-13T22:25:12.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.758+0000 7f4fab746b80 20 client.27334 trim_cache size 0 max 16384 2023-07-13T22:25:12.742 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 msgr2=0x7f4f7c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 0x7f4f7c0a3800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e330 tx=0x7f4f8800b040 comp rx=0 tx=0).stop 2023-07-13T22:25:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 msgr2=0x7f4f901a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b530 tx=0x7f4f8c00b8f0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 shutdown_connections 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f4f7c0a1350 0x7f4f7c0a3800 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f9010bd60 0x7f4f901abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4f90070ee0 0x7f4f901a46d0 secure :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b530 tx=0x7f4f8c00b8f0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 --2- 172.21.15.26:0/3145225378 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f90071830 0x7f4f901a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.761+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 >> 172.21.15.26:0/3145225378 conn(0x7f4f9006ca90 msgr2=0x7f4f9006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.762+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 shutdown_connections 2023-07-13T22:25:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:12.762+0000 7f4f97dff700 1 -- 172.21.15.26:0/3145225378 wait complete. 2023-07-13T22:25:12.783 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:25:12.788 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-07-13T22:25:12.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-07-13T22:25:13.092 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:13 smithi026 ceph-mon[95647]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 93 op/s 2023-07-13T22:25:13.094 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2023-07-13T22:25:13.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-13T22:25:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.122+0000 7f5ee4dff700 1 -- 172.21.15.26:0/4246437078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0102e80 msgr2=0x7f5ee0105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.122+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/4246437078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0102e80 0x7f5ee0105260 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009990 tx=0x7f5ed4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 -- 172.21.15.26:0/4246437078 shutdown_connections 2023-07-13T22:25:13.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/4246437078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0100560 0x7f5ee0102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/4246437078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 0x7f5ee0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/4246437078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0102e80 0x7f5ee0105260 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 -- 172.21.15.26:0/4246437078 >> 172.21.15.26:0/4246437078 conn(0x7f5ee00fa000 msgr2=0x7f5ee00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 -- 172.21.15.26:0/4246437078 shutdown_connections 2023-07-13T22:25:13.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.123+0000 7f5ee4dff700 1 -- 172.21.15.26:0/4246437078 wait complete. 2023-07-13T22:25:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.124+0000 7f5ee4dff700 1 Processor -- start 2023-07-13T22:25:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.124+0000 7f5ee4dff700 1 -- start start 2023-07-13T22:25:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.124+0000 7f5ee4dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.124+0000 7f5ee4dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0102e80 0x7f5ee0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ee4dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 0x7f5ee019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ee4dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee019f5f0 con 0x7f5ee0102e80 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ee4dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee019f730 con 0x7f5ee01057a0 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ee4dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee019f8c0 con 0x7f5ee0100560 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40310/0 (socket says 172.21.15.26:40310) 2023-07-13T22:25:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 -- 172.21.15.26:0/3035339909 learned_addr learned my addr 172.21.15.26:0/3035339909 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:13.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0102e80 0x7f5ee0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5eded9d700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 0x7f5ee019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 -- 172.21.15.26:0/3035339909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 msgr2=0x7f5ee019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 0x7f5ee019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 -- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0102e80 msgr2=0x7f5ee0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0102e80 0x7f5ee0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.125+0000 7f5ede59c700 1 -- 172.21.15.26:0/3035339909 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40095f0 con 0x7f5ee0100560 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ede59c700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000e9a0 tx=0x7f5ed000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/3035339909 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0009790 con 0x7f5ee0100560 2023-07-13T22:25:13.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/3035339909 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed000cbd0 con 0x7f5ee0100560 2023-07-13T22:25:13.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee019fba0 con 0x7f5ee0100560 2023-07-13T22:25:13.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/3035339909 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0018680 con 0x7f5ee0100560 2023-07-13T22:25:13.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.126+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee01a0100 con 0x7f5ee0100560 2023-07-13T22:25:13.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.135+0000 7f5ef8785b80 1 Processor -- start 2023-07-13T22:25:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.135+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/3035339909 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ed000cd40 con 0x7f5ee0100560 2023-07-13T22:25:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 -- start start 2023-07-13T22:25:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029557e20 0x55b0295586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02945ccb0 con 0x55b029557e20 2023-07-13T22:25:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0294afa40 con 0x55b029558c70 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.136+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0294a8ed0 con 0x55b0295574d0 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40312/0 (socket says 172.21.15.26:40312) 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40078/0 (socket says 172.21.15.26:40078) 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 -- 172.21.15.26:0/1205439575 learned_addr learned my addr 172.21.15.26:0/1205439575 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5ede59c700 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029557e20 0x55b0295586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 -- 172.21.15.26:0/1205439575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 msgr2=0x55b0295590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 -- 172.21.15.26:0/1205439575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029557e20 msgr2=0x55b0295586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029557e20 0x55b0295586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.137+0000 7f5eded9d700 1 -- 172.21.15.26:0/1205439575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0292d6de0 con 0x55b0295574d0 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.138+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009440 tx=0x7f5ed8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc7449109b3da56 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.138+0000 7f5eccff9700 1 -- 172.21.15.26:0/1205439575 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800e070 con 0x55b0295574d0 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.138+0000 7f5eccff9700 1 -- 172.21.15.26:0/1205439575 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed800f040 con 0x55b0295574d0 2023-07-13T22:25:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.138+0000 7f5eccff9700 1 -- 172.21.15.26:0/1205439575 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8015b60 con 0x55b0295574d0 2023-07-13T22:25:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.140+0000 7f5ecf7fe700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 0x7f5ec80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.140+0000 7f5ecf7fe700 1 -- 172.21.15.26:0/3035339909 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ed0014070 con 0x7f5ee0100560 2023-07-13T22:25:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.140+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1205439575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 msgr2=0x55b0295578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.140+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009440 tx=0x7f5ed8009920 comp rx=0 tx=0).stop 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1205439575 shutdown_connections 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1205439575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029557e20 0x55b0295586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1205439575 >> 172.21.15.26:0/1205439575 conn(0x55b02954cda0 msgr2=0x55b0295521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ede59c700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 0x7f5ec80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.141+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1205439575 shutdown_connections 2023-07-13T22:25:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.142+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1205439575 wait complete. 2023-07-13T22:25:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.142+0000 7f5ede59c700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 0x7f5ec80a3700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0096a00 tx=0x7f5ed001f000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.143+0000 7f5ef8785b80 1 -- start start 2023-07-13T22:25:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.143+0000 7f5ef8785b80 1 Processor -- start 2023-07-13T22:25:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 0x55b029561160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b029448ac0 con 0x55b0295574d0 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0294487c0 con 0x55b029558c70 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.145+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02932c920 con 0x55b029557e20 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5ef8785b80 10 client.0 _ll_register_callbacks cb 0x55b029446760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40322/0 (socket says 172.21.15.26:40322) 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 -- 172.21.15.26:0/1356457480 learned_addr learned my addr 172.21.15.26:0/1356457480 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 0x55b029561160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 -- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 msgr2=0x55b0295686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 -- 172.21.15.26:0/1356457480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 msgr2=0x55b029561160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 0x55b029561160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eddd9b700 1 -- 172.21.15.26:0/1356457480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed80090f0 con 0x55b029557e20 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 0x55b029561160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.146+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0294024f0 con 0x55b029557e20 2023-07-13T22:25:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4015000 tx=0x7f5ed400bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9881566c477839 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ec5ffb700 10 client.27346 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/1356457480 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4027070 con 0x55b029557e20 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/1356457480 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ed4004500 con 0x55b029557e20 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/1356457480 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed40046e0 con 0x55b029557e20 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ef8785b80 10 client.27346 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.147+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x55b0292bfc60 con 0x55b029557e20 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/1356457480 <== mon.1 v2:172.21.15.121:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ed402a020 con 0x55b029557e20 2023-07-13T22:25:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 10 client.27346 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 10 client.27346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 20 client.27346 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b02956cdc0 0x55b02956f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 4 client.27346 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0294a1eb0 con 0x55b02956cdc0 2023-07-13T22:25:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.148+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b02956cdc0 0x55b02956f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.149+0000 7f5ec5ffb700 10 client.27346 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.149+0000 7f5eded9d700 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b02956cdc0 0x55b02956f1d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b2f506a71fe9d1 server_cookie=d5b92f940ae3e87f in_seq=0 out_seq=0 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.149+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/1356457480 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b0294a1eb0 con 0x55b02956cdc0 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.149+0000 7f5ec5ffb700 10 client.27346 handle_command_reply: tid=0 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.149+0000 7f5ef8785b80 1 client.27346 shutdown 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 msgr2=0x55b0295616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4015000 tx=0x7f5ed400bf60 comp rx=0 tx=0).stop 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 shutdown_connections 2023-07-13T22:25:13.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b0295574d0 0x55b029561160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b02956cdc0 0x55b02956f1d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029558c70 0x55b0295686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ec5ffb700 0 client.27346 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/1356457480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b029557e20 0x55b0295616a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 >> 172.21.15.26:0/1356457480 conn(0x55b02954cda0 msgr2=0x55b0295518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 shutdown_connections 2023-07-13T22:25:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.150+0000 7f5ef8785b80 1 -- 172.21.15.26:0/1356457480 wait complete. 2023-07-13T22:25:13.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.151+0000 7f5ef8785b80 20 client.27346 trim_cache size 0 max 16384 2023-07-13T22:25:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.172+0000 7f5ef8785b80 1 Processor -- start 2023-07-13T22:25:13.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 -- start start 2023-07-13T22:25:13.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b02955fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b02955eef0 con 0x55b029558c70 2023-07-13T22:25:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0295622e0 con 0x55b029557e20 2023-07-13T22:25:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0295603c0 con 0x55b0295574d0 2023-07-13T22:25:13.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5eded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.173+0000 7f5eddd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40090/0 (socket says 172.21.15.26:40090) 2023-07-13T22:25:13.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eddd9b700 1 -- 172.21.15.26:0/3571715472 learned_addr learned my addr 172.21.15.26:0/3571715472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b02955fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 -- 172.21.15.26:0/3571715472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 msgr2=0x55b0295686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 -- 172.21.15.26:0/3571715472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 msgr2=0x55b02955fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b02955fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 -- 172.21.15.26:0/3571715472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0292fd250 con 0x55b0295574d0 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:13.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.174+0000 7f5eded9d700 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55b02956fb00 tx=0x7f5ed8017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd85697e2d8f5bb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.175+0000 7f5ecd7fa700 1 -- 172.21.15.26:0/3571715472 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed800e040 con 0x55b0295574d0 2023-07-13T22:25:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.175+0000 7f5ecd7fa700 1 -- 172.21.15.26:0/3571715472 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed802b020 con 0x55b0295574d0 2023-07-13T22:25:13.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.175+0000 7f5ecd7fa700 1 -- 172.21.15.26:0/3571715472 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8015c00 con 0x55b0295574d0 2023-07-13T22:25:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 -- 172.21.15.26:0/3571715472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 msgr2=0x55b0295578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55b02956fb00 tx=0x7f5ed8017040 comp rx=0 tx=0).stop 2023-07-13T22:25:13.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 -- 172.21.15.26:0/3571715472 shutdown_connections 2023-07-13T22:25:13.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b02955fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295578a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/3571715472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b0295686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 -- 172.21.15.26:0/3571715472 >> 172.21.15.26:0/3571715472 conn(0x55b02954cda0 msgr2=0x55b029573b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 -- 172.21.15.26:0/3571715472 shutdown_connections 2023-07-13T22:25:13.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.176+0000 7f5ef8785b80 1 -- 172.21.15.26:0/3571715472 wait complete. 2023-07-13T22:25:13.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.177+0000 7f5ef8785b80 1 -- start start 2023-07-13T22:25:13.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.177+0000 7f5ef8785b80 1 Processor -- start 2023-07-13T22:25:13.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b029564c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b0295651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b029565880 con 0x55b029558c70 2023-07-13T22:25:13.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b029568840 con 0x55b029557e20 2023-07-13T22:25:13.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0295689d0 con 0x55b0295574d0 2023-07-13T22:25:13.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5ef8785b80 10 client.0 _ll_register_callbacks cb 0x7f5ed0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:13.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5eded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5eded9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40334/0 (socket says 172.21.15.26:40334) 2023-07-13T22:25:13.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5eded9d700 1 -- 172.21.15.26:0/4122868009 learned_addr learned my addr 172.21.15.26:0/4122868009 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:13.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.178+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b029564c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eded9d700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b0295651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 -- 172.21.15.26:0/4122868009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 msgr2=0x55b0295681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 -- 172.21.15.26:0/4122868009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 msgr2=0x55b0295651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:13.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b0295651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 -- 172.21.15.26:0/4122868009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0292bfc60 con 0x55b029557e20 2023-07-13T22:25:13.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eded9d700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:13.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5eddd9b700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b029564c60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4001eb0 tx=0x7f5ed4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b7715a70097335 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:13.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.179+0000 7f5ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:13.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4027070 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5ed4000e40 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ed800f040 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed4004700 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ef8785b80 10 client.27359 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.180+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b0292d6de0 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ed800f040 con 0x55b029557e20 2023-07-13T22:25:13.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5ed402a070 con 0x55b029557e20 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 10 client.27359 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 20 client.27359 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b0295ca7d0 0x55b0295ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 4 client.27359 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b029464050 con 0x55b0295ca7d0 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.181+0000 7f5eded9d700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b0295ca7d0 0x55b0295ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:13.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.182+0000 7f5ec5ffb700 10 client.27359 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.182+0000 7f5eded9d700 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b0295ca7d0 0x55b0295ccbb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6262cef100b084c server_cookie=9dd0b7bd6c5d82b3 in_seq=0 out_seq=0 2023-07-13T22:25:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.182+0000 7f5ec5ffb700 1 -- 172.21.15.26:0/4122868009 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b029464050 con 0x55b0295ca7d0 2023-07-13T22:25:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.182+0000 7f5ec5ffb700 10 client.27359 handle_command_reply: tid=0 2023-07-13T22:25:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.182+0000 7f5ef8785b80 1 client.27359 shutdown 2023-07-13T22:25:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 msgr2=0x55b029564c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b029564c60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4001eb0 tx=0x7f5ed4009310 comp rx=0 tx=0).stop 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 shutdown_connections 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55b029558c70 0x55b0295651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55b0295ca7d0 0x55b0295ccbb0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b0295574d0 0x55b0295681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 --2- 172.21.15.26:0/4122868009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b029557e20 0x55b029564c60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 >> 172.21.15.26:0/4122868009 conn(0x55b02954cda0 msgr2=0x55b029568fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ec5ffb700 0 client.27359 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:13.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 shutdown_connections 2023-07-13T22:25:13.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.183+0000 7f5ef8785b80 1 -- 172.21.15.26:0/4122868009 wait complete. 2023-07-13T22:25:13.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ef8785b80 20 client.27359 trim_cache size 0 max 16384 2023-07-13T22:25:13.176 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 msgr2=0x7f5ec80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 0x7f5ec80a3700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0096a00 tx=0x7f5ed001f000 comp rx=0 tx=0).stop 2023-07-13T22:25:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 msgr2=0x7f5ee0197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000e9a0 tx=0x7f5ed000ed60 comp rx=0 tx=0).stop 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 shutdown_connections 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee01057a0 0x7f5ee019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ee0100560 0x7f5ee0197a70 secure :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5ed000e9a0 tx=0x7f5ed000ed60 comp rx=0 tx=0).stop 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f5ec80a1250 0x7f5ec80a3700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 --2- 172.21.15.26:0/3035339909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ee0102e80 0x7f5ee0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.184+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 >> 172.21.15.26:0/3035339909 conn(0x7f5ee00fa000 msgr2=0x7f5ee00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:13.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.185+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 shutdown_connections 2023-07-13T22:25:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:13.185+0000 7f5ee4dff700 1 -- 172.21.15.26:0/3035339909 wait complete. 2023-07-13T22:25:13.435 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:13 smithi117 ceph-mon[129074]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 93 op/s 2023-07-13T22:25:13.486 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:13 smithi121 ceph-mon[129187]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 277 MiB/s wr, 93 op/s 2023-07-13T22:25:14.204 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-13T22:25:14.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-13T22:25:14.592 DEBUG:teuthology.parallel:result is None 2023-07-13T22:25:14.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-07-13T22:25:14.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.629+0000 7fa21c1ff700 1 -- 172.21.15.26:0/798473754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fec60 msgr2=0x7fa2140ff0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.629+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/798473754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fec60 0x7fa2140ff0d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa2040098f0 tx=0x7fa204009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:14.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 -- 172.21.15.26:0/798473754 shutdown_connections 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/798473754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2140ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/798473754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fe310 0x7fa2140fe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/798473754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fec60 0x7fa2140ff0d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 -- 172.21.15.26:0/798473754 >> 172.21.15.26:0/798473754 conn(0x7fa2140f9fa0 msgr2=0x7fa2140fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 -- 172.21.15.26:0/798473754 shutdown_connections 2023-07-13T22:25:14.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.630+0000 7fa21c1ff700 1 -- 172.21.15.26:0/798473754 wait complete. 2023-07-13T22:25:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.631+0000 7fa21c1ff700 1 Processor -- start 2023-07-13T22:25:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 -- start start 2023-07-13T22:25:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fe310 0x7fa214103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 0x7fa214104380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2141009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa214101170 con 0x7fa2140fe310 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141012e0 con 0x7fa2140ff610 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.632+0000 7fa21c1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141014a0 con 0x7fa2140fec60 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.633+0000 7fa21979a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 0x7fa214104380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.633+0000 7fa21a79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2141009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.633+0000 7fa21a79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2141009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40116/0 (socket says 172.21.15.26:40116) 2023-07-13T22:25:14.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.633+0000 7fa21a79c700 1 -- 172.21.15.26:0/3959507066 learned_addr learned my addr 172.21.15.26:0/3959507066 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa219f9b700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fe310 0x7fa214103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa21979a700 1 -- 172.21.15.26:0/3959507066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 msgr2=0x7fa2141009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa21979a700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2141009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa21979a700 1 -- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fe310 msgr2=0x7fa214103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa21979a700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fe310 0x7fa214103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.634+0000 7fa21979a700 1 -- 172.21.15.26:0/3959507066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009520 con 0x7fa2140fec60 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.635+0000 7fa21979a700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 0x7fa214104380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa20400b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.636+0000 7fa20affd700 1 -- 172.21.15.26:0/3959507066 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20401c070 con 0x7fa2140fec60 2023-07-13T22:25:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.636+0000 7fa20affd700 1 -- 172.21.15.26:0/3959507066 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa204005040 con 0x7fa2140fec60 2023-07-13T22:25:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.636+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa214197710 con 0x7fa2140fec60 2023-07-13T22:25:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.636+0000 7fa20affd700 1 -- 172.21.15.26:0/3959507066 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204003a00 con 0x7fa2140fec60 2023-07-13T22:25:14.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.636+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa214197be0 con 0x7fa2140fec60 2023-07-13T22:25:14.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.645+0000 7fa22fb7eb80 1 Processor -- start 2023-07-13T22:25:14.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.646+0000 7fa22fb7eb80 1 -- start start 2023-07-13T22:25:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.646+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e5fe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.646+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5ffcf0 0x56456e600160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e503d30 con 0x56456e5fe550 2023-07-13T22:25:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e556ac0 con 0x56456e5feea0 2023-07-13T22:25:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e54ff50 con 0x56456e5ffcf0 2023-07-13T22:25:14.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40128/0 (socket says 172.21.15.26:40128) 2023-07-13T22:25:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa219f9b700 1 -- 172.21.15.26:0/856901546 learned_addr learned my addr 172.21.15.26:0/856901546 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:14.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.647+0000 7fa21979a700 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5ffcf0 0x56456e600160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 -- 172.21.15.26:0/856901546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5ffcf0 msgr2=0x56456e600160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5ffcf0 0x56456e600160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 -- 172.21.15.26:0/856901546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 msgr2=0x56456e5fe960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e5fe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 -- 172.21.15.26:0/856901546 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456e37de60 con 0x56456e5feea0 2023-07-13T22:25:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.648+0000 7fa219f9b700 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa210009ce0 tx=0x7fa2100113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b942d0a2be5493 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.649+0000 7fa1fffff700 1 -- 172.21.15.26:0/856901546 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210011d10 con 0x56456e5feea0 2023-07-13T22:25:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.649+0000 7fa1fffff700 1 -- 172.21.15.26:0/856901546 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210011e70 con 0x56456e5feea0 2023-07-13T22:25:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.649+0000 7fa1fffff700 1 -- 172.21.15.26:0/856901546 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000f820 con 0x56456e5feea0 2023-07-13T22:25:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.650+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/856901546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 msgr2=0x56456e5ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.650+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa210009ce0 tx=0x7fa2100113e0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.650+0000 7fa20affd700 1 -- 172.21.15.26:0/3959507066 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa204020030 con 0x7fa2140fec60 2023-07-13T22:25:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.651+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/856901546 shutdown_connections 2023-07-13T22:25:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.651+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e5fe960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.651+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5ffcf0 0x56456e600160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.651+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/856901546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5feea0 0x56456e5ff760 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.651+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/856901546 >> 172.21.15.26:0/856901546 conn(0x56456e5f3e20 msgr2=0x56456e5f9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.654+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/856901546 shutdown_connections 2023-07-13T22:25:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.654+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/856901546 wait complete. 2023-07-13T22:25:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.656+0000 7fa22fb7eb80 1 -- start start 2023-07-13T22:25:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.657+0000 7fa22fb7eb80 1 Processor -- start 2023-07-13T22:25:14.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.659+0000 7fa20affd700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 0x7fa2000a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.659+0000 7fa20affd700 1 -- 172.21.15.26:0/3959507066 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa2040cd710 con 0x7fa2140fec60 2023-07-13T22:25:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.660+0000 7fa21a79c700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 0x7fa2000a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.661+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.661+0000 7fa21a79c700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 0x7fa2000a3610 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004b10 tx=0x7fa20c009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.661+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40134/0 (socket says 172.21.15.26:40134) 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 0x56456e60f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e4efb40 con 0x56456e5ffcf0 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa21979a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e4ef840 con 0x56456e5fe550 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e3d39a0 con 0x56456e5feea0 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa21979a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40396/0 (socket says 172.21.15.26:40396) 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa21979a700 1 -- 172.21.15.26:0/2622335562 learned_addr learned my addr 172.21.15.26:0/2622335562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.662+0000 7fa21a79c700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 0x56456e60f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa22fb7eb80 10 client.0 _ll_register_callbacks cb 0x56456e4ed7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 -- 172.21.15.26:0/2622335562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 msgr2=0x56456e6087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 -- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 msgr2=0x56456e608280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 -- 172.21.15.26:0/2622335562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa210009940 con 0x56456e5ffcf0 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21979a700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56456e4a9570 con 0x56456e5ffcf0 2023-07-13T22:25:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa21a79c700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 0x56456e60f800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa20c008a90 tx=0x7fa20c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab387cefe645aa32 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.663+0000 7fa1fd7fa700 10 client.16707 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:25:14.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.664+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2622335562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c026070 con 0x56456e5ffcf0 2023-07-13T22:25:14.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.664+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2622335562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa20c017ae0 con 0x56456e5ffcf0 2023-07-13T22:25:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.664+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2622335562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa20c01c040 con 0x56456e5ffcf0 2023-07-13T22:25:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 10 client.16707 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({fsmap=29}) v3 -- 0x56456e366ce0 con 0x56456e5ffcf0 2023-07-13T22:25:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2622335562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa20c029030 con 0x56456e5ffcf0 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 10 client.16707 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 20 client.16707 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:14.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e613eb0 0x56456e6162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 4 client.16707 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.665+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56456e548f30 con 0x56456e613eb0 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.666+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e613eb0 0x56456e6162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.668+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e613eb0 0x56456e6162c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431992a17b70a18 server_cookie=b3976de7e7064278 in_seq=0 out_seq=0 2023-07-13T22:25:14.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.669+0000 7fa1fd7fa700 10 client.16707 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.669+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2622335562 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56456e548f30 con 0x56456e613eb0 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.669+0000 7fa1fd7fa700 10 client.16707 handle_command_reply: tid=0 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.670+0000 7fa22fb7eb80 1 client.16707 shutdown 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.670+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 msgr2=0x56456e60f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.670+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 0x56456e60f800 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa20c008a90 tx=0x7fa20c004b40 comp rx=0 tx=0).stop 2023-07-13T22:25:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 shutdown_connections 2023-07-13T22:25:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5ffcf0 0x56456e60f800 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e613eb0 0x56456e6162c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5fe550 0x56456e608280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2622335562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e6087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.672+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 >> 172.21.15.26:0/2622335562 conn(0x56456e5f3e20 msgr2=0x56456e5f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.677+0000 7fa1fd7fa700 0 client.16707 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:14.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.678+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 shutdown_connections 2023-07-13T22:25:14.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.678+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2622335562 wait complete. 2023-07-13T22:25:14.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.679+0000 7fa22fb7eb80 20 client.16707 trim_cache size 0 max 16384 2023-07-13T22:25:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 Processor -- start 2023-07-13T22:25:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 -- start start 2023-07-13T22:25:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e603130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e5ff270 con 0x56456e5fe550 2023-07-13T22:25:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e5fa540 con 0x56456e5ffcf0 2023-07-13T22:25:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.717+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e5f8510 con 0x56456e5feea0 2023-07-13T22:25:14.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.719+0000 7fa21a79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.719+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.719+0000 7fa219f9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59500/0 (socket says 172.21.15.26:59500) 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.719+0000 7fa219f9b700 1 -- 172.21.15.26:0/2065208328 learned_addr learned my addr 172.21.15.26:0/2065208328 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 -- 172.21.15.26:0/2065208328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 msgr2=0x56456e603130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e603130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 -- 172.21.15.26:0/2065208328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 msgr2=0x56456e60f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 -- 172.21.15.26:0/2065208328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56456e5f5660 con 0x56456e5fe550 2023-07-13T22:25:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.720+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa210011e40 tx=0x7fa2100098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c117ed2d0f4bec90 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.721+0000 7fa208ff9700 1 -- 172.21.15.26:0/2065208328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210005400 con 0x56456e5fe550 2023-07-13T22:25:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.721+0000 7fa208ff9700 1 -- 172.21.15.26:0/2065208328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210005560 con 0x56456e5fe550 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.721+0000 7fa208ff9700 1 -- 172.21.15.26:0/2065208328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210012870 con 0x56456e5fe550 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.721+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2065208328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 msgr2=0x56456e602bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.721+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa210011e40 tx=0x7fa2100098a0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.722+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2065208328 shutdown_connections 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.722+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e5fe550 0x56456e602bf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.722+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.722+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2065208328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5feea0 0x56456e603130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.722+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2065208328 >> 172.21.15.26:0/2065208328 conn(0x56456e5f3e20 msgr2=0x56456e606090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.723+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2065208328 shutdown_connections 2023-07-13T22:25:14.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.723+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2065208328 wait complete. 2023-07-13T22:25:14.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.723+0000 7fa22fb7eb80 1 -- start start 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 Processor -- start 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 0x56456e60a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e60a750 0x56456e678db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e60ad60 con 0x56456e60a750 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e6792f0 con 0x56456e5ffcf0 2023-07-13T22:25:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x56456e679430 con 0x56456e5fe550 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.724+0000 7fa22fb7eb80 10 client.0 _ll_register_callbacks cb 0x7fa20c0093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa21979a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa21979a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:40154/0 (socket says 172.21.15.26:40154) 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa21979a700 1 -- 172.21.15.26:0/2744179078 learned_addr learned my addr 172.21.15.26:0/2744179078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 0x56456e60a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa21a79c700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e60a750 0x56456e678db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 -- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 msgr2=0x56456e60f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 -- 172.21.15.26:0/2744179078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e60a750 msgr2=0x56456e678db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e60a750 0x56456e678db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.725+0000 7fa219f9b700 1 -- 172.21.15.26:0/2744179078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56456e37de60 con 0x56456e5fe550 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa219f9b700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 0x56456e60a210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa210005010 tx=0x7fa210003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e3b0da9f519ced server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:14.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa1fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:14.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210003a60 con 0x56456e5fe550 2023-07-13T22:25:14.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa210003bc0 con 0x56456e5fe550 2023-07-13T22:25:14.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa210011e70 con 0x56456e5fe550 2023-07-13T22:25:14.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.726+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa210003d80 con 0x56456e5fe550 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.727+0000 7fa22fb7eb80 10 client.27367 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.727+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56456e3a42d0 con 0x56456e5fe550 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.727+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210011e70 con 0x56456e5fe550 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fa21002e740 con 0x56456e5fe550 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 10 client.27367 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 10 client.27367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:14.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 20 client.27367 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e67a9d0 0x56456e67cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 4 client.27367 mds_command: new command op to 24575 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56456e50b0d0 con 0x56456e67a9d0 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.728+0000 7fa21979a700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e67a9d0 0x56456e67cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.729+0000 7fa21979a700 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e67a9d0 0x56456e67cdb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4dcd2dfe7d3e93 server_cookie=949e047c51fb657a in_seq=0 out_seq=0 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.729+0000 7fa1fd7fa700 10 client.27367 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa1fd7fa700 1 -- 172.21.15.26:0/2744179078 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56456e50b0d0 con 0x56456e67a9d0 2023-07-13T22:25:14.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa1fd7fa700 10 client.27367 handle_command_reply: tid=0 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 client.27367 shutdown 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 msgr2=0x56456e60a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 0x56456e60a210 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa210005010 tx=0x7fa210003730 comp rx=0 tx=0).stop 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 shutdown_connections 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56456e60a750 0x56456e678db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56456e67a9d0 0x56456e67cdb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56456e5fe550 0x56456e60a210 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 --2- 172.21.15.26:0/2744179078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56456e5ffcf0 0x56456e60f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa1fd7fa700 0 client.27367 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.730+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 >> 172.21.15.26:0/2744179078 conn(0x56456e5f3e20 msgr2=0x56456e602bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.731+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 shutdown_connections 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.731+0000 7fa22fb7eb80 1 -- 172.21.15.26:0/2744179078 wait complete. 2023-07-13T22:25:14.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.731+0000 7fa22fb7eb80 20 client.27367 trim_cache size 0 max 16384 2023-07-13T22:25:14.718 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:14.718 INFO:teuthology.orchestra.run.smithi026.stdout: "return_code": 0, 2023-07-13T22:25:14.718 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_tag": "15cc61e9-d393-4879-b1d1-4eba9441c4f3", 2023-07-13T22:25:14.718 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": "asynchronous" 2023-07-13T22:25:14.718 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:14.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 msgr2=0x7fa2000a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 0x7fa2000a3610 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa20c004b10 tx=0x7fa20c009920 comp rx=0 tx=0).stop 2023-07-13T22:25:14.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 msgr2=0x7fa214104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:14.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 0x7fa214104380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa204000c00 tx=0x7fa20400b560 comp rx=0 tx=0).stop 2023-07-13T22:25:14.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 shutdown_connections 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fa2000a1160 0x7fa2000a3610 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2140fe310 0x7fa214103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2140ff610 0x7fa2141009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 --2- 172.21.15.26:0/3959507066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa2140fec60 0x7fa214104380 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.732+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 >> 172.21.15.26:0/3959507066 conn(0x7fa2140f9fa0 msgr2=0x7fa2140fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.733+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 shutdown_connections 2023-07-13T22:25:14.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:14.733+0000 7fa21c1ff700 1 -- 172.21.15.26:0/3959507066 wait complete. 2023-07-13T22:25:14.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:25:14.799 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-07-13T22:25:14.799 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-13T22:25:14.810 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-13T22:25:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.072+0000 7fe28ebff700 1 -- 172.21.15.26:0/4280209216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 msgr2=0x7fe2880733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.072+0000 7fe28ebff700 1 --2- 172.21.15.26:0/4280209216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe2880733e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe274009990 tx=0x7fe274009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.072+0000 7fe28ebff700 1 -- 172.21.15.26:0/4280209216 shutdown_connections 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 --2- 172.21.15.26:0/4280209216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2881079a0 0x7fe288109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 --2- 172.21.15.26:0/4280209216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe288074550 0x7fe2880729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 --2- 172.21.15.26:0/4280209216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe2880733e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 -- 172.21.15.26:0/4280209216 >> 172.21.15.26:0/4280209216 conn(0x7fe2880fb0e0 msgr2=0x7fe2880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 -- 172.21.15.26:0/4280209216 shutdown_connections 2023-07-13T22:25:15.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.073+0000 7fe28ebff700 1 -- 172.21.15.26:0/4280209216 wait complete. 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 Processor -- start 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 -- start start 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe288074550 0x7fe288198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2881079a0 0x7fe28819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819f650 con 0x7fe288074550 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819f790 con 0x7fe2881079a0 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.074+0000 7fe28ebff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28819f920 con 0x7fe288072f70 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe288074550 0x7fe288198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28d19c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2881079a0 0x7fe28819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40442/0 (socket says 172.21.15.26:40442) 2023-07-13T22:25:15.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 -- 172.21.15.26:0/872247782 learned_addr learned my addr 172.21.15.26:0/872247782 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 -- 172.21.15.26:0/872247782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2881079a0 msgr2=0x7fe28819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2881079a0 0x7fe28819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 -- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe288074550 msgr2=0x7fe288198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe288074550 0x7fe288198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 -- 172.21.15.26:0/872247782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740095f0 con 0x7fe288072f70 2023-07-13T22:25:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.075+0000 7fe28c99b700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e970 tx=0x7fe27c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.076+0000 7fe285ffb700 1 -- 172.21.15.26:0/872247782 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c009790 con 0x7fe288072f70 2023-07-13T22:25:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.076+0000 7fe285ffb700 1 -- 172.21.15.26:0/872247782 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27c00c7e0 con 0x7fe288072f70 2023-07-13T22:25:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.076+0000 7fe285ffb700 1 -- 172.21.15.26:0/872247782 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0054e0 con 0x7fe288072f70 2023-07-13T22:25:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.076+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28819fc00 con 0x7fe288072f70 2023-07-13T22:25:15.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.076+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2881a00c0 con 0x7fe288072f70 2023-07-13T22:25:15.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.081+0000 7fe285ffb700 1 -- 172.21.15.26:0/872247782 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe27c00c950 con 0x7fe288072f70 2023-07-13T22:25:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 Processor -- start 2023-07-13T22:25:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 -- start start 2023-07-13T22:25:15.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c6550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 0x55825c655ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c656480 0x55825c6568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c551cb0 con 0x55825c654ce0 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c5a4a40 con 0x55825c655630 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.082+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c59ded0 con 0x55825c656480 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe28d19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c6550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 0x55825c655ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe28d19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c6550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59522/0 (socket says 172.21.15.26:59522) 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe28d19c700 1 -- 172.21.15.26:0/479768246 learned_addr learned my addr 172.21.15.26:0/479768246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe287fff700 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c656480 0x55825c6568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe285ffb700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 0x7fe2780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.083+0000 7fe28c99b700 1 -- 172.21.15.26:0/479768246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c656480 msgr2=0x55825c6568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28c99b700 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c656480 0x55825c6568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28c99b700 1 -- 172.21.15.26:0/479768246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 msgr2=0x55825c6550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28c99b700 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c6550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28c99b700 1 -- 172.21.15.26:0/479768246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55825c3cbde0 con 0x55825c655630 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28d19c700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 0x7fe2780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe28c99b700 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 0x55825c655ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0ca600 tx=0x7fe27c096130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f525f9ce4bd2aeca server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe272ffd700 1 -- 172.21.15.26:0/479768246 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c099d10 con 0x55825c655630 2023-07-13T22:25:15.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe272ffd700 1 -- 172.21.15.26:0/479768246 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27c099a80 con 0x55825c655630 2023-07-13T22:25:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.084+0000 7fe272ffd700 1 -- 172.21.15.26:0/479768246 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0b0d50 con 0x55825c655630 2023-07-13T22:25:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.085+0000 7fe285ffb700 1 -- 172.21.15.26:0/872247782 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe27c014070 con 0x7fe288072f70 2023-07-13T22:25:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.085+0000 7fe2a246bb80 1 -- 172.21.15.26:0/479768246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 msgr2=0x55825c655ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.085+0000 7fe28d19c700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 0x7fe2780a3600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2800075c0 tx=0x7fe280009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.085+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 0x55825c655ef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0ca600 tx=0x7fe27c096130 comp rx=0 tx=0).stop 2023-07-13T22:25:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 -- 172.21.15.26:0/479768246 shutdown_connections 2023-07-13T22:25:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c6550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c656480 0x55825c6568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/479768246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c655630 0x55825c655ef0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 -- 172.21.15.26:0/479768246 >> 172.21.15.26:0/479768246 conn(0x55825c641da0 msgr2=0x55825c6471d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 -- 172.21.15.26:0/479768246 shutdown_connections 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 -- 172.21.15.26:0/479768246 wait complete. 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.086+0000 7fe2a246bb80 1 -- start start 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 Processor -- start 2023-07-13T22:25:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c665dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c53dac0 con 0x55825c654ce0 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.087+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c53d7c0 con 0x55825c656480 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c421920 con 0x55825c655630 2023-07-13T22:25:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe2a246bb80 10 client.0 _ll_register_callbacks cb 0x55825c53b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40460/0 (socket says 172.21.15.26:40460) 2023-07-13T22:25:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 -- 172.21.15.26:0/734805853 learned_addr learned my addr 172.21.15.26:0/734805853 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe28d19c700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c665dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe28c99b700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 -- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 msgr2=0x55825c665dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c665dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 -- 172.21.15.26:0/734805853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 msgr2=0x55825c65e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.088+0000 7fe287fff700 1 -- 172.21.15.26:0/734805853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2881a00c0 con 0x55825c655630 2023-07-13T22:25:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.089+0000 7fe287fff700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe288074270 tx=0x7fe274009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=632d889d7813c401 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.089+0000 7fe267fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.089+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27401d030 con 0x55825c655630 2023-07-13T22:25:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.089+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe274004700 con 0x55825c655630 2023-07-13T22:25:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.089+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55825c4f74f0 con 0x55825c655630 2023-07-13T22:25:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2740056f0 con 0x55825c655630 2023-07-13T22:25:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 10 client.27379 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55825c3b4c60 con 0x55825c655630 2023-07-13T22:25:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe274034020 con 0x55825c655630 2023-07-13T22:25:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe2740309f0 con 0x55825c655630 2023-07-13T22:25:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 10 client.27379 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 20 client.27379 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c67b6a0 0x55825c67dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 4 client.27379 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:15.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.090+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55825c596eb0 con 0x55825c67b6a0 2023-07-13T22:25:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.091+0000 7fe28c99b700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c67b6a0 0x55825c67dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.091+0000 7fe28c99b700 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c67b6a0 0x55825c67dab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d37765333be45a server_cookie=cc3967b468abf4c in_seq=0 out_seq=0 2023-07-13T22:25:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.091+0000 7fe267fff700 10 client.27379 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe267fff700 1 -- 172.21.15.26:0/734805853 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55825c596eb0 con 0x55825c67b6a0 2023-07-13T22:25:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe267fff700 10 client.27379 handle_command_reply: tid=0 2023-07-13T22:25:15.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 client.27379 shutdown 2023-07-13T22:25:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 msgr2=0x55825c65ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe288074270 tx=0x7fe274009570 comp rx=0 tx=0).stop 2023-07-13T22:25:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 shutdown_connections 2023-07-13T22:25:15.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c67b6a0 0x55825c67dab0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c665dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/734805853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c65ed80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 >> 172.21.15.26:0/734805853 conn(0x55825c641da0 msgr2=0x55825c6468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe267fff700 0 client.27379 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.092+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 shutdown_connections 2023-07-13T22:25:15.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.093+0000 7fe2a246bb80 1 -- 172.21.15.26:0/734805853 wait complete. 2023-07-13T22:25:15.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.093+0000 7fe2a246bb80 20 client.27379 trim_cache size 0 max 16384 2023-07-13T22:25:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 Processor -- start 2023-07-13T22:25:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 -- start start 2023-07-13T22:25:15.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c665d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c65ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c65f580 con 0x55825c654ce0 2023-07-13T22:25:15.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c65dcb0 con 0x55825c656480 2023-07-13T22:25:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c65c5a0 con 0x55825c655630 2023-07-13T22:25:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59536/0 (socket says 172.21.15.26:59536) 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe28c99b700 1 -- 172.21.15.26:0/3874780509 learned_addr learned my addr 172.21.15.26:0/3874780509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe28d19c700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c65ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.111+0000 7fe287fff700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c665d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 -- 172.21.15.26:0/3874780509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 msgr2=0x55825c665d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c665d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 -- 172.21.15.26:0/3874780509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 msgr2=0x55825c65e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 -- 172.21.15.26:0/3874780509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55825c656d70 con 0x55825c656480 2023-07-13T22:25:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe287fff700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c665d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:15.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28c99b700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe28d19c700 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c65ed80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2800178f0 tx=0x7fe280004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a99704c34d2ff25 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.112+0000 7fe2737fe700 1 -- 172.21.15.26:0/3874780509 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28000e040 con 0x55825c656480 2023-07-13T22:25:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2737fe700 1 -- 172.21.15.26:0/3874780509 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe28000f040 con 0x55825c656480 2023-07-13T22:25:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2737fe700 1 -- 172.21.15.26:0/3874780509 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280024d30 con 0x55825c656480 2023-07-13T22:25:15.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3874780509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 msgr2=0x55825c65ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c65ed80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe2800178f0 tx=0x7fe280004b40 comp rx=0 tx=0).stop 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3874780509 shutdown_connections 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c654ce0 0x55825c65e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c65ed80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3874780509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c655630 0x55825c665d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3874780509 >> 172.21.15.26:0/3874780509 conn(0x55825c641da0 msgr2=0x55825c67d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3874780509 shutdown_connections 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.113+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3874780509 wait complete. 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.114+0000 7fe2a246bb80 1 -- start start 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.114+0000 7fe2a246bb80 1 Processor -- start 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 0x55825c6625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c662af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c663080 con 0x55825c655630 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c6631f0 con 0x55825c656480 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55825c6cf5a0 con 0x55825c654ce0 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe2a246bb80 10 client.0 _ll_register_callbacks cb 0x7fe27400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 0x55825c6625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe287fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 0x55825c6625b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59538/0 (socket says 172.21.15.26:59538) 2023-07-13T22:25:15.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe287fff700 1 -- 172.21.15.26:0/3588797117 learned_addr learned my addr 172.21.15.26:0/3588797117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe28c99b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:40486/0 (socket says 172.21.15.26:40486) 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.115+0000 7fe28d19c700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c662af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 -- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 msgr2=0x55825c662af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c662af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 -- 172.21.15.26:0/3588797117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 msgr2=0x55825c6625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 0x55825c6625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 -- 172.21.15.26:0/3588797117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55825c3cbde0 con 0x55825c654ce0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe28c99b700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0ca680 tx=0x7fe27c0ccc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=362f288b4b61d5b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe267fff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0998a0 con 0x55825c654ce0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe27c099f40 con 0x55825c654ce0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe27c099a80 con 0x55825c654ce0 2023-07-13T22:25:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe27c09df70 con 0x55825c654ce0 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe2a246bb80 10 client.27385 fetch_fsmap learned FSMap version 29 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.116+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55825c3b4c60 con 0x55825c654ce0 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe27c099a80 con 0x55825c654ce0 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe27c0d4020 con 0x55825c654ce0 2023-07-13T22:25:15.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 10 client.27385 fetch_fsmap finished waiting for FSMap version 29 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 20 client.27385 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c6d0d00 0x55825c6d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 4 client.27385 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:25:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55825c559050 con 0x55825c6d0d00 2023-07-13T22:25:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.117+0000 7fe287fff700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c6d0d00 0x55825c6d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe287fff700 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c6d0d00 0x55825c6d3110 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff10f59b68af770 server_cookie=66d30a66b75526d8 in_seq=0 out_seq=0 2023-07-13T22:25:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe267fff700 10 client.27385 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe267fff700 1 -- 172.21.15.26:0/3588797117 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55825c559050 con 0x55825c6d0d00 2023-07-13T22:25:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe267fff700 10 client.27385 handle_command_reply: tid=0 2023-07-13T22:25:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 client.27385 shutdown 2023-07-13T22:25:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 msgr2=0x55825c665b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0ca680 tx=0x7fe27c0ccc20 comp rx=0 tx=0).stop 2023-07-13T22:25:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 shutdown_connections 2023-07-13T22:25:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x55825c6d0d00 0x55825c6d3110 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55825c656480 0x55825c662af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe267fff700 0 client.27385 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:15.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55825c654ce0 0x55825c665b10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 --2- 172.21.15.26:0/3588797117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55825c655630 0x55825c6625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 >> 172.21.15.26:0/3588797117 conn(0x55825c641da0 msgr2=0x55825c659340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 shutdown_connections 2023-07-13T22:25:15.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 1 -- 172.21.15.26:0/3588797117 wait complete. 2023-07-13T22:25:15.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.118+0000 7fe2a246bb80 20 client.27385 trim_cache size 0 max 16384 2023-07-13T22:25:15.116 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:15.116 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-13T22:25:15.116 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": { 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: "15cc61e9-d393-4879-b1d1-4eba9441c4f3": { 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: "path": "/", 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: "tag": "15cc61e9-d393-4879-b1d1-4eba9441c4f3", 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: "options": "recursive,force" 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:15.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:15.118 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 msgr2=0x7fe2780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 0x7fe2780a3600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2800075c0 tx=0x7fe280009040 comp rx=0 tx=0).stop 2023-07-13T22:25:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 msgr2=0x7fe288197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e970 tx=0x7fe27c00ec80 comp rx=0 tx=0).stop 2023-07-13T22:25:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 shutdown_connections 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2881079a0 0x7fe28819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe288072f70 0x7fe288197ad0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe2780a1150 0x7fe2780a3600 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 --2- 172.21.15.26:0/872247782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe288074550 0x7fe288198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 >> 172.21.15.26:0/872247782 conn(0x7fe2880fb0e0 msgr2=0x7fe2880fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 shutdown_connections 2023-07-13T22:25:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:15.119+0000 7fe28ebff700 1 -- 172.21.15.26:0/872247782 wait complete. 2023-07-13T22:25:15.122 INFO:tasks.cephfs.filesystem:scrub status for tag:15cc61e9-d393-4879-b1d1-4eba9441c4f3 - {'path': '/', 'tag': '15cc61e9-d393-4879-b1d1-4eba9441c4f3', 'options': 'recursive,force'} 2023-07-13T22:25:15.936 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:15 smithi117 ceph-mon[129074]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 104 op/s 2023-07-13T22:25:15.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:15 smithi026 ceph-mon[95647]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 104 op/s 2023-07-13T22:25:15.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:15 smithi121 ceph-mon[129187]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 318 MiB/s wr, 104 op/s 2023-07-13T22:25:16.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:16 smithi026 ceph-mon[95647]: scrub queued for path: / 2023-07-13T22:25:16.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:16 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:16.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:16 smithi026 ceph-mon[95647]: scrub summary: active paths [/] 2023-07-13T22:25:16.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:16 smithi026 ceph-mon[95647]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:16.945 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:16 smithi026 ceph-mon[95647]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:25:16.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:16 smithi121 ceph-mon[129187]: scrub queued for path: / 2023-07-13T22:25:16.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:16 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:16.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:16 smithi121 ceph-mon[129187]: scrub summary: active paths [/] 2023-07-13T22:25:16.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:16 smithi121 ceph-mon[129187]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:16.987 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:16 smithi121 ceph-mon[129187]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:25:17.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:16 smithi117 ceph-mon[129074]: scrub queued for path: / 2023-07-13T22:25:17.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:16 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:17.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:16 smithi117 ceph-mon[129074]: scrub summary: active paths [/] 2023-07-13T22:25:17.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:16 smithi117 ceph-mon[129074]: scrub summary: idle+waiting paths [/] 2023-07-13T22:25:17.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:16 smithi117 ceph-mon[129074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-07-13T22:25:17.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:17 smithi121 ceph-mon[129187]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 89 op/s 2023-07-13T22:25:17.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:17 smithi121 ceph-mon[129187]: scrub summary: idle 2023-07-13T22:25:18.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:17 smithi117 ceph-mon[129074]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 89 op/s 2023-07-13T22:25:18.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:17 smithi117 ceph-mon[129074]: scrub summary: idle 2023-07-13T22:25:18.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:17 smithi026 ceph-mon[95647]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 271 MiB/s wr, 89 op/s 2023-07-13T22:25:18.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:17 smithi026 ceph-mon[95647]: scrub summary: idle 2023-07-13T22:25:19.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:19 smithi121 ceph-mon[129187]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 335 MiB/s wr, 120 op/s 2023-07-13T22:25:20.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:19 smithi117 ceph-mon[129074]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 335 MiB/s wr, 120 op/s 2023-07-13T22:25:20.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:19 smithi026 ceph-mon[95647]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 335 MiB/s wr, 120 op/s 2023-07-13T22:25:21.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:21 smithi121 ceph-mon[129187]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 311 MiB/s wr, 116 op/s 2023-07-13T22:25:22.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:21 smithi117 ceph-mon[129074]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 311 MiB/s wr, 116 op/s 2023-07-13T22:25:22.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:21 smithi026 ceph-mon[95647]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 311 MiB/s wr, 116 op/s 2023-07-13T22:25:23.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:23 smithi121 ceph-mon[129187]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 303 MiB/s wr, 124 op/s 2023-07-13T22:25:24.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:23 smithi117 ceph-mon[129074]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 303 MiB/s wr, 124 op/s 2023-07-13T22:25:24.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:23 smithi026 ceph-mon[95647]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 303 MiB/s wr, 124 op/s 2023-07-13T22:25:25.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:25 smithi121 ceph-mon[129187]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 363 MiB/s wr, 163 op/s 2023-07-13T22:25:26.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:25 smithi117 ceph-mon[129074]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 363 MiB/s wr, 163 op/s 2023-07-13T22:25:26.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:25 smithi026 ceph-mon[95647]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 363 MiB/s wr, 163 op/s 2023-07-13T22:25:28.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:27 smithi117 ceph-mon[129074]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 325 MiB/s wr, 161 op/s 2023-07-13T22:25:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:27 smithi026 ceph-mon[95647]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 325 MiB/s wr, 161 op/s 2023-07-13T22:25:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:27 smithi121 ceph-mon[129187]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 325 MiB/s wr, 161 op/s 2023-07-13T22:25:30.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:29 smithi117 ceph-mon[129074]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 397 MiB/s wr, 242 op/s 2023-07-13T22:25:30.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:29 smithi026 ceph-mon[95647]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 397 MiB/s wr, 242 op/s 2023-07-13T22:25:30.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:29 smithi121 ceph-mon[129187]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 397 MiB/s wr, 242 op/s 2023-07-13T22:25:32.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:31 smithi117 ceph-mon[129074]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 349 MiB/s wr, 256 op/s 2023-07-13T22:25:32.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:31 smithi026 ceph-mon[95647]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 349 MiB/s wr, 256 op/s 2023-07-13T22:25:32.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:31 smithi121 ceph-mon[129187]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 349 MiB/s wr, 256 op/s 2023-07-13T22:25:34.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:33 smithi117 ceph-mon[129074]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 302 MiB/s wr, 254 op/s 2023-07-13T22:25:34.201 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:33 smithi026 ceph-mon[95647]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 302 MiB/s wr, 254 op/s 2023-07-13T22:25:34.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:33 smithi121 ceph-mon[129187]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 302 MiB/s wr, 254 op/s 2023-07-13T22:25:36.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:35 smithi117 ceph-mon[129074]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 274 MiB/s wr, 248 op/s 2023-07-13T22:25:36.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:35 smithi026 ceph-mon[95647]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 274 MiB/s wr, 248 op/s 2023-07-13T22:25:36.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:35 smithi121 ceph-mon[129187]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 274 MiB/s wr, 248 op/s 2023-07-13T22:25:38.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:37 smithi117 ceph-mon[129074]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 196 MiB/s wr, 216 op/s 2023-07-13T22:25:38.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:37 smithi026 ceph-mon[95647]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 196 MiB/s wr, 216 op/s 2023-07-13T22:25:38.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:37 smithi121 ceph-mon[129187]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 196 MiB/s wr, 216 op/s 2023-07-13T22:25:40.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:39 smithi117 ceph-mon[129074]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 MiB/s wr, 360 op/s 2023-07-13T22:25:40.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:39 smithi026 ceph-mon[95647]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 MiB/s wr, 360 op/s 2023-07-13T22:25:40.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:39 smithi121 ceph-mon[129187]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 232 MiB/s wr, 360 op/s 2023-07-13T22:25:42.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:41 smithi117 ceph-mon[129074]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 194 MiB/s wr, 463 op/s 2023-07-13T22:25:42.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:41 smithi026 ceph-mon[95647]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 194 MiB/s wr, 463 op/s 2023-07-13T22:25:42.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:41 smithi121 ceph-mon[129187]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 194 MiB/s wr, 463 op/s 2023-07-13T22:25:44.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:43 smithi117 ceph-mon[129074]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 182 MiB/s wr, 591 op/s 2023-07-13T22:25:44.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:43 smithi026 ceph-mon[95647]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 182 MiB/s wr, 591 op/s 2023-07-13T22:25:44.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:43 smithi121 ceph-mon[129187]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 182 MiB/s wr, 591 op/s 2023-07-13T22:25:45.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-13T22:25:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/3049331257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a080ffae0 msgr2=0x7f1a080ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/3049331257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a080ffae0 0x7f1a080ffef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009990 tx=0x7f19f8009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/3049331257 shutdown_connections 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/3049331257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a080ffae0 0x7f1a080ffef0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/3049331257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a08101660 0x7f1a08107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/3049331257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 0x7f1a08101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/3049331257 >> 172.21.15.26:0/3049331257 conn(0x7f1a080fb0e0 msgr2=0x7f1a080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/3049331257 shutdown_connections 2023-07-13T22:25:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.359+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/3049331257 wait complete. 2023-07-13T22:25:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.360+0000 7f1a0e1ff700 1 Processor -- start 2023-07-13T22:25:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.360+0000 7f1a0e1ff700 1 -- start start 2023-07-13T22:25:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a080ffae0 0x7f1a08197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 0x7f1a08198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0819f640 con 0x7f1a08101660 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0819f780 con 0x7f1a08100cd0 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a0e1ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0819f910 con 0x7f1a080ffae0 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33092/0 (socket says 172.21.15.26:33092) 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 -- 172.21.15.26:0/792029440 learned_addr learned my addr 172.21.15.26:0/792029440 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f19fffff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 0x7f1a08198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a080ffae0 0x7f1a08197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 -- 172.21.15.26:0/792029440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a080ffae0 msgr2=0x7f1a08197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a080ffae0 0x7f1a08197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 -- 172.21.15.26:0/792029440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 msgr2=0x7f1a08198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 0x7f1a08198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.361+0000 7f1a07fff700 1 -- 172.21.15.26:0/792029440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80095f0 con 0x7f1a08101660 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a07fff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a00007ee0 tx=0x7f1a0000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a057fa700 1 -- 172.21.15.26:0/792029440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000dd20 con 0x7f1a08101660 2023-07-13T22:25:45.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a057fa700 1 -- 172.21.15.26:0/792029440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0000f040 con 0x7f1a08101660 2023-07-13T22:25:45.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0819fbf0 con 0x7f1a08101660 2023-07-13T22:25:45.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a057fa700 1 -- 172.21.15.26:0/792029440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a000048a0 con 0x7f1a08101660 2023-07-13T22:25:45.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.362+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081a0100 con 0x7f1a08101660 2023-07-13T22:25:45.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.365+0000 7f1a057fa700 1 -- 172.21.15.26:0/792029440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1a00003a00 con 0x7f1a08101660 2023-07-13T22:25:45.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f1a21ba8b80 1 Processor -- start 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f1a21ba8b80 1 -- start start 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f1a057fa700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 0x7f19f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f1a057fa700 1 -- 172.21.15.26:0/792029440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1a000cc170 con 0x7f1a08101660 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x5607659288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.367+0000 7f19fffff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 0x7f19f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765929c70 0x56076592a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076582dcb0 con 0x5607659284d0 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765880a40 con 0x560765929c70 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765879ed0 con 0x560765928e20 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x5607659288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44278/0 (socket says 172.21.15.26:44278) 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 -- 172.21.15.26:0/4006215832 learned_addr learned my addr 172.21.15.26:0/4006215832 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a07fff700 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765929c70 0x56076592a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 0x7f19f40a3600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f19f0013650 tx=0x7f19f0013a10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 -- 172.21.15.26:0/4006215832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765929c70 msgr2=0x56076592a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765929c70 0x56076592a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 -- 172.21.15.26:0/4006215832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 msgr2=0x5607659288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x5607659288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f19fffff700 1 -- 172.21.15.26:0/4006215832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607656a7de0 con 0x560765928e20 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.368+0000 7f1a077fe700 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x5607659288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.369+0000 7f19fffff700 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c10 tx=0x7f19f0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78d284bba6c85686 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.369+0000 7f19fdffb700 1 -- 172.21.15.26:0/4006215832 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0010430 con 0x560765928e20 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.369+0000 7f19fdffb700 1 -- 172.21.15.26:0/4006215832 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f0010a50 con 0x560765928e20 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.369+0000 7f19fdffb700 1 -- 172.21.15.26:0/4006215832 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f000f680 con 0x560765928e20 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/4006215832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 msgr2=0x5607659296e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c10 tx=0x7f19f0020000 comp rx=0 tx=0).stop 2023-07-13T22:25:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/4006215832 shutdown_connections 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x5607659288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765929c70 0x56076592a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/4006215832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x5607659296e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/4006215832 >> 172.21.15.26:0/4006215832 conn(0x56076591dda0 msgr2=0x5607659231d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/4006215832 shutdown_connections 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/4006215832 wait complete. 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.370+0000 7f1a21ba8b80 1 -- start start 2023-07-13T22:25:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 Processor -- start 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x5607659320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765939670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765819ac0 con 0x560765928e20 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607658197c0 con 0x5607659284d0 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607656fd920 con 0x560765929c70 2023-07-13T22:25:45.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a21ba8b80 10 client.0 _ll_register_callbacks cb 0x560765817760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f19fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f19fffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33112/0 (socket says 172.21.15.26:33112) 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f19fffff700 1 -- 172.21.15.26:0/2645626522 learned_addr learned my addr 172.21.15.26:0/2645626522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a077fe700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x5607659320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.371+0000 7f1a07fff700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765939670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 -- 172.21.15.26:0/2645626522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 msgr2=0x560765939670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765939670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 -- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 msgr2=0x5607659320f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x5607659320f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 -- 172.21.15.26:0/2645626522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a08108850 con 0x560765928e20 2023-07-13T22:25:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19fffff700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010f80 tx=0x7f19f0005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d7339bd9cc515 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0010630 con 0x560765928e20 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607657d34f0 con 0x560765928e20 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.372+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19f0010790 con 0x560765928e20 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f0039030 con 0x560765928e20 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 10 client.16740 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f19f0020540 con 0x560765928e20 2023-07-13T22:25:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607657d34f0 con 0x560765928e20 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f19f000f4b0 con 0x560765928e20 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 10 client.16740 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 20 client.16740 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56076593de00 0x560765940210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 4 client.16740 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.373+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560765872eb0 con 0x56076593de00 2023-07-13T22:25:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.374+0000 7f1a077fe700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56076593de00 0x560765940210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.374+0000 7f1a077fe700 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56076593de00 0x560765940210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7609758cf5e4e314 server_cookie=d2a3335c667896ca in_seq=0 out_seq=0 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.374+0000 7f19e6ffd700 10 client.16740 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f19e6ffd700 1 -- 172.21.15.26:0/2645626522 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560765872eb0 con 0x56076593de00 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f19e6ffd700 10 client.16740 handle_command_reply: tid=0 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 client.16740 shutdown 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 msgr2=0x560765932630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010f80 tx=0x7f19f0005700 comp rx=0 tx=0).stop 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 shutdown_connections 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x56076593de00 0x560765940210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765939670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x5607659320f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f19e6ffd700 0 client.16740 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/2645626522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765928e20 0x560765932630 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 >> 172.21.15.26:0/2645626522 conn(0x56076591dda0 msgr2=0x5607659228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 shutdown_connections 2023-07-13T22:25:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.375+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/2645626522 wait complete. 2023-07-13T22:25:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.376+0000 7f1a21ba8b80 20 client.16740 trim_cache size 0 max 16384 2023-07-13T22:25:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 Processor -- start 2023-07-13T22:25:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 -- start start 2023-07-13T22:25:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x560765944b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x560765939630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607659240d0 con 0x560765929c70 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765924a00 con 0x5607659284d0 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.393+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765932d70 con 0x560765928e20 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33126/0 (socket says 172.21.15.26:33126) 2023-07-13T22:25:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f19fffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x560765939630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 -- 172.21.15.26:0/1614915915 learned_addr learned my addr 172.21.15.26:0/1614915915 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a077fe700 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x560765944b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 -- 172.21.15.26:0/1614915915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 msgr2=0x560765939630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x560765939630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 -- 172.21.15.26:0/1614915915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 msgr2=0x560765944b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x560765944b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 -- 172.21.15.26:0/1614915915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560765690c60 con 0x560765929c70 2023-07-13T22:25:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a077fe700 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x560765944b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f1a07fff700 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f80 tx=0x7f1a000b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f98174695e18897 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.394+0000 7f19fe7fc700 1 -- 172.21.15.26:0/1614915915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00099530 con 0x560765929c70 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f19fe7fc700 1 -- 172.21.15.26:0/1614915915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a0009d560 con 0x560765929c70 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f19fe7fc700 1 -- 172.21.15.26:0/1614915915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00097960 con 0x560765929c70 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/1614915915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 msgr2=0x5607659228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f80 tx=0x7f1a000b9040 comp rx=0 tx=0).stop 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/1614915915 shutdown_connections 2023-07-13T22:25:45.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560765929c70 0x5607659228b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607659284d0 0x560765944b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/1614915915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765928e20 0x560765939630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/1614915915 >> 172.21.15.26:0/1614915915 conn(0x56076591dda0 msgr2=0x560765930e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/1614915915 shutdown_connections 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.395+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/1614915915 wait complete. 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 -- start start 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 Processor -- start 2023-07-13T22:25:45.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x560765939360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765936340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607659368c0 con 0x5607659284d0 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765936a00 con 0x560765928e20 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560765991c70 con 0x560765929c70 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.396+0000 7f1a21ba8b80 10 client.0 _ll_register_callbacks cb 0x7f19f0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:45.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f1a077fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x560765939360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52550/0 (socket says 172.21.15.26:52550) 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 -- 172.21.15.26:0/3780110785 learned_addr learned my addr 172.21.15.26:0/3780110785 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f1a07fff700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765936340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 -- 172.21.15.26:0/3780110785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 msgr2=0x560765936340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765936340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 -- 172.21.15.26:0/3780110785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 msgr2=0x560765939360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:45.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x560765939360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 -- 172.21.15.26:0/3780110785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f8005400 con 0x560765928e20 2023-07-13T22:25:45.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19fffff700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009b30 tx=0x7f19f0009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b35733e9793935 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.397+0000 7f19e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0039040 con 0x560765928e20 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19f0010a50 con 0x560765928e20 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f19f003a070 con 0x560765928e20 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19f00122c0 con 0x560765928e20 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 10 client.27413 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1a081a0100 con 0x560765928e20 2023-07-13T22:25:45.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f0010a50 con 0x560765928e20 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f19f0029860 con 0x560765928e20 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 10 client.27413 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 10 client.27413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 20 client.27413 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560765993330 0x560765995710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 4 client.27413 mds_command: new command op to 24575 tid=0[{"prefix": "scrub status"}] 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.398+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560765835050 con 0x560765993330 2023-07-13T22:25:45.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.399+0000 7f1a077fe700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560765993330 0x560765995710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.399+0000 7f1a077fe700 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560765993330 0x560765995710 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7439903d1064cb server_cookie=dcb737ec8e97e466 in_seq=0 out_seq=0 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.399+0000 7f19e6ffd700 10 client.27413 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f19e6ffd700 1 -- 172.21.15.26:0/3780110785 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560765835050 con 0x560765993330 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f19e6ffd700 10 client.27413 handle_command_reply: tid=0 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 client.27413 shutdown 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 msgr2=0x560765935e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009b30 tx=0x7f19f0009c80 comp rx=0 tx=0).stop 2023-07-13T22:25:45.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 shutdown_connections 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5607659284d0 0x560765939360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x560765993330 0x560765995710 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560765929c70 0x560765936340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 --2- 172.21.15.26:0/3780110785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560765928e20 0x560765935e00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f19e6ffd700 0 client.27413 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 >> 172.21.15.26:0/3780110785 conn(0x56076591dda0 msgr2=0x5607659228b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 shutdown_connections 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 1 -- 172.21.15.26:0/3780110785 wait complete. 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.400+0000 7f1a21ba8b80 20 client.27413 trim_cache size 0 max 16384 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "no active scrubs running", 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stdout: "scrubs": {} 2023-07-13T22:25:45.390 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 msgr2=0x7f19f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 0x7f19f40a3600 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f19f0013650 tx=0x7f19f0013a10 comp rx=0 tx=0).stop 2023-07-13T22:25:45.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 msgr2=0x7f1a0819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a00007ee0 tx=0x7f1a0000d3b0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 shutdown_connections 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f19f40a1150 0x7f19f40a3600 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a08101660 0x7f1a0819f070 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a080ffae0 0x7f1a08197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 --2- 172.21.15.26:0/792029440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a08100cd0 0x7f1a08198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 >> 172.21.15.26:0/792029440 conn(0x7f1a080fb0e0 msgr2=0x7f1a080fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 shutdown_connections 2023-07-13T22:25:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.401+0000 7f1a0e1ff700 1 -- 172.21.15.26:0/792029440 wait complete. 2023-07-13T22:25:45.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-13T22:25:45.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:25:45.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 -- 172.21.15.26:0/3814928572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8101680 msgr2=0x7f2cc8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/3814928572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8101680 0x7f2cc8107f40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009f90 tx=0x7f2cc000b330 comp rx=0 tx=0).stop 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 -- 172.21.15.26:0/3814928572 shutdown_connections 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/3814928572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8101680 0x7f2cc8107f40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/3814928572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/3814928572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 -- 172.21.15.26:0/3814928572 >> 172.21.15.26:0/3814928572 conn(0x7f2cc80fb0e0 msgr2=0x7f2cc80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 -- 172.21.15.26:0/3814928572 shutdown_connections 2023-07-13T22:25:45.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.624+0000 7f2cce9ff700 1 -- 172.21.15.26:0/3814928572 wait complete. 2023-07-13T22:25:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.625+0000 7f2cce9ff700 1 Processor -- start 2023-07-13T22:25:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.625+0000 7f2cce9ff700 1 -- start start 2023-07-13T22:25:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 0x7f2cc8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc80735e0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8073750 con 0x7f2cc8100cf0 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cce9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc819f970 con 0x7f2cc8101680 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33158/0 (socket says 172.21.15.26:33158) 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 -- 172.21.15.26:0/1663605607 learned_addr learned my addr 172.21.15.26:0/1663605607 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cccf9c700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 0x7f2cc8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc77fe700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 -- 172.21.15.26:0/1663605607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 msgr2=0x7f2cc8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 0x7f2cc8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 -- 172.21.15.26:0/1663605607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 msgr2=0x7f2cc8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc7fff700 1 -- 172.21.15.26:0/1663605607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0009c40 con 0x7f2cc80ffb00 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cc77fe700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.626+0000 7f2cccf9c700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 0x7f2cc8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cc7fff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e970 tx=0x7f2cb800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8009790 con 0x7f2cc80ffb00 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cb800eec0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc819fc50 con 0x7f2cc80ffb00 2023-07-13T22:25:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8003cf0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.627+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81a01b0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.628+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8106fd0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.630+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2cb800ccd0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.633+0000 7f2cc57fa700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 0x7f2cb00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:45.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.633+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2cb8014070 con 0x7f2cc80ffb00 2023-07-13T22:25:45.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.633+0000 7f2cc77fe700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 0x7f2cb00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:45.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.634+0000 7f2cc77fe700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 0x7f2cb00a36d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc005fd0 tx=0x7f2cbc009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:45.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.635+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2cb80963d0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.890+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cc802ccd0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.891+0000 7f2cc57fa700 1 -- 172.21.15.26:0/1663605607 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f2cc802ccd0 con 0x7f2cc80ffb00 2023-07-13T22:25:45.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:25:45.874 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:25:15.677742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:25:45.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.893+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 msgr2=0x7f2cb00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.893+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 0x7f2cb00a36d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc005fd0 tx=0x7f2cbc009310 comp rx=0 tx=0).stop 2023-07-13T22:25:45.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.893+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 msgr2=0x7f2cc80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.893+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800e970 tx=0x7f2cb800ec80 comp rx=0 tx=0).stop 2023-07-13T22:25:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 shutdown_connections 2023-07-13T22:25:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2cb00a1220 0x7f2cb00a36d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cc80ffb00 0x7f2cc80742c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cc8101680 0x7f2cc8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 --2- 172.21.15.26:0/1663605607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cc8100cf0 0x7f2cc8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 >> 172.21.15.26:0/1663605607 conn(0x7f2cc80fb0e0 msgr2=0x7f2cc8104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 shutdown_connections 2023-07-13T22:25:45.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:45.894+0000 7f2cce9ff700 1 -- 172.21.15.26:0/1663605607 wait complete. 2023-07-13T22:25:45.879 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2023-07-13T22:25:45.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-07-13T22:25:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 -- 172.21.15.26:0/2997616541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4102f20 msgr2=0x7efcd4105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 --2- 172.21.15.26:0/2997616541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4102f20 0x7efcd4105300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efcc40098f0 tx=0x7efcc4009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 -- 172.21.15.26:0/2997616541 shutdown_connections 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 --2- 172.21.15.26:0/2997616541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4100600 0x7efcd41029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 --2- 172.21.15.26:0/2997616541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd4107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 --2- 172.21.15.26:0/2997616541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4102f20 0x7efcd4105300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 -- 172.21.15.26:0/2997616541 >> 172.21.15.26:0/2997616541 conn(0x7efcd40fa000 msgr2=0x7efcd40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 -- 172.21.15.26:0/2997616541 shutdown_connections 2023-07-13T22:25:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.120+0000 7efcdbdff700 1 -- 172.21.15.26:0/2997616541 wait complete. 2023-07-13T22:25:46.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.121+0000 7efcdbdff700 1 Processor -- start 2023-07-13T22:25:46.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.121+0000 7efcdbdff700 1 -- start start 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 0x7efcd4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419f5d0 con 0x7efcd4102f20 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419f710 con 0x7efcd4100600 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcdbdff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd419f8a0 con 0x7efcd4105840 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcda39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 0x7efcd4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33170/0 (socket says 172.21.15.26:33170) 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcda39c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44322/0 (socket says 172.21.15.26:44322) 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd939a700 1 -- 172.21.15.26:0/3701000838 learned_addr learned my addr 172.21.15.26:0/3701000838 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3701000838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 msgr2=0x7efcd419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 msgr2=0x7efcd4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.122+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3701000838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00095f0 con 0x7efcd4100600 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efcd939a700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 0x7efcd4197a50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcd000eb90 tx=0x7efcd000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efccaffd700 1 -- 172.21.15.26:0/3701000838 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0009970 con 0x7efcd4100600 2023-07-13T22:25:46.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efccaffd700 1 -- 172.21.15.26:0/3701000838 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd00044e0 con 0x7efcd4100600 2023-07-13T22:25:46.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc4009520 con 0x7efcd4100600 2023-07-13T22:25:46.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efccaffd700 1 -- 172.21.15.26:0/3701000838 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0003e40 con 0x7efcd4100600 2023-07-13T22:25:46.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.123+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd419ff50 con 0x7efcd4100600 2023-07-13T22:25:46.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.128+0000 7efccaffd700 1 -- 172.21.15.26:0/3701000838 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efcd0003fe0 con 0x7efcd4100600 2023-07-13T22:25:46.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 Processor -- start 2023-07-13T22:25:46.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 -- start start 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be07518e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be0656cb0 con 0x560be0752c70 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be06a9a40 con 0x560be0751e20 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.129+0000 7efcef71fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be06a2ed0 con 0x560be07514d0 2023-07-13T22:25:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd9b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33172/0 (socket says 172.21.15.26:33172) 2023-07-13T22:25:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 -- 172.21.15.26:0/2704385930 learned_addr learned my addr 172.21.15.26:0/2704385930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcda39c700 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be07518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efccaffd700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 0x7efcc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 -- 172.21.15.26:0/2704385930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 msgr2=0x560be07518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be07518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 -- 172.21.15.26:0/2704385930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 msgr2=0x560be07526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd939a700 1 -- 172.21.15.26:0/2704385930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be04d0de0 con 0x560be0752c70 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efccaffd700 1 -- 172.21.15.26:0/3701000838 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcd0014070 con 0x7efcd4100600 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.130+0000 7efcda39c700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 0x7efcc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.131+0000 7efcd939a700 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005b70 tx=0x7efcc4005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8443364fa03ab4b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.131+0000 7efcbffff700 1 -- 172.21.15.26:0/2704385930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc401c050 con 0x560be0752c70 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.131+0000 7efcbffff700 1 -- 172.21.15.26:0/2704385930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc401d040 con 0x560be0752c70 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.131+0000 7efcbffff700 1 -- 172.21.15.26:0/2704385930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc40054a0 con 0x560be0752c70 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- 172.21.15.26:0/2704385930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 msgr2=0x560be07530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcda39c700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 0x7efcc00a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efccc0075c0 tx=0x7efccc009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005b70 tx=0x7efcc4005ba0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- 172.21.15.26:0/2704385930 shutdown_connections 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be07530e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be07518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2704385930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- 172.21.15.26:0/2704385930 >> 172.21.15.26:0/2704385930 conn(0x560be0746da0 msgr2=0x560be074c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- 172.21.15.26:0/2704385930 shutdown_connections 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- 172.21.15.26:0/2704385930 wait complete. 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.132+0000 7efcef71fb80 1 -- start start 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 Processor -- start 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 0x560be075b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0752c70 0x560be0762630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be0642ac0 con 0x560be0751e20 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be06427c0 con 0x560be07514d0 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be0526920 con 0x560be0752c70 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcef71fb80 10 client.0 _ll_register_callbacks cb 0x560be0640760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcd939a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33176/0 (socket says 172.21.15.26:33176) 2023-07-13T22:25:46.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcd939a700 1 -- 172.21.15.26:0/3100392244 learned_addr learned my addr 172.21.15.26:0/3100392244 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcda39c700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0752c70 0x560be0762630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.133+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 0x560be075b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 -- 172.21.15.26:0/3100392244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0752c70 msgr2=0x560be0762630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0752c70 0x560be0762630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 -- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 msgr2=0x560be075b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 0x560be075b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 -- 172.21.15.26:0/3100392244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd419ff50 con 0x560be0751e20 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 0x560be075b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcd939a700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efcc400c010 tx=0x7efcc4005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ace7ebb2abe9e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc400b770 con 0x560be0751e20 2023-07-13T22:25:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560be05fc4f0 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.134+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcc400b8d0 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcc400ba90 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 10 client.16764 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7efcd41087f0 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560be05fc4f0 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efcc4015020 con 0x560be0751e20 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 10 client.16764 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 20 client.16764 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07668f0 0x560be0768cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 4 client.16764 mds_command: new command op to 24467 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.135+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560be069beb0 con 0x560be07668f0 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.136+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07668f0 0x560be0768cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.136+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07668f0 0x560be0768cd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98b871328740fef server_cookie=6d276dfc3aaf770d in_seq=0 out_seq=0 2023-07-13T22:25:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.136+0000 7efcbd7fa700 10 client.16764 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcbd7fa700 1 -- 172.21.15.26:0/3100392244 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560be069beb0 con 0x560be07668f0 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcbd7fa700 10 client.16764 handle_command_reply: tid=0 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 client.16764 shutdown 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 msgr2=0x560be075b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efcc400c010 tx=0x7efcc4005490 comp rx=0 tx=0).stop 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 shutdown_connections 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07668f0 0x560be0768cd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0752c70 0x560be0762630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be07514d0 0x560be075b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcbd7fa700 0 client.16764 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3100392244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0751e20 0x560be075b5f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 >> 172.21.15.26:0/3100392244 conn(0x560be0746da0 msgr2=0x560be074b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 shutdown_connections 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 1 -- 172.21.15.26:0/3100392244 wait complete. 2023-07-13T22:25:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.137+0000 7efcef71fb80 20 client.16764 trim_cache size 0 max 16384 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 Processor -- start 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 -- start start 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0751e20 0x560be07625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0752c70 0x560be075b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be075ac20 con 0x560be07514d0 2023-07-13T22:25:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be074ced0 con 0x560be0752c70 2023-07-13T22:25:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcef71fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be07666e0 con 0x560be0751e20 2023-07-13T22:25:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcd9b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcd9b9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33192/0 (socket says 172.21.15.26:33192) 2023-07-13T22:25:46.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3084174751 learned_addr learned my addr 172.21.15.26:0/3084174751 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.151+0000 7efcda39c700 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0752c70 0x560be075b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd939a700 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0751e20 0x560be07625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3084174751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0751e20 msgr2=0x560be07625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0751e20 0x560be07625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3084174751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0752c70 msgr2=0x560be075b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0752c70 0x560be075b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 -- 172.21.15.26:0/3084174751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560be04b9c60 con 0x560be07514d0 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efcd0096ca0 tx=0x7efcd00ca740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c07f8302e7fa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcc8bff700 1 -- 172.21.15.26:0/3084174751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00d1040 con 0x560be07514d0 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcc8bff700 1 -- 172.21.15.26:0/3084174751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd009a000 con 0x560be07514d0 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.152+0000 7efcc8bff700 1 -- 172.21.15.26:0/3084174751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd009e1b0 con 0x560be07514d0 2023-07-13T22:25:46.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- 172.21.15.26:0/3084174751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 msgr2=0x560be0762f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efcd0096ca0 tx=0x7efcd00ca740 comp rx=0 tx=0).stop 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- 172.21.15.26:0/3084174751 shutdown_connections 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be07514d0 0x560be0762f20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0752c70 0x560be075b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 --2- 172.21.15.26:0/3084174751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be0751e20 0x560be07625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- 172.21.15.26:0/3084174751 >> 172.21.15.26:0/3084174751 conn(0x560be0746da0 msgr2=0x560be076d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- 172.21.15.26:0/3084174751 shutdown_connections 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- 172.21.15.26:0/3084174751 wait complete. 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.153+0000 7efcef71fb80 1 -- start start 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 Processor -- start 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be0761e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be075e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be075ee40 con 0x560be0752c70 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be075efb0 con 0x560be0751e20 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x560be075f170 con 0x560be07514d0 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcef71fb80 10 client.0 _ll_register_callbacks cb 0x7efcd00d04d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcda39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be075e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcda39c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be075e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33202/0 (socket says 172.21.15.26:33202) 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcda39c700 1 -- 172.21.15.26:0/2508959724 learned_addr learned my addr 172.21.15.26:0/2508959724 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be0761e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.154+0000 7efcd939a700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 -- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 msgr2=0x560be07623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 -- 172.21.15.26:0/2508959724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 msgr2=0x560be075e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be075e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 -- 172.21.15.26:0/2508959724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc4009520 con 0x560be07514d0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcd9b9b700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be0761e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcd0096ca0 tx=0x7efcd0005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e37d2d04d2f176 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0099d70 con 0x560be07514d0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7efcd0094e40 con 0x560be07514d0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcc401d040 con 0x560be07514d0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd0099b90 con 0x560be07514d0 2023-07-13T22:25:46.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcef71fb80 10 client.27427 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.155+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x560be04f7250 con 0x560be07514d0 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc401d040 con 0x560be07514d0 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efcd009a5b0 con 0x560be07514d0 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 10 client.27427 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 10 client.27427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 20 client.27427 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07bb820 0x560be07bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 4 client.27427 mds_command: new command op to 24467 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 --> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560be065e050 con 0x560be07bb820 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.156+0000 7efcd939a700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07bb820 0x560be07bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcd939a700 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07bb820 0x560be07bdc00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc1ef3533778decb server_cookie=a7baeefa2455bb81 in_seq=0 out_seq=0 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcbd7fa700 10 client.27427 ms_handle_connect on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcbd7fa700 1 -- 172.21.15.26:0/2508959724 <== mds.4 v2:172.21.15.117:6832/2196716047 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560be065e050 con 0x560be07bb820 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcbd7fa700 10 client.27427 handle_command_reply: tid=0 2023-07-13T22:25:46.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 client.27427 shutdown 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 msgr2=0x560be0761e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be0761e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcd0096ca0 tx=0x7efcd0005a90 comp rx=0 tx=0).stop 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 shutdown_connections 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:6832/2196716047,v1:172.21.15.117:6833/2196716047] conn(0x560be07bb820 0x560be07bdc00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x560be0752c70 0x560be075e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be07514d0 0x560be0761e60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 --2- 172.21.15.26:0/2508959724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560be0751e20 0x560be07623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcbd7fa700 0 client.27427 ms_handle_reset on v2:172.21.15.117:6832/2196716047 2023-07-13T22:25:46.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 >> 172.21.15.26:0/2508959724 conn(0x560be0746da0 msgr2=0x560be0762f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 shutdown_connections 2023-07-13T22:25:46.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.157+0000 7efcef71fb80 1 -- 172.21.15.26:0/2508959724 wait complete. 2023-07-13T22:25:46.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcef71fb80 20 client.27427 trim_cache size 0 max 16384 2023-07-13T22:25:46.152 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:46.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 msgr2=0x7efcc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 0x7efcc00a3680 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efccc0075c0 tx=0x7efccc009040 comp rx=0 tx=0).stop 2023-07-13T22:25:46.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 msgr2=0x7efcd4197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 0x7efcd4197a50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcd000eb90 tx=0x7efcd000eea0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 shutdown_connections 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4105840 0x7efcd419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efcd4100600 0x7efcd4197a50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7efcc00a11d0 0x7efcc00a3680 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 --2- 172.21.15.26:0/3701000838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efcd4102f20 0x7efcd4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 >> 172.21.15.26:0/3701000838 conn(0x7efcd40fa000 msgr2=0x7efcd4101260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 shutdown_connections 2023-07-13T22:25:46.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.158+0000 7efcdbdff700 1 -- 172.21.15.26:0/3701000838 wait complete. 2023-07-13T22:25:46.156 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:45 smithi026 ceph-mon[95647]: pgmap v671: 97 pgs: 97 active+clean; 9.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 228 MiB/s wr, 985 op/s 2023-07-13T22:25:46.158 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-07-13T22:25:46.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:45 smithi117 ceph-mon[129074]: pgmap v671: 97 pgs: 97 active+clean; 9.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 228 MiB/s wr, 985 op/s 2023-07-13T22:25:46.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:45 smithi121 ceph-mon[129187]: pgmap v671: 97 pgs: 97 active+clean; 9.0 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 228 MiB/s wr, 985 op/s 2023-07-13T22:25:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 -- 172.21.15.26:0/904595644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea40ffaf0 msgr2=0x7f1ea40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 --2- 172.21.15.26:0/904595644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea40fff00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009990 tx=0x7f1ea0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 -- 172.21.15.26:0/904595644 shutdown_connections 2023-07-13T22:25:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 --2- 172.21.15.26:0/904595644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea4101670 0x7f1ea4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 --2- 172.21.15.26:0/904595644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea40fff00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 --2- 172.21.15.26:0/904595644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 -- 172.21.15.26:0/904595644 >> 172.21.15.26:0/904595644 conn(0x7f1ea40fb130 msgr2=0x7f1ea40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 -- 172.21.15.26:0/904595644 shutdown_connections 2023-07-13T22:25:46.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.383+0000 7f1eabbff700 1 -- 172.21.15.26:0/904595644 wait complete. 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 Processor -- start 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 -- start start 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea4101670 0x7f1ea419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419f5f0 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419f730 con 0x7f1ea4101670 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.384+0000 7f1eabbff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea419f8c0 con 0x7f1ea4100ce0 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33216/0 (socket says 172.21.15.26:33216) 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 -- 172.21.15.26:0/3236712358 learned_addr learned my addr 172.21.15.26:0/3236712358 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea4101670 0x7f1ea419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea919a700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 -- 172.21.15.26:0/3236712358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 msgr2=0x7f1ea4197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 -- 172.21.15.26:0/3236712358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea4101670 msgr2=0x7f1ea419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea4101670 0x7f1ea419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 -- 172.21.15.26:0/3236712358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00095f0 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea919a700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.385+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0004730 tx=0x7f1ea0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.386+0000 7f1e9affd700 1 -- 172.21.15.26:0/3236712358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea001d070 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.386+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea419fb40 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.386+0000 7f1e9affd700 1 -- 172.21.15.26:0/3236712358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea0021410 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.386+0000 7f1e9affd700 1 -- 172.21.15.26:0/3236712358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0017650 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.386+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea419ffc0 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.389+0000 7f1e9affd700 1 -- 172.21.15.26:0/3236712358 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ea0021810 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 Processor -- start 2023-07-13T22:25:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 -- start start 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff92ff10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ef4cb0 con 0x55ff92ff0c70 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92f47a40 con 0x55ff92fef4d0 2023-07-13T22:25:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92f40ed0 con 0x55ff92fefe20 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.391+0000 7f1e9affd700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 0x7f1e900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1e9affd700 1 -- 172.21.15.26:0/3236712358 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ea00cde30 con 0x7f1ea40ffaf0 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44376/0 (socket says 172.21.15.26:44376) 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 -- 172.21.15.26:0/2193676489 learned_addr learned my addr 172.21.15.26:0/2193676489 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 0x7f1e900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea919a700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff92ff10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 -- 172.21.15.26:0/2193676489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 msgr2=0x55ff92fef8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fef8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 -- 172.21.15.26:0/2193676489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 msgr2=0x55ff92ff10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff92ff10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 -- 172.21.15.26:0/2193676489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff92d6ede0 con 0x55ff92fefe20 2023-07-13T22:25:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fef8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:25:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.392+0000 7f1ea999b700 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009af80 tx=0x7f1ea009b180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26468a2c29191f47 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 0x7f1e900a3600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0086b0 tx=0x7f1e9c009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1e8ffff700 1 -- 172.21.15.26:0/2193676489 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009f2b0 con 0x55ff92fefe20 2023-07-13T22:25:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1e8ffff700 1 -- 172.21.15.26:0/2193676489 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea00a3640 con 0x55ff92fefe20 2023-07-13T22:25:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1e8ffff700 1 -- 172.21.15.26:0/2193676489 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009b740 con 0x55ff92fefe20 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1ebf543b80 1 -- 172.21.15.26:0/2193676489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 msgr2=0x55ff92ff06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.393+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009af80 tx=0x7f1ea009b180 comp rx=0 tx=0).stop 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 -- 172.21.15.26:0/2193676489 shutdown_connections 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff92ff10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fef8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/2193676489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ff06e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 -- 172.21.15.26:0/2193676489 >> 172.21.15.26:0/2193676489 conn(0x55ff92fe4da0 msgr2=0x55ff92fea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 -- 172.21.15.26:0/2193676489 shutdown_connections 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 -- 172.21.15.26:0/2193676489 wait complete. 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 -- start start 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.394+0000 7f1ebf543b80 1 Processor -- start 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fefe20 0x55ff92ff92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff93000320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ee0ac0 con 0x55ff92fefe20 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ee07c0 con 0x55ff92ff0c70 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92dc4920 con 0x55ff92fef4d0 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ebf543b80 10 client.0 _ll_register_callbacks cb 0x55ff92ede760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea919a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fefe20 0x55ff92ff92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44386/0 (socket says 172.21.15.26:44386) 2023-07-13T22:25:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 -- 172.21.15.26:0/4285729463 learned_addr learned my addr 172.21.15.26:0/4285729463 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff93000320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 -- 172.21.15.26:0/4285729463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 msgr2=0x55ff93000320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff93000320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.395+0000 7f1ea999b700 1 -- 172.21.15.26:0/4285729463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fefe20 msgr2=0x55ff92ff92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ea999b700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fefe20 0x55ff92ff92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ea999b700 1 -- 172.21.15.26:0/4285729463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea419ffc0 con 0x55ff92fef4d0 2023-07-13T22:25:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ea999b700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000b2e0 tx=0x7f1ea0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a35c01934d8076a server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1e8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0099620 con 0x55ff92fef4d0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ea009f2b0 con 0x55ff92fef4d0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff92e9a4f0 con 0x55ff92fef4d0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea009f020 con 0x55ff92fef4d0 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ebf543b80 10 client.27439 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.396+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ff92d57c60 con 0x55ff92fef4d0 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff92e9a4f0 con 0x55ff92fef4d0 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ea0084ad0 con 0x55ff92fef4d0 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 10 client.27439 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 10 client.27439 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 20 client.27439 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93004600 0x55ff93006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 4 client.27439 mds_command: new command op to 24493 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff92f39eb0 con 0x55ff93004600 2023-07-13T22:25:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.397+0000 7f1ea919a700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93004600 0x55ff93006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.398+0000 7f1ea919a700 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93004600 0x55ff93006a10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=20dc4987b94e3cac server_cookie=3cf0492692e4dc15 in_seq=0 out_seq=0 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.398+0000 7f1e8d7fa700 10 client.27439 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.398+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/4285729463 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff92f39eb0 con 0x55ff93004600 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.398+0000 7f1e8d7fa700 10 client.27439 handle_command_reply: tid=0 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.398+0000 7f1ebf543b80 1 client.27439 shutdown 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 msgr2=0x55ff92ff8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ea000b2e0 tx=0x7f1ea0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 shutdown_connections 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93004600 0x55ff93006a10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff93000320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fef4d0 0x55ff92ff8da0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/4285729463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fefe20 0x55ff92ff92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 >> 172.21.15.26:0/4285729463 conn(0x55ff92fe4da0 msgr2=0x55ff92fe98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1e8d7fa700 0 client.27439 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:46.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 shutdown_connections 2023-07-13T22:25:46.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 1 -- 172.21.15.26:0/4285729463 wait complete. 2023-07-13T22:25:46.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.399+0000 7f1ebf543b80 20 client.27439 trim_cache size 0 max 16384 2023-07-13T22:25:46.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 Processor -- start 2023-07-13T22:25:46.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 -- start start 2023-07-13T22:25:46.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff930002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff93000c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ff8970 con 0x55ff92ff0c70 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ff8280 con 0x55ff92fef4d0 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92ff0520 con 0x55ff92fefe20 2023-07-13T22:25:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1eaa19c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff93000c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ea919a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff930002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52692/0 (socket says 172.21.15.26:52692) 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.413+0000 7f1ea999b700 1 -- 172.21.15.26:0/3095865003 learned_addr learned my addr 172.21.15.26:0/3095865003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 -- 172.21.15.26:0/3095865003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 msgr2=0x55ff930002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff930002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 -- 172.21.15.26:0/3095865003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 msgr2=0x55ff93000c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff93000c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 -- 172.21.15.26:0/3095865003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff92d95250 con 0x55ff92fef4d0 2023-07-13T22:25:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea919a700 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff930002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1ea999b700 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0097cf0 tx=0x7f1ea009b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d97bf289caf42977 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1e98bff700 1 -- 172.21.15.26:0/3095865003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009f2b0 con 0x55ff92fef4d0 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1e98bff700 1 -- 172.21.15.26:0/3095865003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea0095e90 con 0x55ff92fef4d0 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.414+0000 7f1e98bff700 1 -- 172.21.15.26:0/3095865003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ac1f0 con 0x55ff92fef4d0 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- 172.21.15.26:0/3095865003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 msgr2=0x55ff92fe6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0097cf0 tx=0x7f1ea009b400 comp rx=0 tx=0).stop 2023-07-13T22:25:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- 172.21.15.26:0/3095865003 shutdown_connections 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92ff0c70 0x55ff93000c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92fef4d0 0x55ff92fe6310 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/3095865003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff930002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- 172.21.15.26:0/3095865003 >> 172.21.15.26:0/3095865003 conn(0x55ff92fe4da0 msgr2=0x55ff9300b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- 172.21.15.26:0/3095865003 shutdown_connections 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- 172.21.15.26:0/3095865003 wait complete. 2023-07-13T22:25:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.415+0000 7f1ebf543b80 1 -- start start 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 Processor -- start 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ffc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff92fff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92fff8e0 con 0x55ff92fef4d0 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92fffa50 con 0x55ff92ff0c70 2023-07-13T22:25:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92fffc10 con 0x55ff92fefe20 2023-07-13T22:25:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.416+0000 7f1ebf543b80 10 client.0 _ll_register_callbacks cb 0x7f1ea009b050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33234/0 (socket says 172.21.15.26:33234) 2023-07-13T22:25:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 -- 172.21.15.26:0/1091113439 learned_addr learned my addr 172.21.15.26:0/1091113439 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1eaa19c700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff92fff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea919a700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ffc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 -- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 msgr2=0x55ff92ffc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ffc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 -- 172.21.15.26:0/1091113439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 msgr2=0x55ff92fff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff92fff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 -- 172.21.15.26:0/1091113439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e94009af0 con 0x55ff92fef4d0 2023-07-13T22:25:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1ea999b700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00098e0 tx=0x7f1ea009aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85c084a62dc55ed server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1e8d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:25:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009f330 con 0x55ff92fef4d0 2023-07-13T22:25:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.417+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ea0084b40 con 0x55ff92fef4d0 2023-07-13T22:25:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ea00a3640 con 0x55ff92fef4d0 2023-07-13T22:25:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea00a31b0 con 0x55ff92fef4d0 2023-07-13T22:25:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 10 client.16788 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ff92d57c60 con 0x55ff92fef4d0 2023-07-13T22:25:46.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea00a3640 con 0x55ff92fef4d0 2023-07-13T22:25:46.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1ea009c060 con 0x55ff92fef4d0 2023-07-13T22:25:46.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 10 client.16788 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 10 client.16788 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-07-13T22:25:46.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 20 client.16788 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93058b90 0x55ff9305af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 4 client.16788 mds_command: new command op to 24493 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:46.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.418+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 --> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff92efc050 con 0x55ff93058b90 2023-07-13T22:25:46.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ea919a700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93058b90 0x55ff9305af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ea919a700 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93058b90 0x55ff9305af70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=490b3df45df557b3 server_cookie=f770107f5ed9d34d in_seq=0 out_seq=0 2023-07-13T22:25:46.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1e8d7fa700 10 client.16788 ms_handle_connect on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:46.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1e8d7fa700 1 -- 172.21.15.26:0/1091113439 <== mds.3 v2:172.21.15.121:6834/59384395 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff92efc050 con 0x55ff93058b90 2023-07-13T22:25:46.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1e8d7fa700 10 client.16788 handle_command_reply: tid=0 2023-07-13T22:25:46.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 client.16788 shutdown 2023-07-13T22:25:46.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 msgr2=0x55ff92ffbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1ea00098e0 tx=0x7f1ea009aef0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 shutdown_connections 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:6834/59384395,v1:172.21.15.121:6835/59384395] conn(0x55ff93058b90 0x55ff9305af70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ff92fef4d0 0x55ff92ffbe40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ff92ff0c70 0x55ff92fff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1ebf543b80 1 --2- 172.21.15.26:0/1091113439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ff92fefe20 0x55ff92ffc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.419+0000 7f1e8d7fa700 0 client.16788 ms_handle_reset on v2:172.21.15.121:6834/59384395 2023-07-13T22:25:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 >> 172.21.15.26:0/1091113439 conn(0x55ff92fe4da0 msgr2=0x55ff93000c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 shutdown_connections 2023-07-13T22:25:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1ebf543b80 1 -- 172.21.15.26:0/1091113439 wait complete. 2023-07-13T22:25:46.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1ebf543b80 20 client.16788 trim_cache size 0 max 16384 2023-07-13T22:25:46.416 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:46.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 msgr2=0x7f1e900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 0x7f1e900a3600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0086b0 tx=0x7f1e9c009920 comp rx=0 tx=0).stop 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 msgr2=0x7f1ea4197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0004730 tx=0x7f1ea0004760 comp rx=0 tx=0).stop 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 shutdown_connections 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f1e900a1150 0x7f1e900a3600 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ea40ffaf0 0x7f1ea4197a70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ea4101670 0x7f1ea419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 --2- 172.21.15.26:0/3236712358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ea4100ce0 0x7f1ea4197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.420+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 >> 172.21.15.26:0/3236712358 conn(0x7f1ea40fb130 msgr2=0x7f1ea40fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.421+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 shutdown_connections 2023-07-13T22:25:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.421+0000 7f1eabbff700 1 -- 172.21.15.26:0/3236712358 wait complete. 2023-07-13T22:25:46.421 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-13T22:25:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 -- 172.21.15.26:0/530068893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 msgr2=0x7f2fb00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/530068893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb00fff00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009990 tx=0x7f2fa0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 -- 172.21.15.26:0/530068893 shutdown_connections 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/530068893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb00fff00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/530068893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 0x7f2fb0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/530068893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 -- 172.21.15.26:0/530068893 >> 172.21.15.26:0/530068893 conn(0x7f2fb00fb130 msgr2=0x7f2fb00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 -- 172.21.15.26:0/530068893 shutdown_connections 2023-07-13T22:25:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.646+0000 7f2fb69ff700 1 -- 172.21.15.26:0/530068893 wait complete. 2023-07-13T22:25:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.647+0000 7f2fb69ff700 1 Processor -- start 2023-07-13T22:25:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.647+0000 7f2fb69ff700 1 -- start start 2023-07-13T22:25:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 0x7f2fb019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019d400 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019d540 con 0x7f2fb0101670 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb69ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb019d6d0 con 0x7f2fb0100ce0 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44432/0 (socket says 172.21.15.26:44432) 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33252/0 (socket says 172.21.15.26:33252) 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faf7fe700 1 -- 172.21.15.26:0/840938155 learned_addr learned my addr 172.21.15.26:0/840938155 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 0x7f2fb019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 -- 172.21.15.26:0/840938155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 msgr2=0x7f2fb0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 -- 172.21.15.26:0/840938155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 msgr2=0x7f2fb019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.648+0000 7f2faffff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 0x7f2fb019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2faffff700 1 -- 172.21.15.26:0/840938155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00095f0 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2faffff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000b620 tx=0x7f2fa0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2fad7fa700 1 -- 172.21.15.26:0/840938155 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa001d070 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2fad7fa700 1 -- 172.21.15.26:0/840938155 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa0021410 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2fad7fa700 1 -- 172.21.15.26:0/840938155 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0017650 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb019d950 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.649+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb019dd70 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.653+0000 7f2fad7fa700 1 -- 172.21.15.26:0/840938155 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2fa0021580 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.654+0000 7f2fca265b80 1 Processor -- start 2023-07-13T22:25:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.654+0000 7f2fca265b80 1 -- start start 2023-07-13T22:25:46.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7b76cb0 con 0x5599c7c76230 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7bc9a40 con 0x5599c7c77080 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fca265b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7bc2ed0 con 0x5599c7c758e0 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fad7fa700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 0x7f2f980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52722/0 (socket says 172.21.15.26:52722) 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33266/0 (socket says 172.21.15.26:33266) 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2faf7fe700 1 -- 172.21.15.26:0/2292153982 learned_addr learned my addr 172.21.15.26:0/2292153982 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fad7fa700 1 -- 172.21.15.26:0/840938155 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa00cdd80 con 0x7f2fb00ffaf0 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.655+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 0x7f2f980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 -- 172.21.15.26:0/2292153982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 msgr2=0x5599c7c75cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c75cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 -- 172.21.15.26:0/2292153982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 msgr2=0x5599c7c76af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 -- 172.21.15.26:0/2292153982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c79f0de0 con 0x5599c7c77080 2023-07-13T22:25:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faffff700 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40105e0 tx=0x7f2fa40108f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a6d2c780165236 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 0x7f2f980a3680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009fd0 tx=0x7f2fa800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2f967fc700 1 -- 172.21.15.26:0/2292153982 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011640 con 0x5599c7c77080 2023-07-13T22:25:46.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2f967fc700 1 -- 172.21.15.26:0/2292153982 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa4015070 con 0x5599c7c77080 2023-07-13T22:25:46.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.656+0000 7f2f967fc700 1 -- 172.21.15.26:0/2292153982 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4019b80 con 0x5599c7c77080 2023-07-13T22:25:46.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 -- 172.21.15.26:0/2292153982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 msgr2=0x5599c7c774f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40105e0 tx=0x7f2fa40108f0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 -- 172.21.15.26:0/2292153982 shutdown_connections 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c774f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 --2- 172.21.15.26:0/2292153982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c76af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 -- 172.21.15.26:0/2292153982 >> 172.21.15.26:0/2292153982 conn(0x5599c7c66da0 msgr2=0x5599c7c6c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 -- 172.21.15.26:0/2292153982 shutdown_connections 2023-07-13T22:25:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.657+0000 7f2fca265b80 1 -- 172.21.15.26:0/2292153982 wait complete. 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.658+0000 7f2fca265b80 1 -- start start 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.658+0000 7f2fca265b80 1 Processor -- start 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.658+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c7f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.658+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fca265b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7b62ac0 con 0x5599c7c77080 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fca265b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7b627c0 con 0x5599c7c76230 2023-07-13T22:25:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fca265b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7a46920 con 0x5599c7c758e0 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fca265b80 10 client.0 _ll_register_callbacks cb 0x5599c7b60760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fb4f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c7f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52726/0 (socket says 172.21.15.26:52726) 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fb4f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33268/0 (socket says 172.21.15.26:33268) 2023-07-13T22:25:46.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faf7fe700 1 -- 172.21.15.26:0/3341713186 learned_addr learned my addr 172.21.15.26:0/3341713186 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 -- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 msgr2=0x5599c7c7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 -- 172.21.15.26:0/3341713186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 msgr2=0x5599c7c86a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faffff700 1 -- 172.21.15.26:0/3341713186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb019dd70 con 0x5599c7c758e0 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.659+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2faffff700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c7f4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00098e0 tx=0x7f2fa0027640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496e17208fac750 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa009f020 con 0x5599c7c758e0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa00aee40 con 0x5599c7c758e0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c7b1c4f0 con 0x5599c7c758e0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa00a10b0 con 0x5599c7c758e0 2023-07-13T22:25:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2fca265b80 10 client.27457 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2fa4007e40 con 0x5599c7c758e0 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599c7b1c4f0 con 0x5599c7c758e0 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.660+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2fa00a3190 con 0x5599c7c758e0 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 10 client.27457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 20 client.27457 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7c8af30 0x5599c7c8d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 4 client.27457 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c7bbbeb0 con 0x5599c7c8af30 2023-07-13T22:25:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.661+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7c8af30 0x5599c7c8d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2f8f7fe700 10 client.27457 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7c8af30 0x5599c7c8d340 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e77670ea840862d8 server_cookie=a772c2ada0cdc79a in_seq=0 out_seq=0 2023-07-13T22:25:46.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3341713186 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599c7bbbeb0 con 0x5599c7c8af30 2023-07-13T22:25:46.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2f8f7fe700 10 client.27457 handle_command_reply: tid=0 2023-07-13T22:25:46.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 client.27457 shutdown 2023-07-13T22:25:46.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 msgr2=0x5599c7c7f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c7f4d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00098e0 tx=0x7f2fa0027640 comp rx=0 tx=0).stop 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 shutdown_connections 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c77080 0x5599c7c86a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.662+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7c8af30 0x5599c7c8d340 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c758e0 0x5599c7c7f4d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3341713186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c76230 0x5599c7c7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2f8f7fe700 0 client.27457 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 >> 172.21.15.26:0/3341713186 conn(0x5599c7c66da0 msgr2=0x5599c7c6b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 shutdown_connections 2023-07-13T22:25:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 1 -- 172.21.15.26:0/3341713186 wait complete. 2023-07-13T22:25:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.663+0000 7f2fca265b80 20 client.27457 trim_cache size 0 max 16384 2023-07-13T22:25:46.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.676+0000 7f2fca265b80 1 Processor -- start 2023-07-13T22:25:46.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.676+0000 7f2fca265b80 1 -- start start 2023-07-13T22:25:46.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 0x5599c7c86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c7d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c6d4c0 con 0x5599c7c758e0 2023-07-13T22:25:46.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c7e980 con 0x5599c7c77080 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fca265b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c8ad50 con 0x5599c7c76230 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33284/0 (socket says 172.21.15.26:33284) 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 -- 172.21.15.26:0/1124545139 learned_addr learned my addr 172.21.15.26:0/1124545139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 0x5599c7c86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c7d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 -- 172.21.15.26:0/1124545139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 msgr2=0x5599c7c86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 0x5599c7c86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 -- 172.21.15.26:0/1124545139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 msgr2=0x5599c7c7d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c7d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faffff700 1 -- 172.21.15.26:0/1124545139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c79d9c60 con 0x5599c7c758e0 2023-07-13T22:25:46.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 0x5599c7c86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:46.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.677+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c7d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-13T22:25:46.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2faffff700 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0097c40 tx=0x7f2fa009adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218be2aec76f77a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2f96ffd700 1 -- 172.21.15.26:0/1124545139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0066800 con 0x5599c7c758e0 2023-07-13T22:25:46.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2f96ffd700 1 -- 172.21.15.26:0/1124545139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa009b1d0 con 0x5599c7c758e0 2023-07-13T22:25:46.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2f96ffd700 1 -- 172.21.15.26:0/1124545139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa009a920 con 0x5599c7c758e0 2023-07-13T22:25:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2fca265b80 1 -- 172.21.15.26:0/1124545139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 msgr2=0x5599c7c87280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.678+0000 7f2fca265b80 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0097c40 tx=0x7f2fa009adf0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 -- 172.21.15.26:0/1124545139 shutdown_connections 2023-07-13T22:25:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c758e0 0x5599c7c87280 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c77080 0x5599c7c7d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 --2- 172.21.15.26:0/1124545139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c76230 0x5599c7c86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 -- 172.21.15.26:0/1124545139 >> 172.21.15.26:0/1124545139 conn(0x5599c7c66da0 msgr2=0x5599c7c91ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 -- 172.21.15.26:0/1124545139 shutdown_connections 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 -- 172.21.15.26:0/1124545139 wait complete. 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 -- start start 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.679+0000 7f2fca265b80 1 Processor -- start 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c758e0 0x5599c7c829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c77080 0x5599c7c85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c86460 con 0x5599c7c76230 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c865d0 con 0x5599c7c758e0 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c7c86790 con 0x5599c7c77080 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fca265b80 10 client.0 _ll_register_callbacks cb 0x7f2fa0097510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2faf7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33294/0 (socket says 172.21.15.26:33294) 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2faf7fe700 1 -- 172.21.15.26:0/3266108331 learned_addr learned my addr 172.21.15.26:0/3266108331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2fb4f9c700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c77080 0x5599c7c85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.680+0000 7f2faffff700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c758e0 0x5599c7c829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 -- 172.21.15.26:0/3266108331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c77080 msgr2=0x5599c7c85f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c77080 0x5599c7c85f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 -- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c758e0 msgr2=0x5599c7c829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c758e0 0x5599c7c829c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 -- 172.21.15.26:0/3266108331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4007e40 con 0x5599c7c76230 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2faf7fe700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4008010 tx=0x7f2fa4005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4902e95f184463f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4019650 con 0x5599c7c76230 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fa40199a0 con 0x5599c7c76230 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.681+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa4015070 con 0x5599c7c76230 2023-07-13T22:25:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa40197b0 con 0x5599c7c76230 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 10 client.16806 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5599c7a17250 con 0x5599c7c76230 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fa4015070 con 0x5599c7c76230 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f2fa4010610 con 0x5599c7c76230 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 10 client.16806 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 20 client.16806 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7cdff80 0x5599c7ce2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 4 client.16806 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 --> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599c7b7e050 con 0x5599c7cdff80 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.682+0000 7f2faffff700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7cdff80 0x5599c7ce2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2faffff700 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7cdff80 0x5599c7ce2390 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9dc7f192ec6c7ae1 server_cookie=467f24a97084a468 in_seq=0 out_seq=0 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2f8f7fe700 10 client.16806 ms_handle_connect on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2f8f7fe700 1 -- 172.21.15.26:0/3266108331 <== mds.2 v2:172.21.15.117:6836/1128080760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599c7b7e050 con 0x5599c7cdff80 2023-07-13T22:25:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2f8f7fe700 10 client.16806 handle_command_reply: tid=0 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 client.16806 shutdown 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 msgr2=0x5599c7c82f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4008010 tx=0x7f2fa4005f60 comp rx=0 tx=0).stop 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 shutdown_connections 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:6836/1128080760,v1:172.21.15.117:6837/1128080760] conn(0x5599c7cdff80 0x5599c7ce2390 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599c7c77080 0x5599c7c85f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5599c7c758e0 0x5599c7c829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 --2- 172.21.15.26:0/3266108331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599c7c76230 0x5599c7c82f00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2f8f7fe700 0 client.16806 ms_handle_reset on v2:172.21.15.117:6836/1128080760 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 >> 172.21.15.26:0/3266108331 conn(0x5599c7c66da0 msgr2=0x5599c7c87280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 shutdown_connections 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.683+0000 7f2fca265b80 1 -- 172.21.15.26:0/3266108331 wait complete. 2023-07-13T22:25:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fca265b80 20 client.16806 trim_cache size 0 max 16384 2023-07-13T22:25:46.678 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 msgr2=0x7f2f980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 0x7f2f980a3680 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009fd0 tx=0x7f2fa800b040 comp rx=0 tx=0).stop 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 msgr2=0x7f2fb0195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000b620 tx=0x7f2fa0005740 comp rx=0 tx=0).stop 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 shutdown_connections 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2f980a11d0 0x7f2f980a3680 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fb00ffaf0 0x7f2fb0195880 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fb0101670 0x7f2fb019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 --2- 172.21.15.26:0/840938155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2fb0100ce0 0x7f2fb0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 >> 172.21.15.26:0/840938155 conn(0x7f2fb00fb130 msgr2=0x7f2fb01041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 shutdown_connections 2023-07-13T22:25:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.684+0000 7f2fb69ff700 1 -- 172.21.15.26:0/840938155 wait complete. 2023-07-13T22:25:46.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-13T22:25:46.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:46 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1663605607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.984+0000 7f8802fff700 1 -- 172.21.15.26:0/2307671431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 msgr2=0x7f87fc1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.984+0000 7f8802fff700 1 --2- 172.21.15.26:0/2307671431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc1033a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009990 tx=0x7f87ec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 -- 172.21.15.26:0/2307671431 shutdown_connections 2023-07-13T22:25:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 --2- 172.21.15.26:0/2307671431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc10a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 --2- 172.21.15.26:0/2307671431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 --2- 172.21.15.26:0/2307671431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc1033a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 -- 172.21.15.26:0/2307671431 >> 172.21.15.26:0/2307671431 conn(0x7f87fc0fd3c0 msgr2=0x7f87fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 -- 172.21.15.26:0/2307671431 shutdown_connections 2023-07-13T22:25:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.985+0000 7f8802fff700 1 -- 172.21.15.26:0/2307671431 wait complete. 2023-07-13T22:25:46.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 Processor -- start 2023-07-13T22:25:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 -- start start 2023-07-13T22:25:46.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19f5c0 con 0x7f87fc1038e0 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19f700 con 0x7f87fc101d60 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.987+0000 7f8802fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19f890 con 0x7f87fc102f50 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52768/0 (socket says 172.21.15.26:52768) 2023-07-13T22:25:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 -- 172.21.15.26:0/1781058193 learned_addr learned my addr 172.21.15.26:0/1781058193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f880159c700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f87fbfff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 -- 172.21.15.26:0/1781058193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 msgr2=0x7f87fc197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 -- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 msgr2=0x7f87fc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f8800d9b700 1 -- 172.21.15.26:0/1781058193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0095f0 con 0x7f87fc101d60 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.988+0000 7f880159c700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f8800d9b700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f87f9ffb700 1 -- 172.21.15.26:0/1781058193 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4018070 con 0x7f87fc101d60 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f87f9ffb700 1 -- 172.21.15.26:0/1781058193 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f400f040 con 0x7f87fc101d60 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f87f9ffb700 1 -- 172.21.15.26:0/1781058193 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004280 con 0x7f87fc101d60 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc19fb10 con 0x7f87fc101d60 2023-07-13T22:25:46.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.989+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc19ffd0 con 0x7f87fc101d60 2023-07-13T22:25:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.996+0000 7f88169abb80 1 Processor -- start 2023-07-13T22:25:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.996+0000 7f88169abb80 1 -- start start 2023-07-13T22:25:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.996+0000 7f88169abb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f88169abb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f88169abb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3cc70 0x5580aad3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f88169abb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aac40cb0 con 0x5580aad3b4d0 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f88169abb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aac93a40 con 0x5580aad3cc70 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f88169abb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aac8ced0 con 0x5580aad3be20 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f87f9ffb700 1 -- 172.21.15.26:0/1781058193 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f87f4003a00 con 0x7f87fc101d60 2023-07-13T22:25:46.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44500/0 (socket says 172.21.15.26:44500) 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.997+0000 7f8800d9b700 1 -- 172.21.15.26:0/4182182126 learned_addr learned my addr 172.21.15.26:0/4182182126 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f87fbfff700 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3cc70 0x5580aad3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f8800d9b700 1 -- 172.21.15.26:0/4182182126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3cc70 msgr2=0x5580aad3d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f8800d9b700 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3cc70 0x5580aad3d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f8800d9b700 1 -- 172.21.15.26:0/4182182126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 msgr2=0x5580aad3b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f8800d9b700 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.998+0000 7f8800d9b700 1 -- 172.21.15.26:0/4182182126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580aaabade0 con 0x5580aad3be20 2023-07-13T22:25:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:46.999+0000 7f8800d9b700 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87f40d2a30 tx=0x7f87f40d2d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc46f10359e9ff7 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.000+0000 7f87f9ffb700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 0x7f87e80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.000+0000 7f87f9ffb700 1 -- 172.21.15.26:0/1781058193 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87f40cc790 con 0x7f87fc101d60 2023-07-13T22:25:46.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.002+0000 7f87f2ffd700 1 -- 172.21.15.26:0/4182182126 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40d47b0 con 0x5580aad3be20 2023-07-13T22:25:46.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.002+0000 7f87f2ffd700 1 -- 172.21.15.26:0/4182182126 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f40d4dd0 con 0x5580aad3be20 2023-07-13T22:25:46.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.002+0000 7f87f2ffd700 1 -- 172.21.15.26:0/4182182126 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40d8460 con 0x5580aad3be20 2023-07-13T22:25:46.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.003+0000 7f880159c700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 0x7f87e80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.003+0000 7f88169abb80 1 -- 172.21.15.26:0/4182182126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 msgr2=0x5580aad3c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:46.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.003+0000 7f88169abb80 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f87f40d2a30 tx=0x7f87f40d2d40 comp rx=0 tx=0).stop 2023-07-13T22:25:46.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f88169abb80 1 -- 172.21.15.26:0/4182182126 shutdown_connections 2023-07-13T22:25:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f88169abb80 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f88169abb80 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3cc70 0x5580aad3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f88169abb80 1 --2- 172.21.15.26:0/4182182126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3be20 0x5580aad3c6e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f88169abb80 1 -- 172.21.15.26:0/4182182126 >> 172.21.15.26:0/4182182126 conn(0x5580aad30da0 msgr2=0x5580aad361d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:46.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.004+0000 7f880159c700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 0x7f87e80a3700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f87e4005d90 tx=0x7f87e4005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.007+0000 7f88169abb80 1 -- 172.21.15.26:0/4182182126 shutdown_connections 2023-07-13T22:25:46.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.007+0000 7f88169abb80 1 -- 172.21.15.26:0/4182182126 wait complete. 2023-07-13T22:25:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.008+0000 7f88169abb80 1 -- start start 2023-07-13T22:25:46.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.008+0000 7f88169abb80 1 Processor -- start 2023-07-13T22:25:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aac2cac0 con 0x5580aad3be20 2023-07-13T22:25:46.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aac2c7c0 con 0x5580aad3b4d0 2023-07-13T22:25:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aab10920 con 0x5580aad3cc70 2023-07-13T22:25:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f88169abb80 10 client.0 _ll_register_callbacks cb 0x5580aac2a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f880159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f880159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44508/0 (socket says 172.21.15.26:44508) 2023-07-13T22:25:46.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.010+0000 7f880159c700 1 -- 172.21.15.26:0/3608624511 learned_addr learned my addr 172.21.15.26:0/3608624511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f87fbfff700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f880159c700 1 -- 172.21.15.26:0/3608624511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 msgr2=0x5580aad451b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f880159c700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f880159c700 1 -- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 msgr2=0x5580aad456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f880159c700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:46.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.011+0000 7f880159c700 1 -- 172.21.15.26:0/3608624511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc19ffd0 con 0x5580aad3cc70 2023-07-13T22:25:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.013+0000 7f880159c700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f87e4011710 tx=0x7f87e400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a60e3e114a05b86 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:46.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.013+0000 7f87dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:46.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.014+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e401e830 con 0x5580aad3cc70 2023-07-13T22:25:46.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.015+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87e4022070 con 0x5580aad3cc70 2023-07-13T22:25:46.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.015+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580aabe64f0 con 0x5580aad3cc70 2023-07-13T22:25:46.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.015+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e401e990 con 0x5580aad3cc70 2023-07-13T22:25:46.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.015+0000 7f88169abb80 10 client.27472 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:46.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.015+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580aaaa3c60 con 0x5580aad3cc70 2023-07-13T22:25:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5580aabe64f0 con 0x5580aad3cc70 2023-07-13T22:25:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f87e402f3d0 con 0x5580aad3cc70 2023-07-13T22:25:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 10 client.27472 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:46.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 10 client.27472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:25:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 20 client.27472 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aad51040 0x5580aad53450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 4 client.27472 mds_command: new command op to 24563 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.016+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580aac85eb0 con 0x5580aad51040 2023-07-13T22:25:46.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.017+0000 7f8800d9b700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aad51040 0x5580aad53450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.017+0000 7f8800d9b700 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aad51040 0x5580aad53450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d01ba0f64b66872 server_cookie=b1b6aaad5826e222 in_seq=0 out_seq=0 2023-07-13T22:25:47.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.017+0000 7f87dbfff700 10 client.27472 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:47.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.018+0000 7f87dbfff700 1 -- 172.21.15.26:0/3608624511 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580aac85eb0 con 0x5580aad51040 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.018+0000 7f87dbfff700 10 client.27472 handle_command_reply: tid=0 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.018+0000 7f88169abb80 1 client.27472 shutdown 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 msgr2=0x5580aad4c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f87e4011710 tx=0x7f87e400b390 comp rx=0 tx=0).stop 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 shutdown_connections 2023-07-13T22:25:47.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad4c730 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aad51040 0x5580aad53450 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 --2- 172.21.15.26:0/3608624511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 >> 172.21.15.26:0/3608624511 conn(0x5580aad30da0 msgr2=0x5580aad358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f87dbfff700 0 client.27472 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 shutdown_connections 2023-07-13T22:25:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.019+0000 7f88169abb80 1 -- 172.21.15.26:0/3608624511 wait complete. 2023-07-13T22:25:47.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.020+0000 7f88169abb80 20 client.27472 trim_cache size 0 max 16384 2023-07-13T22:25:47.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.045+0000 7f88169abb80 1 Processor -- start 2023-07-13T22:25:47.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 -- start start 2023-07-13T22:25:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3be20 0x5580aad4c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad370b0 con 0x5580aad3b4d0 2023-07-13T22:25:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad3c1f0 con 0x5580aad3be20 2023-07-13T22:25:47.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.046+0000 7f88169abb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad37d00 con 0x5580aad3cc70 2023-07-13T22:25:47.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f8800d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33332/0 (socket says 172.21.15.26:33332) 2023-07-13T22:25:47.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f8800d9b700 1 -- 172.21.15.26:0/2631449314 learned_addr learned my addr 172.21.15.26:0/2631449314 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f87fbfff700 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3be20 0x5580aad4c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 -- 172.21.15.26:0/2631449314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3be20 msgr2=0x5580aad4c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3be20 0x5580aad4c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 -- 172.21.15.26:0/2631449314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 msgr2=0x5580aad451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.047+0000 7f880159c700 1 -- 172.21.15.26:0/2631449314 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580aaae1250 con 0x5580aad3cc70 2023-07-13T22:25:47.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.048+0000 7f880159c700 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad3fb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87e40116e0 tx=0x7f87e400bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916a3a69649b3fb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.048+0000 7f87f37fe700 1 -- 172.21.15.26:0/2631449314 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e402f920 con 0x5580aad3cc70 2023-07-13T22:25:47.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.049+0000 7f87f37fe700 1 -- 172.21.15.26:0/2631449314 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e4015070 con 0x5580aad3cc70 2023-07-13T22:25:47.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.049+0000 7f87f37fe700 1 -- 172.21.15.26:0/2631449314 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e40038e0 con 0x5580aad3cc70 2023-07-13T22:25:47.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 -- 172.21.15.26:0/2631449314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 msgr2=0x5580aad3fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad3fb90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f87e40116e0 tx=0x7f87e400bef0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 -- 172.21.15.26:0/2631449314 shutdown_connections 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3b4d0 0x5580aad451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad3fb90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 --2- 172.21.15.26:0/2631449314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3be20 0x5580aad4c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.050+0000 7f88169abb80 1 -- 172.21.15.26:0/2631449314 >> 172.21.15.26:0/2631449314 conn(0x5580aad30da0 msgr2=0x5580aad43eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.051+0000 7f88169abb80 1 -- 172.21.15.26:0/2631449314 shutdown_connections 2023-07-13T22:25:47.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.051+0000 7f88169abb80 1 -- 172.21.15.26:0/2631449314 wait complete. 2023-07-13T22:25:47.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.051+0000 7f88169abb80 1 -- start start 2023-07-13T22:25:47.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 Processor -- start 2023-07-13T22:25:47.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad4c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad4c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad48b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad490f0 con 0x5580aad3be20 2023-07-13T22:25:47.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad49260 con 0x5580aad3b4d0 2023-07-13T22:25:47.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580aad49420 con 0x5580aad3cc70 2023-07-13T22:25:47.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.052+0000 7f88169abb80 10 client.0 _ll_register_callbacks cb 0x5580aac2a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.054+0000 7f87fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad4c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.054+0000 7f87fbfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad4c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33336/0 (socket says 172.21.15.26:33336) 2023-07-13T22:25:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.054+0000 7f87fbfff700 1 -- 172.21.15.26:0/405479140 learned_addr learned my addr 172.21.15.26:0/405479140 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.054+0000 7f8800d9b700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad4c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.055+0000 7f880159c700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad48b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 -- 172.21.15.26:0/405479140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 msgr2=0x5580aad4c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad4c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 -- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 msgr2=0x5580aad4c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad4c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 -- 172.21.15.26:0/405479140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580aaaa3c60 con 0x5580aad3cc70 2023-07-13T22:25:47.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.056+0000 7f880159c700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad48b20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f87e400bb90 tx=0x7f87e402f5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ac5fcbecda6516 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f87dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:47.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003680 con 0x5580aad3cc70 2023-07-13T22:25:47.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f87e4022040 con 0x5580aad3cc70 2023-07-13T22:25:47.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87f40d4dd0 con 0x5580aad3cc70 2023-07-13T22:25:47.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e40037e0 con 0x5580aad3cc70 2023-07-13T22:25:47.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f88169abb80 10 client.27478 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:47.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.057+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5580aaab92b0 con 0x5580aad3cc70 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f40d4dd0 con 0x5580aad3cc70 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f87e401ec90 con 0x5580aad3cc70 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 10 client.27478 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 10 client.27478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 20 client.27478 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:47.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aada60c0 0x5580aada84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 4 client.27478 mds_command: new command op to 24563 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:47.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.058+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 --> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5580aac48050 con 0x5580aada60c0 2023-07-13T22:25:47.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.059+0000 7f8800d9b700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aada60c0 0x5580aada84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.060+0000 7f8800d9b700 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aada60c0 0x5580aada84d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fd8e4b731bd6c75 server_cookie=79b0759af93214df in_seq=0 out_seq=0 2023-07-13T22:25:47.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.060+0000 7f87dbfff700 10 client.27478 ms_handle_connect on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:47.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.060+0000 7f87dbfff700 1 -- 172.21.15.26:0/405479140 <== mds.1 v2:172.21.15.26:6840/2542800089 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5580aac48050 con 0x5580aada60c0 2023-07-13T22:25:47.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.060+0000 7f87dbfff700 10 client.27478 handle_command_reply: tid=0 2023-07-13T22:25:47.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.060+0000 7f88169abb80 1 client.27478 shutdown 2023-07-13T22:25:47.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 msgr2=0x5580aad48b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad48b20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f87e400bb90 tx=0x7f87e402f5e0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 shutdown_connections 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5580aad3cc70 0x5580aad48b20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580aad3b4d0 0x5580aad4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:6840/2542800089,v1:172.21.15.26:6841/2542800089] conn(0x5580aada60c0 0x5580aada84d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 --2- 172.21.15.26:0/405479140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5580aad3be20 0x5580aad4c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 >> 172.21.15.26:0/405479140 conn(0x5580aad30da0 msgr2=0x5580aad45150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f87dbfff700 0 client.27478 ms_handle_reset on v2:172.21.15.26:6840/2542800089 2023-07-13T22:25:47.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.061+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 shutdown_connections 2023-07-13T22:25:47.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.062+0000 7f88169abb80 1 -- 172.21.15.26:0/405479140 wait complete. 2023-07-13T22:25:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.062+0000 7f88169abb80 20 client.27478 trim_cache size 0 max 16384 2023-07-13T22:25:47.046 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:47.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 msgr2=0x7f87e80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 0x7f87e80a3700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f87e4005d90 tx=0x7f87e4005d20 comp rx=0 tx=0).stop 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 msgr2=0x7f87fc197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87f400efc0 tx=0x7f87f400c3a0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 shutdown_connections 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f87e80a1250 0x7f87e80a3700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87fc1038e0 0x7f87fc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87fc101d60 0x7f87fc197a40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 --2- 172.21.15.26:0/1781058193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87fc102f50 0x7f87fc197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 >> 172.21.15.26:0/1781058193 conn(0x7f87fc0fd3c0 msgr2=0x7f87fc1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 shutdown_connections 2023-07-13T22:25:47.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.063+0000 7f8802fff700 1 -- 172.21.15.26:0/1781058193 wait complete. 2023-07-13T22:25:47.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-13T22:25:47.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:46 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1663605607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:47.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:46 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1663605607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:25:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.454+0000 7fae893ff700 1 -- 172.21.15.26:0/1017305245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84070ee0 msgr2=0x7fae840712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.454+0000 7fae893ff700 1 --2- 172.21.15.26:0/1017305245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84070ee0 0x7fae840712f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fae74009990 tx=0x7fae74009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 -- 172.21.15.26:0/1017305245 shutdown_connections 2023-07-13T22:25:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 --2- 172.21.15.26:0/1017305245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae8410bd60 0x7fae8410e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 --2- 172.21.15.26:0/1017305245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84070ee0 0x7fae840712f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 --2- 172.21.15.26:0/1017305245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae84071830 0x7fae8410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 -- 172.21.15.26:0/1017305245 >> 172.21.15.26:0/1017305245 conn(0x7fae8406ca90 msgr2=0x7fae8406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 -- 172.21.15.26:0/1017305245 shutdown_connections 2023-07-13T22:25:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.455+0000 7fae893ff700 1 -- 172.21.15.26:0/1017305245 wait complete. 2023-07-13T22:25:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.457+0000 7fae893ff700 1 Processor -- start 2023-07-13T22:25:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 -- start start 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84070ee0 0x7fae84113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8410bd60 0x7fae841b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84117050 con 0x7fae8410bd60 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae841171c0 con 0x7fae84070ee0 2023-07-13T22:25:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae893ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae841b4dd0 con 0x7fae84071830 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae827fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae827fc700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44532/0 (socket says 172.21.15.26:44532) 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.458+0000 7fae827fc700 1 -- 172.21.15.26:0/182197631 learned_addr learned my addr 172.21.15.26:0/182197631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 -- 172.21.15.26:0/182197631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84070ee0 msgr2=0x7fae84113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84070ee0 0x7fae84113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 -- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8410bd60 msgr2=0x7fae841b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8410bd60 0x7fae841b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 -- 172.21.15.26:0/182197631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740095f0 con 0x7fae84071830 2023-07-13T22:25:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.459+0000 7fae827fc700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00e970 tx=0x7fae6c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.466+0000 7fae7bfff700 1 -- 172.21.15.26:0/182197631 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c009790 con 0x7fae84071830 2023-07-13T22:25:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.466+0000 7fae7bfff700 1 -- 172.21.15.26:0/182197631 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae6c0044e0 con 0x7fae84071830 2023-07-13T22:25:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.466+0000 7fae7bfff700 1 -- 172.21.15.26:0/182197631 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c003d30 con 0x7fae84071830 2023-07-13T22:25:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.466+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae841b4f70 con 0x7fae84071830 2023-07-13T22:25:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.466+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae841b53a0 con 0x7fae84071830 2023-07-13T22:25:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.473+0000 7fae7bfff700 1 -- 172.21.15.26:0/182197631 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae6c003ed0 con 0x7fae84071830 2023-07-13T22:25:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.473+0000 7fae9cc86b80 1 Processor -- start 2023-07-13T22:25:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 -- start start 2023-07-13T22:25:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 0x562da81c06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81c0c70 0x562da81c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da80c4cb0 con 0x562da81c0c70 2023-07-13T22:25:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da8117a40 con 0x562da81bf4d0 2023-07-13T22:25:47.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.474+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da8110ed0 con 0x562da81bfe20 2023-07-13T22:25:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.475+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 0x562da81c06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.475+0000 7fae82ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 0x562da81c06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44538/0 (socket says 172.21.15.26:44538) 2023-07-13T22:25:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.475+0000 7fae82ffd700 1 -- 172.21.15.26:0/1834522133 learned_addr learned my addr 172.21.15.26:0/1834522133 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.475+0000 7fae837fe700 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae837fe700 1 -- 172.21.15.26:0/1834522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 msgr2=0x562da81c06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae837fe700 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 0x562da81c06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae837fe700 1 -- 172.21.15.26:0/1834522133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81c0c70 msgr2=0x562da81c10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae837fe700 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81c0c70 0x562da81c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae837fe700 1 -- 172.21.15.26:0/1834522133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da7f3ede0 con 0x562da81bf4d0 2023-07-13T22:25:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.476+0000 7fae7bfff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 0x7fae700a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae7bfff700 1 -- 172.21.15.26:0/182197631 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae6c014070 con 0x7fae84071830 2023-07-13T22:25:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae837fe700 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81bf8e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e490 tx=0x7fae7c00e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81718b0e84fcee03 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae82ffd700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 0x7fae700a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae797fa700 1 -- 172.21.15.26:0/1834522133 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c004db0 con 0x562da81bf4d0 2023-07-13T22:25:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae797fa700 1 -- 172.21.15.26:0/1834522133 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c00f040 con 0x562da81bf4d0 2023-07-13T22:25:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.477+0000 7fae797fa700 1 -- 172.21.15.26:0/1834522133 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0043f0 con 0x562da81bf4d0 2023-07-13T22:25:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 -- 172.21.15.26:0/1834522133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 msgr2=0x562da81bf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae82ffd700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 0x7fae700a3800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae74006010 tx=0x7fae7400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81bf8e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e490 tx=0x7fae7c00e7a0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 -- 172.21.15.26:0/1834522133 shutdown_connections 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81c0c70 0x562da81c10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81bf8e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/1834522133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bfe20 0x562da81c06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 -- 172.21.15.26:0/1834522133 >> 172.21.15.26:0/1834522133 conn(0x562da81b4da0 msgr2=0x562da81ba1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.479+0000 7fae9cc86b80 1 -- 172.21.15.26:0/1834522133 shutdown_connections 2023-07-13T22:25:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.480+0000 7fae9cc86b80 1 -- 172.21.15.26:0/1834522133 wait complete. 2023-07-13T22:25:47.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.480+0000 7fae9cc86b80 1 -- start start 2023-07-13T22:25:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.482+0000 7fae9cc86b80 1 Processor -- start 2023-07-13T22:25:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bf4d0 0x562da81c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da80b0ac0 con 0x562da81bfe20 2023-07-13T22:25:47.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da80b07c0 con 0x562da81c0c70 2023-07-13T22:25:47.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da7f94920 con 0x562da81bf4d0 2023-07-13T22:25:47.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.483+0000 7fae9cc86b80 10 client.0 _ll_register_callbacks cb 0x562da80ae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:47.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.484+0000 7fae837fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.484+0000 7fae837fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52832/0 (socket says 172.21.15.26:52832) 2023-07-13T22:25:47.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.484+0000 7fae837fe700 1 -- 172.21.15.26:0/3963723676 learned_addr learned my addr 172.21.15.26:0/3963723676 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.484+0000 7fae82ffd700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bf4d0 0x562da81c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 -- 172.21.15.26:0/3963723676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bf4d0 msgr2=0x562da81c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bf4d0 0x562da81c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 -- 172.21.15.26:0/3963723676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 msgr2=0x562da81c9740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-13T22:25:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81c9740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 -- 172.21.15.26:0/3963723676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c00e140 con 0x562da81c0c70 2023-07-13T22:25:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae837fe700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009350 tx=0x7fae7c004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67e5bf0b192d8cff server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.485+0000 7fae6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2023-07-13T22:25:47.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c003c90 con 0x562da81c0c70 2023-07-13T22:25:47.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da806a4f0 con 0x562da81c0c70 2023-07-13T22:25:47.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae7c004130 con 0x562da81c0c70 2023-07-13T22:25:47.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7c003df0 con 0x562da81c0c70 2023-07-13T22:25:47.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae9cc86b80 10 client.27494 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:47.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.486+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562da7f27c60 con 0x562da81c0c70 2023-07-13T22:25:47.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.487+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562da806a4f0 con 0x562da81c0c70 2023-07-13T22:25:47.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.487+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fae7c030070 con 0x562da81c0c70 2023-07-13T22:25:47.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 10 client.27494 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:47.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 10 client.27494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:47.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 20 client.27494 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:47.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da81d4ed0 0x562da81d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 4 client.27494 mds_command: new command op to 24575 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-13T22:25:47.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da8109eb0 con 0x562da81d4ed0 2023-07-13T22:25:47.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.488+0000 7fae82ffd700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da81d4ed0 0x562da81d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.489+0000 7fae82ffd700 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da81d4ed0 0x562da81d72e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe51e8536dbf51e server_cookie=15a926165162dde0 in_seq=0 out_seq=0 2023-07-13T22:25:47.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.489+0000 7fae6a7fc700 10 client.27494 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:47.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.490+0000 7fae6a7fc700 1 -- 172.21.15.26:0/3963723676 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562da8109eb0 con 0x562da81d4ed0 2023-07-13T22:25:47.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.490+0000 7fae6a7fc700 10 client.27494 handle_command_reply: tid=0 2023-07-13T22:25:47.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.490+0000 7fae9cc86b80 1 client.27494 shutdown 2023-07-13T22:25:47.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 msgr2=0x562da81d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009350 tx=0x7fae7c004f10 comp rx=0 tx=0).stop 2023-07-13T22:25:47.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 shutdown_connections 2023-07-13T22:25:47.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da81d4ed0 0x562da81d72e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81c0c70 0x562da81d0780 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81bf4d0 0x562da81c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/3963723676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.492+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 >> 172.21.15.26:0/3963723676 conn(0x562da81b4da0 msgr2=0x562da81b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.493+0000 7fae6a7fc700 0 client.27494 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:47.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.493+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 shutdown_connections 2023-07-13T22:25:47.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.493+0000 7fae9cc86b80 1 -- 172.21.15.26:0/3963723676 wait complete. 2023-07-13T22:25:47.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.494+0000 7fae9cc86b80 20 client.27494 trim_cache size 0 max 16384 2023-07-13T22:25:47.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.522+0000 7fae9cc86b80 1 Processor -- start 2023-07-13T22:25:47.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.522+0000 7fae9cc86b80 1 -- start start 2023-07-13T22:25:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81c86b0 con 0x562da81bfe20 2023-07-13T22:25:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81c8d70 con 0x562da81bf4d0 2023-07-13T22:25:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.523+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81b4970 con 0x562da81c0c70 2023-07-13T22:25:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.524+0000 7fae837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.524+0000 7fae837fe700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44550/0 (socket says 172.21.15.26:44550) 2023-07-13T22:25:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.524+0000 7fae837fe700 1 -- 172.21.15.26:0/491339163 learned_addr learned my addr 172.21.15.26:0/491339163 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.524+0000 7fae82ffd700 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.528+0000 7fae827fc700 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.528+0000 7fae82ffd700 1 -- 172.21.15.26:0/491339163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 msgr2=0x562da81c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.528+0000 7fae82ffd700 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.528+0000 7fae82ffd700 1 -- 172.21.15.26:0/491339163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 msgr2=0x562da81d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.528+0000 7fae82ffd700 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.529+0000 7fae82ffd700 1 -- 172.21.15.26:0/491339163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da7f65250 con 0x562da81bf4d0 2023-07-13T22:25:47.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.530+0000 7fae82ffd700 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81b98b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x562da81d7b80 tx=0x7fae74020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7abf1509432bc5c server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.531+0000 7fae79ffb700 1 -- 172.21.15.26:0/491339163 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74032230 con 0x562da81bf4d0 2023-07-13T22:25:47.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.531+0000 7fae79ffb700 1 -- 172.21.15.26:0/491339163 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7402c040 con 0x562da81bf4d0 2023-07-13T22:25:47.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.531+0000 7fae79ffb700 1 -- 172.21.15.26:0/491339163 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74004900 con 0x562da81bf4d0 2023-07-13T22:25:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 -- 172.21.15.26:0/491339163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 msgr2=0x562da81b98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81b98b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x562da81d7b80 tx=0x7fae74020000 comp rx=0 tx=0).stop 2023-07-13T22:25:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 -- 172.21.15.26:0/491339163 shutdown_connections 2023-07-13T22:25:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bf4d0 0x562da81b98b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/491339163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bfe20 0x562da81d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.532+0000 7fae9cc86b80 1 -- 172.21.15.26:0/491339163 >> 172.21.15.26:0/491339163 conn(0x562da81b4da0 msgr2=0x562da81c7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.534+0000 7fae9cc86b80 1 -- 172.21.15.26:0/491339163 shutdown_connections 2023-07-13T22:25:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.534+0000 7fae9cc86b80 1 -- 172.21.15.26:0/491339163 wait complete. 2023-07-13T22:25:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.534+0000 7fae9cc86b80 1 -- start start 2023-07-13T22:25:47.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 Processor -- start 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bf4d0 0x562da81cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 0x562da81d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81ccea0 con 0x562da81bf4d0 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81cd010 con 0x562da81bfe20 2023-07-13T22:25:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da81cd1d0 con 0x562da81c0c70 2023-07-13T22:25:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.535+0000 7fae9cc86b80 10 client.0 _ll_register_callbacks cb 0x7fae7c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-13T22:25:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.536+0000 7fae827fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 0x562da81d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.536+0000 7fae827fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 0x562da81d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52858/0 (socket says 172.21.15.26:52858) 2023-07-13T22:25:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.536+0000 7fae827fc700 1 -- 172.21.15.26:0/2789231194 learned_addr learned my addr 172.21.15.26:0/2789231194 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.536+0000 7fae837fe700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.536+0000 7fae82ffd700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bf4d0 0x562da81cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 -- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 msgr2=0x562da81d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 0x562da81d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 -- 172.21.15.26:0/2789231194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bf4d0 msgr2=0x562da81cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-13T22:25:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bf4d0 0x562da81cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 -- 172.21.15.26:0/2789231194 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae84117640 con 0x562da81c0c70 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae837fe700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81cc960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x562da81d4a80 tx=0x7fae7c007b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825abe7c1db6d519 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.537+0000 7fae6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.538+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c009920 con 0x562da81c0c70 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.538+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mon.1 v2:172.21.15.121:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fae7c014550 con 0x562da81c0c70 2023-07-13T22:25:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.538+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae7c00f040 con 0x562da81c0c70 2023-07-13T22:25:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.540+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae7c014710 con 0x562da81c0c70 2023-07-13T22:25:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.540+0000 7fae9cc86b80 10 client.27496 fetch_fsmap learned FSMap version 30 2023-07-13T22:25:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.541+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562da7f3d2b0 con 0x562da81c0c70 2023-07-13T22:25:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.544+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mon.1 v2:172.21.15.121:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c03f030 con 0x562da81c0c70 2023-07-13T22:25:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.544+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mon.1 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fae7c00e810 con 0x562da81c0c70 2023-07-13T22:25:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 10 client.27496 fetch_fsmap finished waiting for FSMap version 30 2023-07-13T22:25:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 10 client.27496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-13T22:25:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 20 client.27496 populate_metadata read hostname 'smithi026' 2023-07-13T22:25:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da8229d90 0x562da822c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 4 client.27496 mds_command: new command op to 24575 tid=0[{"prefix": "damage ls"}] 2023-07-13T22:25:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 --> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562da80cc050 con 0x562da8229d90 2023-07-13T22:25:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.546+0000 7fae82ffd700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da8229d90 0x562da822c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.550+0000 7fae6a7fc700 10 client.27496 ms_handle_connect on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:47.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.550+0000 7fae82ffd700 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da8229d90 0x562da822c170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33db23c220a55491 server_cookie=2cc7af23a0587bfe in_seq=0 out_seq=0 2023-07-13T22:25:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae6a7fc700 1 -- 172.21.15.26:0/2789231194 <== mds.0 v2:172.21.15.117:6838/1698058160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562da80cc050 con 0x562da8229d90 2023-07-13T22:25:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae6a7fc700 10 client.27496 handle_command_reply: tid=0 2023-07-13T22:25:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 client.27496 shutdown 2023-07-13T22:25:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 msgr2=0x562da81cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81cc960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x562da81d4a80 tx=0x7fae7c007b30 comp rx=0 tx=0).stop 2023-07-13T22:25:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 shutdown_connections 2023-07-13T22:25:47.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562da81bf4d0 0x562da81cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:6838/1698058160,v1:172.21.15.117:6839/1698058160] conn(0x562da8229d90 0x562da822c170 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562da81c0c70 0x562da81cc960 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 --2- 172.21.15.26:0/2789231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562da81bfe20 0x562da81d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 >> 172.21.15.26:0/2789231194 conn(0x562da81b4da0 msgr2=0x562da81d6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.551+0000 7fae6a7fc700 0 client.27496 ms_handle_reset on v2:172.21.15.117:6838/1698058160 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.553+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 shutdown_connections 2023-07-13T22:25:47.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.553+0000 7fae9cc86b80 1 -- 172.21.15.26:0/2789231194 wait complete. 2023-07-13T22:25:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.554+0000 7fae9cc86b80 20 client.27496 trim_cache size 0 max 16384 2023-07-13T22:25:47.537 INFO:teuthology.orchestra.run.smithi026.stdout:[] 2023-07-13T22:25:47.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.556+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 msgr2=0x7fae700a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.556+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 0x7fae700a3800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae74006010 tx=0x7fae7400b260 comp rx=0 tx=0).stop 2023-07-13T22:25:47.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.556+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 msgr2=0x7fae84113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.556+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fae6c00e970 tx=0x7fae6c00ed30 comp rx=0 tx=0).stop 2023-07-13T22:25:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 shutdown_connections 2023-07-13T22:25:47.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fae700a1350 0x7fae700a3800 secure :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fae74006010 tx=0x7fae7400b260 comp rx=0 tx=0).stop 2023-07-13T22:25:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8410bd60 0x7fae841b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae84070ee0 0x7fae84113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 --2- 172.21.15.26:0/182197631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae84071830 0x7fae84113f70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:47.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.557+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 >> 172.21.15.26:0/182197631 conn(0x7fae8406ca90 msgr2=0x7fae8406dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.558+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 shutdown_connections 2023-07-13T22:25:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:47.559+0000 7fae893ff700 1 -- 172.21.15.26:0/182197631 wait complete. 2023-07-13T22:25:47.944 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:47 smithi026 ceph-mon[95647]: pgmap v672: 97 pgs: 97 active+clean; 8.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2023-07-13T22:25:48.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:47 smithi117 ceph-mon[129074]: pgmap v672: 97 pgs: 97 active+clean; 8.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2023-07-13T22:25:48.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:47 smithi121 ceph-mon[129187]: pgmap v672: 97 pgs: 97 active+clean; 8.6 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 230 MiB/s wr, 1.04k op/s 2023-07-13T22:25:48.576 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-07-13T22:25:48.577 INFO:tasks.fwd_scrub:done joining 2023-07-13T22:25:48.577 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-13T22:25:48.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:25:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.828+0000 7f5929fff700 1 -- 172.21.15.26:0/2817897610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 msgr2=0x7f59240733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.828+0000 7f5929fff700 1 --2- 172.21.15.26:0/2817897610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f59240733e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5918009990 tx=0x7f5918009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 -- 172.21.15.26:0/2817897610 shutdown_connections 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 --2- 172.21.15.26:0/2817897610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 0x7f5924109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 --2- 172.21.15.26:0/2817897610 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f59240729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 --2- 172.21.15.26:0/2817897610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f59240733e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 -- 172.21.15.26:0/2817897610 >> 172.21.15.26:0/2817897610 conn(0x7f59240fb0e0 msgr2=0x7f59240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 -- 172.21.15.26:0/2817897610 shutdown_connections 2023-07-13T22:25:48.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.829+0000 7f5929fff700 1 -- 172.21.15.26:0/2817897610 wait complete. 2023-07-13T22:25:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.830+0000 7f5929fff700 1 Processor -- start 2023-07-13T22:25:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.830+0000 7f5929fff700 1 -- start start 2023-07-13T22:25:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f5924197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:48.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 0x7f592419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592419f610 con 0x7f59241079c0 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592419f750 con 0x7f5924072f70 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5929fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592419f8e0 con 0x7f5924074550 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44588/0 (socket says 172.21.15.26:44588) 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 -- 172.21.15.26:0/440246048 learned_addr learned my addr 172.21.15.26:0/440246048 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f59237fe700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f5924197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:48.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5923fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 0x7f592419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 -- 172.21.15.26:0/440246048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 msgr2=0x7f5924197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f5924197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 -- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 msgr2=0x7f592419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 0x7f592419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.831+0000 7f5922ffd700 1 -- 172.21.15.26:0/440246048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c009c40 con 0x7f5924074550 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5922ffd700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5918005720 tx=0x7f5918009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591801d070 con 0x7f5924074550 2023-07-13T22:25:48.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5918021410 con 0x7f5924074550 2023-07-13T22:25:48.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59180095f0 con 0x7f5924074550 2023-07-13T22:25:48.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918017650 con 0x7f5924074550 2023-07-13T22:25:48.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.832+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592419fdf0 con 0x7f5924074550 2023-07-13T22:25:48.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.833+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5924106070 con 0x7f5924074550 2023-07-13T22:25:48.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.836+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5918021580 con 0x7f5924074550 2023-07-13T22:25:48.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.837+0000 7f5920ff9700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 0x7f590c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:48.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.837+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f59180cdb20 con 0x7f5924074550 2023-07-13T22:25:48.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.837+0000 7f59237fe700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 0x7f590c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:48.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.838+0000 7f59237fe700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 0x7f590c0a3650 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5924074920 tx=0x7f5914009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:48.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:48.839+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f59180972b0 con 0x7f5924074550 2023-07-13T22:25:49.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.049+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f592402ca60 con 0x7f5924074550 2023-07-13T22:25:49.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.051+0000 7f5920ff9700 1 -- 172.21.15.26:0/440246048 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f591809a8d0 con 0x7f5924074550 2023-07-13T22:25:49.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:25:49.034 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:25:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 msgr2=0x7f590c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 0x7f590c0a3650 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5924074920 tx=0x7f5914009260 comp rx=0 tx=0).stop 2023-07-13T22:25:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 msgr2=0x7f5924197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5918005720 tx=0x7f5918009580 comp rx=0 tx=0).stop 2023-07-13T22:25:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 shutdown_connections 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f590c0a11a0 0x7f590c0a3650 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59241079c0 0x7f592419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5924072f70 0x7f5924197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 --2- 172.21.15.26:0/440246048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5924074550 0x7f5924197fd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 >> 172.21.15.26:0/440246048 conn(0x7f59240fb0e0 msgr2=0x7f59240fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 shutdown_connections 2023-07-13T22:25:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.054+0000 7f5929fff700 1 -- 172.21.15.26:0/440246048 wait complete. 2023-07-13T22:25:49.055 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:25:49.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:25:49.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 -- 172.21.15.26:0/3408094379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0100ce0 msgr2=0x7f7cc0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/3408094379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0101130 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009990 tx=0x7f7cb4009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 -- 172.21.15.26:0/3408094379 shutdown_connections 2023-07-13T22:25:49.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/3408094379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0101670 0x7f7cc0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/3408094379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 0x7f7cc00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/3408094379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0101130 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 -- 172.21.15.26:0/3408094379 >> 172.21.15.26:0/3408094379 conn(0x7f7cc00fb130 msgr2=0x7f7cc00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:49.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 -- 172.21.15.26:0/3408094379 shutdown_connections 2023-07-13T22:25:49.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.281+0000 7f7cc69ff700 1 -- 172.21.15.26:0/3408094379 wait complete. 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 Processor -- start 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 -- start start 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 0x7f7cc0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019f670 con 0x7f7cc0101670 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019f7b0 con 0x7f7cc0100ce0 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.282+0000 7f7cc69ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc019f940 con 0x7f7cc00ffaf0 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33428/0 (socket says 172.21.15.26:33428) 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 -- 172.21.15.26:0/2251766994 learned_addr learned my addr 172.21.15.26:0/2251766994 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cbffff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 0x7f7cc0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cbf7fe700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 -- 172.21.15.26:0/2251766994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 msgr2=0x7f7cc0197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 0x7f7cc0197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 -- 172.21.15.26:0/2251766994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0100ce0 msgr2=0x7f7cc0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 -- 172.21.15.26:0/2251766994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40095f0 con 0x7f7cc0101670 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc4f9c700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800ef30 tx=0x7f7cb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8018070 con 0x7f7cc0101670 2023-07-13T22:25:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cb800f040 con 0x7f7cc0101670 2023-07-13T22:25:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.283+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc019fbc0 con 0x7f7cc0101670 2023-07-13T22:25:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.284+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800cd00 con 0x7f7cc0101670 2023-07-13T22:25:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.284+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc01a0120 con 0x7f7cc0101670 2023-07-13T22:25:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.285+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc01921d0 con 0x7f7cc0101670 2023-07-13T22:25:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.287+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7cb8003a00 con 0x7f7cc0101670 2023-07-13T22:25:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.288+0000 7f7cbd7fa700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 0x7f7ca80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.288+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7cb80cc370 con 0x7f7cc0101670 2023-07-13T22:25:49.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.289+0000 7f7cbffff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 0x7f7ca80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.289+0000 7f7cbffff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 0x7f7ca80a3680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0100b40 tx=0x7f7cb0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.292+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7cb8095b00 con 0x7f7cc0101670 2023-07-13T22:25:49.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.497+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cc00611c0 con 0x7f7cc0101670 2023-07-13T22:25:49.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.497+0000 7f7cbd7fa700 1 -- 172.21.15.26:0/2251766994 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cb8099120 con 0x7f7cc0101670 2023-07-13T22:25:49.480 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:25:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 msgr2=0x7f7ca80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 0x7f7ca80a3680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0100b40 tx=0x7f7cb0009260 comp rx=0 tx=0).stop 2023-07-13T22:25:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 msgr2=0x7f7cc019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800ef30 tx=0x7f7cb800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 shutdown_connections 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f7ca80a11d0 0x7f7ca80a3680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cc0101670 0x7f7cc019f0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7cc00ffaf0 0x7f7cc0197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 --2- 172.21.15.26:0/2251766994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cc0100ce0 0x7f7cc0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 >> 172.21.15.26:0/2251766994 conn(0x7f7cc00fb130 msgr2=0x7f7cc0106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:49.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 shutdown_connections 2023-07-13T22:25:49.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.500+0000 7f7cc69ff700 1 -- 172.21.15.26:0/2251766994 wait complete. 2023-07-13T22:25:49.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:25:49.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 -- 172.21.15.26:0/19342186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 msgr2=0x7f85300729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 --2- 172.21.15.26:0/19342186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f85300729a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 -- 172.21.15.26:0/19342186 shutdown_connections 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 --2- 172.21.15.26:0/19342186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 0x7f8530109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 --2- 172.21.15.26:0/19342186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f85300729a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 --2- 172.21.15.26:0/19342186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f85300733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 -- 172.21.15.26:0/19342186 >> 172.21.15.26:0/19342186 conn(0x7f85300fb0e0 msgr2=0x7f85300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 -- 172.21.15.26:0/19342186 shutdown_connections 2023-07-13T22:25:49.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.727+0000 7f8534dff700 1 -- 172.21.15.26:0/19342186 wait complete. 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.728+0000 7f8534dff700 1 Processor -- start 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.728+0000 7f8534dff700 1 -- start start 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f8530197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 0x7f853019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853019f600 con 0x7f85301079a0 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853019f740 con 0x7f8530072f70 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f8534dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853019f8d0 con 0x7f8530074550 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52902/0 (socket says 172.21.15.26:52902) 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 -- 172.21.15.26:0/1726853690 learned_addr learned my addr 172.21.15.26:0/1726853690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852dd9b700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f8530197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852ed9d700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 0x7f853019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 -- 172.21.15.26:0/1726853690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 msgr2=0x7f8530197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f8530197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 -- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 msgr2=0x7f853019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 0x7f853019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 -- 172.21.15.26:0/1726853690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200095f0 con 0x7f8530072f70 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.729+0000 7f852e59c700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85200098e0 tx=0x7f8520004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.730+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852001d070 con 0x7f8530072f70 2023-07-13T22:25:49.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.730+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85200054f0 con 0x7f8530072f70 2023-07-13T22:25:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.730+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8520004bc0 con 0x7f8530072f70 2023-07-13T22:25:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.730+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853019fb50 con 0x7f8530072f70 2023-07-13T22:25:49.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.730+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853019ff10 con 0x7f8530072f70 2023-07-13T22:25:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.731+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8530106050 con 0x7f8530072f70 2023-07-13T22:25:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.733+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f852000fbc0 con 0x7f8530072f70 2023-07-13T22:25:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.734+0000 7f851f7fe700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 0x7f85180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.734+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f85200cdac0 con 0x7f8530072f70 2023-07-13T22:25:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.734+0000 7f852dd9b700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 0x7f85180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:49.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.735+0000 7f852dd9b700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 0x7f85180a3680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8530074270 tx=0x7f8524009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:49.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.736+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8520097250 con 0x7f8530072f70 2023-07-13T22:25:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.947+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f853002cac0 con 0x7f8530072f70 2023-07-13T22:25:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.948+0000 7f851f7fe700 1 -- 172.21.15.26:0/1726853690 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f852009a870 con 0x7f8530072f70 2023-07-13T22:25:49.930 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:25:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 msgr2=0x7f85180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 0x7f85180a3680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8530074270 tx=0x7f8524009260 comp rx=0 tx=0).stop 2023-07-13T22:25:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 msgr2=0x7f8530197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85200098e0 tx=0x7f8520004730 comp rx=0 tx=0).stop 2023-07-13T22:25:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 shutdown_connections 2023-07-13T22:25:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f85180a11d0 0x7f85180a3680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85301079a0 0x7f853019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8530072f70 0x7f8530197a80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 --2- 172.21.15.26:0/1726853690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8530074550 0x7f8530197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:49.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 >> 172.21.15.26:0/1726853690 conn(0x7f85300fb0e0 msgr2=0x7f85300fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:49.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 shutdown_connections 2023-07-13T22:25:49.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:49.950+0000 7f8534dff700 1 -- 172.21.15.26:0/1726853690 wait complete. 2023-07-13T22:25:49.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:25:50.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 -- 172.21.15.26:0/3670832869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100100cd0 msgr2=0x7f3100101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 --2- 172.21.15.26:0/3670832869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100100cd0 0x7f3100101120 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f30f40098f0 tx=0x7f30f4009c00 comp rx=0 tx=0).stop 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 -- 172.21.15.26:0/3670832869 shutdown_connections 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 --2- 172.21.15.26:0/3670832869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100101660 0x7f3100107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 --2- 172.21.15.26:0/3670832869 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f31000ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 --2- 172.21.15.26:0/3670832869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100100cd0 0x7f3100101120 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 -- 172.21.15.26:0/3670832869 >> 172.21.15.26:0/3670832869 conn(0x7f31000fb0e0 msgr2=0x7f31000fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 -- 172.21.15.26:0/3670832869 shutdown_connections 2023-07-13T22:25:50.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.175+0000 7f31051ff700 1 -- 172.21.15.26:0/3670832869 wait complete. 2023-07-13T22:25:50.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.176+0000 7f31051ff700 1 Processor -- start 2023-07-13T22:25:50.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 -- start start 2023-07-13T22:25:50.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 0x7f3100198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100101660 0x7f310019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019f640 con 0x7f3100100cd0 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019f780 con 0x7f3100101660 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f31051ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310019f910 con 0x7f31000ffae0 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fe59c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 0x7f3100198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:44650/0 (socket says 172.21.15.26:44650) 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 -- 172.21.15.26:0/940699281 learned_addr learned my addr 172.21.15.26:0/940699281 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30ff59e700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100101660 0x7f310019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 -- 172.21.15.26:0/940699281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100101660 msgr2=0x7f310019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100101660 0x7f310019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 -- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 msgr2=0x7f3100198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 0x7f3100198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fed9d700 1 -- 172.21.15.26:0/940699281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f4009520 con 0x7f31000ffae0 2023-07-13T22:25:50.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.177+0000 7f30fe59c700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 0x7f3100198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f30fed9d700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30f000e970 tx=0x7f30f000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0009790 con 0x7f31000ffae0 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30f000c7e0 con 0x7f31000ffae0 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0003c70 con 0x7f31000ffae0 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310019fbf0 con 0x7f31000ffae0 2023-07-13T22:25:50.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.178+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31001a00b0 con 0x7f31000ffae0 2023-07-13T22:25:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.179+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31001921c0 con 0x7f31000ffae0 2023-07-13T22:25:50.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.181+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f30f000cc50 con 0x7f31000ffae0 2023-07-13T22:25:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.183+0000 7f30e7fff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 0x7f30e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.183+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f30f0014070 con 0x7f31000ffae0 2023-07-13T22:25:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.183+0000 7f30fe59c700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 0x7f30e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.183+0000 7f30fe59c700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 0x7f30e80a3600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005e90 tx=0x7f30f4019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:50.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.186+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f30f00961f0 con 0x7f31000ffae0 2023-07-13T22:25:50.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:49 smithi117 ceph-mon[129074]: pgmap v673: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2023-07-13T22:25:50.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/440246048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:25:50.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:49 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2251766994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:25:50.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:49 smithi026 ceph-mon[95647]: pgmap v673: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2023-07-13T22:25:50.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/440246048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:25:50.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:49 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2251766994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:25:50.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:49 smithi121 ceph-mon[129187]: pgmap v673: 97 pgs: 97 active+clean; 4.5 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 222 MiB/s wr, 1.14k op/s 2023-07-13T22:25:50.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/440246048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:25:50.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:49 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2251766994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:25:50.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.395+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3100061260 con 0x7f31000ffae0 2023-07-13T22:25:50.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.396+0000 7f30e7fff700 1 -- 172.21.15.26:0/940699281 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30f0003dd0 con 0x7f31000ffae0 2023-07-13T22:25:50.378 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:25:50.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 msgr2=0x7f30e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 0x7f30e80a3600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005e90 tx=0x7f30f4019040 comp rx=0 tx=0).stop 2023-07-13T22:25:50.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 msgr2=0x7f3100197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30f000e970 tx=0x7f30f000ec80 comp rx=0 tx=0).stop 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 shutdown_connections 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3100101660 0x7f310019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31000ffae0 0x7f3100197ac0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f30e80a1150 0x7f30e80a3600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 --2- 172.21.15.26:0/940699281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3100100cd0 0x7f3100198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 >> 172.21.15.26:0/940699281 conn(0x7f31000fb0e0 msgr2=0x7f3100106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:50.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 shutdown_connections 2023-07-13T22:25:50.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.398+0000 7f31051ff700 1 -- 172.21.15.26:0/940699281 wait complete. 2023-07-13T22:25:50.399 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-13T22:25:50.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 -- 172.21.15.26:0/1111700917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 msgr2=0x7f94180fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 --2- 172.21.15.26:0/1111700917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f94180fff10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9408009990 tx=0x7f9408009ca0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 -- 172.21.15.26:0/1111700917 shutdown_connections 2023-07-13T22:25:50.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 --2- 172.21.15.26:0/1111700917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f94180fff10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 --2- 172.21.15.26:0/1111700917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 0x7f9418107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 --2- 172.21.15.26:0/1111700917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 0x7f9418101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 -- 172.21.15.26:0/1111700917 >> 172.21.15.26:0/1111700917 conn(0x7f94180fb0e0 msgr2=0x7f94180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 -- 172.21.15.26:0/1111700917 shutdown_connections 2023-07-13T22:25:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.622+0000 7f941e3ff700 1 -- 172.21.15.26:0/1111700917 wait complete. 2023-07-13T22:25:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 Processor -- start 2023-07-13T22:25:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 -- start start 2023-07-13T22:25:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 0x7f9418197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 0x7f941819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819f5f0 con 0x7f94180ffb00 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819f730 con 0x7f9418100cf0 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.623+0000 7f941e3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941819f8c0 con 0x7f9418101680 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33478/0 (socket says 172.21.15.26:33478) 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 -- 172.21.15.26:0/2759648860 learned_addr learned my addr 172.21.15.26:0/2759648860 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:25:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f940ffff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 0x7f9418197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f941c99c700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 0x7f941819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 -- 172.21.15.26:0/2759648860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 msgr2=0x7f941819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 0x7f941819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 -- 172.21.15.26:0/2759648860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 msgr2=0x7f9418197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 0x7f9418197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 -- 172.21.15.26:0/2759648860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94080095f0 con 0x7f94180ffb00 2023-07-13T22:25:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.624+0000 7f9417fff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9408005740 tx=0x7f9408005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.625+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940801d070 con 0x7f94180ffb00 2023-07-13T22:25:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.625+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9408021410 con 0x7f94180ffb00 2023-07-13T22:25:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.625+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941819fb40 con 0x7f94180ffb00 2023-07-13T22:25:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.625+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408017650 con 0x7f94180ffb00 2023-07-13T22:25:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.625+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941819ffc0 con 0x7f94180ffb00 2023-07-13T22:25:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.626+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94181921e0 con 0x7f94180ffb00 2023-07-13T22:25:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.628+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9408021580 con 0x7f94180ffb00 2023-07-13T22:25:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.629+0000 7f9415ffb700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 0x7f94040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:25:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.629+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f94080cdc50 con 0x7f94180ffb00 2023-07-13T22:25:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.629+0000 7f940ffff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 0x7f94040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:25:50.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.630+0000 7f940ffff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 0x7f94040a36d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9400009c60 tx=0x7f9400009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:25:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.633+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9408097470 con 0x7f94180ffb00 2023-07-13T22:25:50.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.902+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f941802c850 con 0x7f94180ffb00 2023-07-13T22:25:50.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.913+0000 7f9415ffb700 1 -- 172.21.15.26:0/2759648860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733243 (secure 0 0 0) 0x7f940809aa90 con 0x7f94180ffb00 2023-07-13T22:25:50.903 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:25:50.903 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 18, 2023-07-13T22:25:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "active_gid": 14148, 2023-07-13T22:25:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "active_name": "x", 2023-07-13T22:25:50.904 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addrs": { 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6800", 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 4053809718 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:6801", 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 4053809718 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "active_addr": "172.21.15.26:6801/4053809718", 2023-07-13T22:25:50.906 INFO:teuthology.orchestra.run.smithi026.stdout: "active_change": "2023-07-13T22:05:51.339378+0000", 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "active_mgr_features": 4540138322906710015, 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "standbys": [ 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24109, 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "z", 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540138322906710015, 2023-07-13T22:25:50.907 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:50.908 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:50.912 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:50.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:50.916 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.917 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.921 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-07-13T22:25:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-07-13T22:25:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-07-13T22:25:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-07-13T22:25:50.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.926 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-07-13T22:25:50.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:50.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-07-13T22:25:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-07-13T22:25:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:50.938 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:50.939 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-07-13T22:25:50.942 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:50.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-07-13T22:25:50.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-07-13T22:25:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:50.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-07-13T22:25:50.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-07-13T22:25:50.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-07-13T22:25:50.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:50.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:50.955 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.956 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.957 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.958 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:50.960 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.962 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.965 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-07-13T22:25:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-07-13T22:25:50.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-13T22:25:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.968 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-13T22:25:50.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:50.973 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.974 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-07-13T22:25:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-07-13T22:25:50.975 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-07-13T22:25:50.977 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.978 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:50.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_password": { 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_password", 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web password", 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_user": { 2023-07-13T22:25:50.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_user", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web user", 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-07-13T22:25:50.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-13T22:25:50.990 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:50.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:50.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:50.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:50.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:50.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.000 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-07-13T22:25:51.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-07-13T22:25:51.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-07-13T22:25:51.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.009 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-13T22:25:51.010 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.011 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.012 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-07-13T22:25:51.013 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.014 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-07-13T22:25:51.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-13T22:25:51.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.020 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-07-13T22:25:51.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-07-13T22:25:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-13T22:25:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.034 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-07-13T22:25:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-07-13T22:25:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.035 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-07-13T22:25:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-07-13T22:25:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-07-13T22:25:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-07-13T22:25:51.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-13T22:25:51.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.041 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-07-13T22:25:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-07-13T22:25:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-07-13T22:25:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-07-13T22:25:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-07-13T22:25:51.045 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-07-13T22:25:51.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.048 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.051 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.052 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-07-13T22:25:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-07-13T22:25:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-13T22:25:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-13T22:25:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.061 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.063 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.065 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.067 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.068 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-13T22:25:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-13T22:25:51.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-13T22:25:51.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.072 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-13T22:25:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-13T22:25:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-07-13T22:25:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-13T22:25:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.075 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-07-13T22:25:51.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-07-13T22:25:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-07-13T22:25:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-07-13T22:25:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-07-13T22:25:51.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-13T22:25:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.083 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-07-13T22:25:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-07-13T22:25:51.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-07-13T22:25:51.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.088 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-07-13T22:25:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-07-13T22:25:51.090 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-07-13T22:25:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-07-13T22:25:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-07-13T22:25:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-07-13T22:25:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.093 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-13T22:25:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-13T22:25:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_password": { 2023-07-13T22:25:51.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_password", 2023-07-13T22:25:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.098 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web password", 2023-07-13T22:25:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_user": { 2023-07-13T22:25:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_user", 2023-07-13T22:25:51.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.101 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.102 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web user", 2023-07-13T22:25:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.106 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-07-13T22:25:51.107 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.108 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-07-13T22:25:51.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.111 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.112 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.113 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.114 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.116 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-07-13T22:25:51.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-13T22:25:51.119 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.120 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.127 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.129 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.130 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.132 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-07-13T22:25:51.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-13T22:25:51.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.139 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.140 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-07-13T22:25:51.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.142 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.143 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.145 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.146 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.147 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.148 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-13T22:25:51.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.154 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.155 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-13T22:25:51.157 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.161 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-07-13T22:25:51.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.165 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-13T22:25:51.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.167 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.168 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.169 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.170 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.171 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.172 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-13T22:25:51.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.175 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.176 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.178 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.180 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-13T22:25:51.182 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.187 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-13T22:25:51.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.190 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.192 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.194 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.195 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-13T22:25:51.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-07-13T22:25:51.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.201 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-07-13T22:25:51.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.204 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.205 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.208 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.210 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.211 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-13T22:25:51.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.214 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.217 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-07-13T22:25:51.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-07-13T22:25:51.220 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.221 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.224 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.229 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.230 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-07-13T22:25:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-07-13T22:25:51.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.240 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-07-13T22:25:51.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.244 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.246 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.247 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.252 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.253 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.255 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.256 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.257 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.258 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.259 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.260 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-07-13T22:25:51.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-13T22:25:51.262 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.263 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.267 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-07-13T22:25:51.268 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.269 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.273 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.274 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.275 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.276 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.277 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.280 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.282 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.283 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.284 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-07-13T22:25:51.285 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-07-13T22:25:51.289 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-07-13T22:25:51.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-07-13T22:25:51.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-07-13T22:25:51.294 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.295 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.296 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.298 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.302 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-07-13T22:25:51.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.313 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-07-13T22:25:51.314 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.315 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.318 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.324 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-07-13T22:25:51.326 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.327 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.328 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.329 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.330 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.332 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.336 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-07-13T22:25:51.338 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.341 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.343 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.344 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.345 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.348 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.349 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-07-13T22:25:51.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.351 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.355 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.356 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.357 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.358 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.360 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.361 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-07-13T22:25:51.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-07-13T22:25:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.363 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.364 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.365 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-07-13T22:25:51.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.368 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-07-13T22:25:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.369 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-07-13T22:25:51.370 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-13T22:25:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-13T22:25:51.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.372 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-07-13T22:25:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.373 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.374 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.375 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.376 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.377 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.378 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.379 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.380 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.381 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.382 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-07-13T22:25:51.385 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.387 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.389 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.390 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.391 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.392 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.393 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.394 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-07-13T22:25:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.398 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.400 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.401 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.402 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.403 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.406 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.409 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.410 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-07-13T22:25:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-07-13T22:25:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.411 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.412 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.413 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.414 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.417 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.419 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.420 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.421 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.423 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-07-13T22:25:51.424 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.425 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.426 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-07-13T22:25:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.430 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.431 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.432 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.433 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.434 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.435 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.436 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.442 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.447 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.448 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.449 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.450 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.453 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-07-13T22:25:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.454 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.457 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.458 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.459 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.460 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.461 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.462 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.463 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.466 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "noautoscale": { 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "noautoscale", 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.467 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "global autoscale flag", 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-13T22:25:51.468 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.469 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.470 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-07-13T22:25:51.471 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.472 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-07-13T22:25:51.474 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-13T22:25:51.475 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.479 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.484 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.486 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-07-13T22:25:51.487 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.488 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:51.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.490 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.492 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.494 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.495 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.496 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.497 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.498 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.499 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.501 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.502 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.503 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.504 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.505 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.507 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.508 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.509 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-07-13T22:25:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.510 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-07-13T22:25:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-07-13T22:25:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.512 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.513 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.514 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-13T22:25:51.515 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:51.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-13T22:25:51.517 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.518 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.519 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-07-13T22:25:51.521 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:51.522 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.523 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.524 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.525 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.526 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.527 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.528 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.529 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.530 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.531 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.532 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.533 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.534 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.535 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.539 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.540 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.541 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.542 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-07-13T22:25:51.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.545 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-07-13T22:25:51.546 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.547 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:51.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.549 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.550 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.551 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.552 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.554 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.555 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.556 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.557 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.558 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.560 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.562 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.563 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:51.564 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.565 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.566 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.567 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.569 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.573 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.574 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.575 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-07-13T22:25:51.578 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-13T22:25:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.580 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.581 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.582 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.583 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.584 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.585 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.586 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.587 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.588 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.589 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-07-13T22:25:51.590 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.592 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.594 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.595 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.596 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.597 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.600 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.601 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.604 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.607 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-07-13T22:25:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-07-13T22:25:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.608 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.609 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-07-13T22:25:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-07-13T22:25:51.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.612 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-07-13T22:25:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.613 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-07-13T22:25:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-07-13T22:25:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.618 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.619 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.621 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.623 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-07-13T22:25:51.624 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.626 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.627 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.629 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.630 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.631 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.632 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-07-13T22:25:51.634 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-07-13T22:25:51.635 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.636 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.637 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.638 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.639 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.640 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.643 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.644 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.645 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.646 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.647 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.648 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.649 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.653 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.655 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.656 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.657 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.659 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.663 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.664 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.667 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-13T22:25:51.668 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.669 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.673 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.674 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.675 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.679 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-07-13T22:25:51.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-07-13T22:25:51.683 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.684 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-07-13T22:25:51.685 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-13T22:25:51.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.691 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.692 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.693 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-07-13T22:25:51.694 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.695 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:51.696 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.697 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.698 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.701 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.702 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.703 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.704 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.705 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.706 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.707 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.708 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.709 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.710 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.711 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.713 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.714 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.715 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.716 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.717 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-07-13T22:25:51.718 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.720 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.722 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.724 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.725 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.726 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.727 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.728 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.730 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.731 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.732 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.737 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.744 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-07-13T22:25:51.745 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.746 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.747 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:51.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.751 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.752 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.753 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.756 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-07-13T22:25:51.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.760 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.761 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.762 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.764 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 24116, 2023-07-13T22:25:51.765 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "y", 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr_features": 4540138322906710015, 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.766 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.768 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.770 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.774 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.775 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.776 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.777 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-07-13T22:25:51.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.780 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.781 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-07-13T22:25:51.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.786 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.787 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-07-13T22:25:51.788 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-13T22:25:51.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.790 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.791 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.792 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.793 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.794 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.795 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-07-13T22:25:51.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:51.798 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.801 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.802 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-07-13T22:25:51.804 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.805 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:51.806 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.807 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.808 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.809 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.811 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.812 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.813 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.814 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.815 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-07-13T22:25:51.816 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.817 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.818 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-07-13T22:25:51.819 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.820 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.821 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:51.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-13T22:25:51.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.824 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-13T22:25:51.825 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.828 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.829 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-07-13T22:25:51.830 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.831 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-07-13T22:25:51.832 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-13T22:25:51.833 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_password": { 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_password", 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.834 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web password", 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_user": { 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_user", 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.836 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web user", 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-07-13T22:25:51.839 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.840 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-07-13T22:25:51.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.843 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.844 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.845 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.846 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.847 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.848 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.849 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-07-13T22:25:51.850 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.852 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.853 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.855 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-13T22:25:51.856 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.857 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.858 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-07-13T22:25:51.859 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-07-13T22:25:51.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.861 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.862 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.865 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.866 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-13T22:25:51.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.868 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.869 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.870 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-07-13T22:25:51.872 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.873 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-07-13T22:25:51.874 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-07-13T22:25:51.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.876 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.877 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.881 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-07-13T22:25:51.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-07-13T22:25:51.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.889 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.890 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.891 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.896 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.897 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.898 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.900 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.901 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-07-13T22:25:51.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.904 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-07-13T22:25:51.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-07-13T22:25:51.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_password": { 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_password", 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web password", 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_user": { 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_user", 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web user", 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.916 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-07-13T22:25:51.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-13T22:25:51.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.924 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.925 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.926 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.927 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.928 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-07-13T22:25:51.929 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.930 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.931 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-07-13T22:25:51.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-07-13T22:25:51.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-13T22:25:51.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-07-13T22:25:51.939 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.940 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.941 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.942 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.944 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:51.946 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:51.947 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.948 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.949 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.950 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:51.951 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.952 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:51.953 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.955 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-13T22:25:51.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.958 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.959 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.960 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-13T22:25:51.961 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.968 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-13T22:25:51.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.979 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.982 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-07-13T22:25:51.983 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.984 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-13T22:25:51.985 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.991 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.992 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.993 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-13T22:25:51.994 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.995 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-13T22:25:51.996 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.997 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-13T22:25:51.998 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:51.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.007 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.008 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-13T22:25:52.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-13T22:25:52.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.009 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-13T22:25:52.010 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.011 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-13T22:25:52.012 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.013 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-13T22:25:52.014 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.017 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.020 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.021 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.022 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.023 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.025 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-07-13T22:25:52.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.029 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.030 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.031 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.032 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-13T22:25:52.033 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-07-13T22:25:52.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.036 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.037 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-07-13T22:25:52.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.042 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.045 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.046 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.047 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.048 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.049 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.050 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.052 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.053 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.054 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:52.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.059 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.060 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:52.061 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.062 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.064 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.065 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.066 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.067 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.068 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.069 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.071 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.072 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.073 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.075 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.076 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.077 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.078 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.079 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.080 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.081 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.083 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.084 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-07-13T22:25:52.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.087 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-07-13T22:25:52.088 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.089 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.090 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-13T22:25:52.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.095 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.096 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.098 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.099 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.100 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.101 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.102 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.103 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-07-13T22:25:52.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-07-13T22:25:52.108 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.109 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.110 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-07-13T22:25:52.112 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.113 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.114 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-07-13T22:25:52.115 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.116 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.119 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.120 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.121 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.122 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.123 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.124 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.125 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.126 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.127 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.128 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-07-13T22:25:52.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.130 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.131 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-07-13T22:25:52.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.136 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.137 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.138 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.139 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.140 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.141 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.142 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.143 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.144 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.147 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.148 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.149 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.151 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.152 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.153 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.154 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.155 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.157 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.160 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.161 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-07-13T22:25:52.162 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.163 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.164 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.165 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.167 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.168 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-07-13T22:25:52.175 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.177 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.178 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.179 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.180 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.182 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.183 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.184 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.184 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.185 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.186 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.187 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-07-13T22:25:52.188 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-07-13T22:25:52.189 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.191 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.197 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.199 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.200 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.201 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.202 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.203 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.204 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.205 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.206 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-07-13T22:25:52.208 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.210 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.211 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.212 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.214 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.215 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.216 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.217 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.218 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.219 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.220 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.225 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.226 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.228 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.229 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.229 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:52.229 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.229 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.232 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.233 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.235 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.235 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.235 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.235 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.237 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.238 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.239 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.240 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.241 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-07-13T22:25:52.243 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.244 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.245 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.246 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.251 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.252 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.253 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.254 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.255 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.257 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.258 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.259 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.260 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.262 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.263 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.266 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.267 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.269 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.269 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.269 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.271 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.272 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.273 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.274 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.275 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "noautoscale": { 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "noautoscale", 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.276 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "global autoscale flag", 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.278 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-07-13T22:25:52.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.280 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-07-13T22:25:52.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:52.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.286 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.287 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.290 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.291 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.292 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.293 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.296 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.297 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.298 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.299 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-07-13T22:25:52.300 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.301 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.302 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-13T22:25:52.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.306 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.307 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.308 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.309 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.310 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-07-13T22:25:52.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-07-13T22:25:52.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.324 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.325 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-07-13T22:25:52.326 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:52.327 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.328 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.329 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.331 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.333 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.334 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.337 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.339 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.340 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-07-13T22:25:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-07-13T22:25:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.341 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.343 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.344 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.345 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-07-13T22:25:52.349 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.350 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.354 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.355 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.356 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.357 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.358 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.359 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.360 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.361 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.362 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.363 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.364 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.365 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.367 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.368 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.369 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.373 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.374 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.376 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.377 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-07-13T22:25:52.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.379 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.380 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.381 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.385 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.386 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-07-13T22:25:52.392 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.393 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.394 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.398 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.399 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.402 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-07-13T22:25:52.404 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.405 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-07-13T22:25:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-07-13T22:25:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.424 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.425 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.426 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.429 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.430 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.431 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.434 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.435 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.436 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.437 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.440 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.441 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.442 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.443 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.444 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.445 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.446 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.450 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.452 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.453 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.454 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.456 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.457 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.458 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.459 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-07-13T22:25:52.460 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.463 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.464 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.465 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.466 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.467 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.468 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.469 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.470 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.471 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.473 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-13T22:25:52.476 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.477 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.478 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.479 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.481 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.483 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.484 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.485 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-07-13T22:25:52.486 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.487 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.488 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.489 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.490 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.491 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.492 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.493 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-07-13T22:25:52.494 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.496 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.497 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.498 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.499 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.500 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.501 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.503 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.503 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.503 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.504 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.504 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.504 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.505 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.505 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.505 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.505 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.506 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.507 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.507 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.507 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.507 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.508 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.508 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.508 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.509 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.509 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.509 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.510 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.510 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.510 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.510 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.511 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.511 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.511 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.511 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.512 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.512 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.512 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.513 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.513 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.513 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.513 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.514 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.514 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.514 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.514 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.515 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.515 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.515 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-07-13T22:25:52.516 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-07-13T22:25:52.516 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.516 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.516 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.517 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.517 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.517 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.517 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.518 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.518 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.518 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.518 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.519 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-07-13T22:25:52.519 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-07-13T22:25:52.520 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.520 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.520 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.520 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.521 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.521 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.521 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.521 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.522 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.522 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.522 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.523 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-07-13T22:25:52.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-07-13T22:25:52.523 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.523 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.524 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.524 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-13T22:25:52.524 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.524 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.525 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.525 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.525 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.525 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.526 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.526 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.526 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.527 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-07-13T22:25:52.527 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.527 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.528 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.528 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.528 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.529 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.529 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.529 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.529 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.530 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.530 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.530 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.530 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.531 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.531 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.531 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.531 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.532 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.532 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.532 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.532 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.533 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.533 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.533 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.534 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.534 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.534 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.534 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.535 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.535 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.535 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.535 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.536 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.536 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.536 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.536 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.537 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.537 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.537 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.537 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.538 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.538 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.538 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.538 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.539 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.539 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.539 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.540 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.540 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.540 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.540 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.541 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.542 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.542 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.542 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.542 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.543 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.543 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.543 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.543 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.544 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.544 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.544 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.544 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.545 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.545 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.545 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.545 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.546 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.546 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.546 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.547 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-07-13T22:25:52.547 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.547 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.547 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.548 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.548 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.548 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.548 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.549 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.549 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.549 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.550 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.550 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.550 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.550 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.551 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.551 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.551 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.551 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.552 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.552 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.552 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.552 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.553 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.553 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.553 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.554 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.554 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.554 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.554 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.555 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.555 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.555 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.555 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.556 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.556 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.556 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.557 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.558 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.558 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.558 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.558 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.559 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.559 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.559 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.559 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.560 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.560 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.560 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.561 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.561 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.561 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.561 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.562 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.562 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.563 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.563 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.563 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.563 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.564 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.564 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.564 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.564 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.565 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.565 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.565 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.566 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-07-13T22:25:52.566 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-07-13T22:25:52.566 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.566 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.567 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.567 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-07-13T22:25:52.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.567 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.568 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.568 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-13T22:25:52.568 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.568 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.569 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.569 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.569 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-07-13T22:25:52.569 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-07-13T22:25:52.570 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.570 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.570 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.570 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.571 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.571 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.571 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.571 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-13T22:25:52.572 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.572 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.572 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.573 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.574 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-07-13T22:25:52.574 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.574 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.574 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.575 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-07-13T22:25:52.575 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-07-13T22:25:52.575 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:52.575 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.576 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.577 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.578 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.579 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.580 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.581 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.582 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.583 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.584 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.585 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.586 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.587 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.588 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.589 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.590 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.591 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.592 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.593 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-13T22:25:52.594 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.595 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-07-13T22:25:52.596 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard", 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "available_modules": [ 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alerts", 2023-07-13T22:25:52.597 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.598 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How frequently to reexamine health status", 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.599 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.600 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.601 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.602 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.603 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.604 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.605 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.606 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.607 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.608 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_destination": { 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_destination", 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.609 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email address to send alerts to", 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_from_name": { 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_from_name", 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "Ceph", 2023-07-13T22:25:52.611 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Email From: name", 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.612 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_host": { 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_host", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.613 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP server", 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_password": { 2023-07-13T22:25:52.614 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_password", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.615 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Password to authenticate with", 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_port": { 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_port", 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.616 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.617 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.617 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "465", 2023-07-13T22:25:52.617 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.617 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.618 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.618 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP port", 2023-07-13T22:25:52.618 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.618 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.619 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.619 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_sender": { 2023-07-13T22:25:52.619 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_sender", 2023-07-13T22:25:52.620 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.620 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.620 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.620 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.621 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.621 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.621 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.621 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SMTP envelope sender", 2023-07-13T22:25:52.622 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.622 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.622 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.623 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_ssl": { 2023-07-13T22:25:52.623 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_ssl", 2023-07-13T22:25:52.623 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.623 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.624 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.624 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.624 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.625 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.625 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.625 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-13T22:25:52.625 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: "smtp_user": { 2023-07-13T22:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "smtp_user", 2023-07-13T22:25:52.627 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.628 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.628 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.628 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.628 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.629 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.629 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.629 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "User to authenticate as", 2023-07-13T22:25:52.629 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.630 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.630 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.630 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.630 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.631 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.631 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "balancer", 2023-07-13T22:25:52.631 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.632 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.632 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.632 INFO:teuthology.orchestra.run.smithi026.stdout: "active": { 2023-07-13T22:25:52.632 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "active", 2023-07-13T22:25:52.633 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.633 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.633 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.633 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically balance PGs across cluster", 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_time": { 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_time", 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0000", 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "beginning time of day to automatically balance", 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.636 INFO:teuthology.orchestra.run.smithi026.stdout: "begin_weekday": { 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "begin_weekday", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:52.637 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_max_iterations": { 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_max_iterations", 2023-07-13T22:25:52.638 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "25", 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "250", 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.639 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_metrics": { 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_metrics", 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.640 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "pgs,objects,bytes", 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-13T22:25:52.641 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "crush_compat_step": { 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crush_compat_step", 2023-07-13T22:25:52.642 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.643 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.5", 2023-07-13T22:25:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0.001", 2023-07-13T22:25:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "0.999", 2023-07-13T22:25:52.644 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "aggressiveness of optimization", 2023-07-13T22:25:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-13T22:25:52.645 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "end_time": { 2023-07-13T22:25:52.646 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_time", 2023-07-13T22:25:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2359", 2023-07-13T22:25:52.647 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ending time of day to automatically balance", 2023-07-13T22:25:52.648 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-13T22:25:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "end_weekday": { 2023-07-13T22:25:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "end_weekday", 2023-07-13T22:25:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:52.650 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "0", 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "6", 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.651 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.652 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.653 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.654 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.655 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.656 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.657 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.658 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.659 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.660 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.661 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "min_score": { 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_score", 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.662 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.663 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.664 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "upmap", 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.665 INFO:teuthology.orchestra.run.smithi026.stdout: "crush-compat", 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "none", 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap" 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Balancer mode", 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.666 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_ids": { 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_ids", 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:52.669 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.670 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_deviation": { 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_deviation", 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:52.671 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.672 INFO:teuthology.orchestra.run.smithi026.stdout: "upmap_max_optimizations": { 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "upmap_max_optimizations", 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.673 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephadm", 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_down_multiplier": { 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_down_multiplier", 2023-07-13T22:25:52.675 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.676 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.676 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.676 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:52.676 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.676 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_refresh_rate": { 2023-07-13T22:25:52.677 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_refresh_rate", 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "20", 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.678 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-13T22:25:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.680 INFO:teuthology.orchestra.run.smithi026.stdout: "agent_starting_port": { 2023-07-13T22:25:52.680 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "agent_starting_port", 2023-07-13T22:25:52.680 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.680 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4721", 2023-07-13T22:25:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-13T22:25:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.683 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.683 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_password": { 2023-07-13T22:25:52.683 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_password", 2023-07-13T22:25:52.684 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.684 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.684 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.684 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.685 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.685 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.685 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.686 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web password", 2023-07-13T22:25:52.686 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.686 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.686 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "alertmanager_web_user": { 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "alertmanager_web_user", 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.687 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Alertmanager web user", 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.688 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_ptrace": { 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_ptrace", 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.689 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.690 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_interval": { 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_interval", 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.691 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-13T22:25:52.692 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "autotune_memory_target_ratio": { 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "autotune_memory_target_ratio", 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:52.693 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0.7", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.694 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "cgroups_split": { 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cgroups_split", 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.695 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.696 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: "config_checks_enabled": { 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_checks_enabled", 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.697 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.698 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "config_dashboard": { 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_dashboard", 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.699 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.700 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_alertmanager": { 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_alertmanager", 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.701 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.702 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_base": { 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_base", 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-13T22:25:52.703 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Container image name, without the tag", 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.704 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_elasticsearch": { 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_elasticsearch", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "elasticsearch container image", 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_grafana": { 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_grafana", 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-13T22:25:52.707 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.708 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_haproxy": { 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_haproxy", 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.709 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "HAproxy container image", 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.710 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_agent": { 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_agent", 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.711 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger agent container image", 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.712 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.713 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.713 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_collector": { 2023-07-13T22:25:52.713 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_collector", 2023-07-13T22:25:52.713 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.713 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.714 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.714 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-13T22:25:52.714 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.715 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.715 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.715 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger collector container image", 2023-07-13T22:25:52.715 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.716 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.716 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.716 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.716 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_jaeger_query": { 2023-07-13T22:25:52.717 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_jaeger_query", 2023-07-13T22:25:52.717 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.717 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.717 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.718 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-13T22:25:52.718 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.718 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.718 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Jaeger query container image", 2023-07-13T22:25:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.719 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.720 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_keepalived": { 2023-07-13T22:25:52.721 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_keepalived", 2023-07-13T22:25:52.721 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.721 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.721 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.722 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-13T22:25:52.722 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.722 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.723 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.723 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Keepalived container image", 2023-07-13T22:25:52.723 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.723 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.724 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.724 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.724 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_loki": { 2023-07-13T22:25:52.724 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_loki", 2023-07-13T22:25:52.724 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.725 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.725 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.725 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-13T22:25:52.725 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.726 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.726 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.726 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Loki container image", 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_node_exporter": { 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_node_exporter", 2023-07-13T22:25:52.727 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-13T22:25:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.728 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_prometheus": { 2023-07-13T22:25:52.729 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_prometheus", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.730 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus container image", 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_promtail": { 2023-07-13T22:25:52.731 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_promtail", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.732 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Promtail container image", 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "container_image_snmp_gateway": { 2023-07-13T22:25:52.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_image_snmp_gateway", 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.734 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "SNMP Gateway container image", 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "container_init": { 2023-07-13T22:25:52.735 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "container_init", 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.736 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Run podman/docker with `--init`", 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "daemon_cache_timeout": { 2023-07-13T22:25:52.737 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "daemon_cache_timeout", 2023-07-13T22:25:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:52.738 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.739 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_cephadm_command_timeout": { 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_cephadm_command_timeout", 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "900", 2023-07-13T22:25:52.740 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.741 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_registry": { 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "default_registry", 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "docker.io", 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.742 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.743 INFO:teuthology.orchestra.run.smithi026.stdout: "device_cache_timeout": { 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_cache_timeout", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1800", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.744 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache device inventory", 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "device_enhanced_scan": { 2023-07-13T22:25:52.745 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_enhanced_scan", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.746 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-13T22:25:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.747 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "facts_cache_timeout": { 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "facts_cache_timeout", 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.748 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "seconds to cache host facts data", 2023-07-13T22:25:52.749 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "host_check_interval": { 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "host_check_interval", 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to perform a host check", 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.751 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.752 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.753 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.754 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "log_refresh_metadata": { 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_refresh_metadata", 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.755 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.756 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.757 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.759 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.760 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.761 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.762 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.763 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-13T22:25:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-13T22:25:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.764 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.765 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.766 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "*", 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.767 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "max_count_per_host": { 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_count_per_host", 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.768 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of daemons per service per host", 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.769 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "max_osd_draining_count": { 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_osd_draining_count", 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.770 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.771 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "migration_current": { 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "migration_current", 2023-07-13T22:25:52.772 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "internal - do not modify", 2023-07-13T22:25:52.773 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mode": { 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mode", 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.774 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "root", 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.775 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm-package", 2023-07-13T22:25:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "root" 2023-07-13T22:25:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.776 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "mode for remote execution of cephadm", 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_alerts_path": { 2023-07-13T22:25:52.777 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_alerts_path", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.778 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_password": { 2023-07-13T22:25:52.779 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_password", 2023-07-13T22:25:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.780 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web password", 2023-07-13T22:25:52.781 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.782 INFO:teuthology.orchestra.run.smithi026.stdout: "prometheus_web_user": { 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus_web_user", 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.783 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Prometheus web user", 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_insecure": { 2023-07-13T22:25:52.784 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_insecure", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.785 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-13T22:25:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.786 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_password": { 2023-07-13T22:25:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_password", 2023-07-13T22:25:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.787 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.788 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.789 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_url": { 2023-07-13T22:25:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_url", 2023-07-13T22:25:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.790 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-13T22:25:52.791 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "registry_username": { 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "registry_username", 2023-07-13T22:25:52.792 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.793 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.794 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-13T22:25:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.795 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "secure_monitoring_stack": { 2023-07-13T22:25:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "secure_monitoring_stack", 2023-07-13T22:25:52.796 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.797 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-13T22:25:52.798 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.799 INFO:teuthology.orchestra.run.smithi026.stdout: "service_discovery_port": { 2023-07-13T22:25:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "service_discovery_port", 2023-07-13T22:25:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.800 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8765", 2023-07-13T22:25:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.801 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "cephadm service discovery port", 2023-07-13T22:25:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.802 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "ssh_config_file": { 2023-07-13T22:25:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssh_config_file", 2023-07-13T22:25:52.803 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.804 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-13T22:25:52.805 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "use_agent": { 2023-07-13T22:25:52.806 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_agent", 2023-07-13T22:25:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.807 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.808 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-13T22:25:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "use_repo_digest": { 2023-07-13T22:25:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "use_repo_digest", 2023-07-13T22:25:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.810 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.811 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-13T22:25:52.812 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.813 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_failed_host_check": { 2023-07-13T22:25:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_failed_host_check", 2023-07-13T22:25:52.814 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.815 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.816 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if the host check fails", 2023-07-13T22:25:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.817 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_daemons": { 2023-07-13T22:25:52.818 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_daemons", 2023-07-13T22:25:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.819 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.820 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.821 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_on_stray_hosts": { 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_on_stray_hosts", 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.822 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-13T22:25:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.824 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crash", 2023-07-13T22:25:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.826 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.827 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.828 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.829 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.831 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.833 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.834 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.835 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.836 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.837 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.838 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.839 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "retain_interval": { 2023-07-13T22:25:52.840 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retain_interval", 2023-07-13T22:25:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.841 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "31536000", 2023-07-13T22:25:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-13T22:25:52.842 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.843 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_recent_interval": { 2023-07-13T22:25:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_recent_interval", 2023-07-13T22:25:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.844 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1209600", 2023-07-13T22:25:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.845 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-13T22:25:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.846 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dashboard", 2023-07-13T22:25:52.848 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-13T22:25:52.849 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-13T22:25:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.850 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.851 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.852 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_HOST": { 2023-07-13T22:25:52.853 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-13T22:25:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.854 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.855 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.856 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.857 INFO:teuthology.orchestra.run.smithi026.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-13T22:25:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-13T22:25:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.858 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.860 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.861 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_ENABLED": { 2023-07-13T22:25:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_ENABLED", 2023-07-13T22:25:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.862 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.863 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.864 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.865 INFO:teuthology.orchestra.run.smithi026.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-13T22:25:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-13T22:25:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.866 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.867 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.868 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "ENABLE_BROWSABLE_API": { 2023-07-13T22:25:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-13T22:25:52.869 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.870 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.871 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.872 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-13T22:25:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-13T22:25:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.873 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.874 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.875 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.876 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-13T22:25:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-13T22:25:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.877 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.878 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.879 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.880 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.881 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.882 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.883 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.884 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.885 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.886 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.887 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.888 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.889 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.890 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_PASSWORD": { 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-13T22:25:52.891 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.892 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.893 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.894 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_URL": { 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_URL", 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.895 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.896 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_API_USERNAME": { 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.897 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.899 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.900 INFO:teuthology.orchestra.run.smithi026.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.901 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-13T22:25:52.902 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.903 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-13T22:25:52.904 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.905 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_HOST": { 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.906 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.907 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-13T22:25:52.908 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.909 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.910 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.911 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-13T22:25:52.912 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.913 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-13T22:25:52.914 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.915 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-13T22:25:52.916 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.917 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.918 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.919 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.920 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.921 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.922 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.923 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_ENABLED": { 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.924 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.925 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.926 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.927 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.928 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.929 INFO:teuthology.orchestra.run.smithi026.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8", 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.930 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-13T22:25:52.931 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "45", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.932 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ACCESS_KEY": { 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-13T22:25:52.933 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.934 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.935 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "admin", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SECRET_KEY": { 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.937 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.938 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "RGW_API_SSL_VERIFY": { 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.939 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:52.941 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.942 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.943 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-13T22:25:52.944 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.945 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "cross_origin_url": { 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cross_origin_url", 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.946 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.947 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "crt_file": { 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "crt_file", 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.948 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "debug": { 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "debug", 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.950 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Enable/disable debug options", 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.951 INFO:teuthology.orchestra.run.smithi026.stdout: "jwt_token_ttl": { 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "jwt_token_ttl", 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "28800", 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.952 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:52.953 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.954 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.955 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.956 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.957 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.958 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.959 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.960 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.961 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.962 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.963 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "motd": { 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "motd", 2023-07-13T22:25:52.964 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.965 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "The message of the day", 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect_resolve_ip_addr": { 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "redirect_resolve_ip_addr", 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.966 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.967 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.968 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.969 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.970 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8080", 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.971 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.972 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl_server_port": { 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl_server_port", 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8443", 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.974 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:52.975 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "redirect", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.976 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "redirect" 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:52.978 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "url_prefix": { 2023-07-13T22:25:52.979 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url_prefix", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.980 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:52.981 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_monitoring": { 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_monitoring", 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.982 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "monitor device health metrics", 2023-07-13T22:25:52.983 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.984 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.985 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.987 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.989 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:52.990 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:52.991 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.992 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "mark_out_threshold": { 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mark_out_threshold", 2023-07-13T22:25:52.993 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "2419200", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-13T22:25:52.994 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: "pool_name": { 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pool_name", 2023-07-13T22:25:52.995 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "device_health_metrics", 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.996 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "retention_period": { 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "retention_period", 2023-07-13T22:25:52.997 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15552000", 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:52.998 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long to retain device health metrics", 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_frequency": { 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_frequency", 2023-07-13T22:25:52.999 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.000 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "self_heal": { 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "self_heal", 2023-07-13T22:25:53.001 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-13T22:25:53.002 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.003 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how frequently to wake up and check device health", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.004 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "warn_threshold": { 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "warn_threshold", 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.005 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7257600", 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.006 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "diskprediction_local", 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.007 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.008 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.009 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.010 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.011 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.012 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.013 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.014 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.015 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.016 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "predict_interval": { 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predict_interval", 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.017 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "86400", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.018 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "predictor_model": { 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "predictor_model", 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.019 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "prophetstor", 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.020 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "600", 2023-07-13T22:25:53.021 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "influx", 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": false, 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "influxdb python module not found", 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "batch_size": { 2023-07-13T22:25:53.023 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "batch_size", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5000", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.024 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "database": { 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "database", 2023-07-13T22:25:53.025 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "ceph", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-13T22:25:53.026 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": { 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "hostname", 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.027 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server hostname", 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.028 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.029 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "30", 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "5", 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.030 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.031 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.032 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.033 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.034 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.035 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.036 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.037 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.038 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.039 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "password": { 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "password", 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.040 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "password of InfluxDB server user", 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "port": { 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "port", 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "8086", 2023-07-13T22:25:53.042 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "InfluxDB server port", 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.043 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "ssl": { 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ssl", 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.044 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "false", 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "threads": { 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threads", 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:53.046 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "32", 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.047 INFO:teuthology.orchestra.run.smithi026.stdout: "username": { 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "username", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.048 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "username of InfluxDB server user", 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "verify_ssl": { 2023-07-13T22:25:53.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "verify_ssl", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "true", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.050 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.051 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "insights", 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.052 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.053 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.054 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.055 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.056 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.057 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.058 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.059 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.060 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.061 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "iostat", 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.062 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.063 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.064 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.065 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.066 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.067 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.068 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.069 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.070 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.071 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "k8sevents", 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.072 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_event_retention_days": { 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ceph_event_retention_days", 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.073 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "7", 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.074 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "config_check_secs": { 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "config_check_secs", 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:53.075 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "10", 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.077 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.078 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.079 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.080 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.081 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.082 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.083 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.084 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.085 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.086 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "localpool", 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "failure_domain": { 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "failure_domain", 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.087 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "host", 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "failure domain for any created local pool", 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-13T22:25:53.088 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.089 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.090 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.091 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.092 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.093 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.094 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.095 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.096 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.097 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "min_size": { 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "min_size", 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.098 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default min_size for any created local pool", 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.099 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "num_rep": { 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "num_rep", 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.100 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3", 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default replica count for any created local pool", 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.101 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_num": { 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_num", 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "128", 2023-07-13T22:25:53.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "default pg_num for any created local pool", 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.103 INFO:teuthology.orchestra.run.smithi026.stdout: "prefix": { 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prefix", 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "name prefix for any created local pool", 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.105 INFO:teuthology.orchestra.run.smithi026.stdout: "subtree": { 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subtree", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "rack", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.106 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.107 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mds_autoscaler", 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.108 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.109 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.110 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.111 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.112 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.113 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.114 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.115 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.116 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.117 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirroring", 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.118 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.119 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.120 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.121 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.122 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.123 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.124 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.125 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.126 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.127 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.128 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.128 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.128 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.128 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "nfs", 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.129 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.130 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.130 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.130 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.130 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.130 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.131 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.132 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.133 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.134 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.135 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.136 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.137 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.138 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.139 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "fail_fs": { 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "fail_fs", 2023-07-13T22:25:53.140 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-13T22:25:53.141 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.142 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.143 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.144 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.145 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.146 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.147 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.148 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.149 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.150 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.151 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator": { 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "orchestrator", 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.152 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "cephadm", 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "rook", 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: "test_orchestrator" 2023-07-13T22:25:53.153 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Orchestrator backend", 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_perf_query", 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.155 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.156 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.157 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.158 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.159 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.160 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.161 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.162 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.163 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.164 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "osd_support", 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.165 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.166 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.167 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.168 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.169 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.171 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.172 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.173 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.174 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "pg_autoscaler", 2023-07-13T22:25:53.175 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.176 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.177 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.180 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.181 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.182 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.183 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.184 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "noautoscale": { 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "noautoscale", 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.185 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "global autoscale flag", 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-13T22:25:53.186 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.187 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.188 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "threshold": { 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "threshold", 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "3.0", 2023-07-13T22:25:53.189 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1.0", 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "scaling threshold", 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "progress", 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_pg_recovery_event": { 2023-07-13T22:25:53.191 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_pg_recovery_event", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.192 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.194 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.195 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.196 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.197 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.198 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.199 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.200 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.201 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "max_completed_events": { 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_completed_events", 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.204 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "50", 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "number of past completed events to remember", 2023-07-13T22:25:53.205 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "sleep_interval": { 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sleep_interval", 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.206 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "5", 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "how long the module is going to sleep", 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.207 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "prometheus", 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.208 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "cache": { 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cache", 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.209 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.210 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "exclude_perf_counters": { 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "exclude_perf_counters", 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.211 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.212 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.213 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.214 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.215 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.216 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.217 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.218 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.219 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.220 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.221 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools": { 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools", 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.222 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.223 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300", 2023-07-13T22:25:53.224 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.225 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "scrape_interval": { 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "scrape_interval", 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15.0", 2023-07-13T22:25:53.226 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.227 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "::", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.228 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.229 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "9283", 2023-07-13T22:25:53.230 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.231 INFO:teuthology.orchestra.run.smithi026.stdout: "stale_cache_strategy": { 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stale_cache_strategy", 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "log", 2023-07-13T22:25:53.232 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_behaviour": { 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_behaviour", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "default", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.234 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "default", 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "error" 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.235 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "standby_error_status_code": { 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "standby_error_status_code", 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "500", 2023-07-13T22:25:53.236 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "400", 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "599", 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.238 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.239 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.240 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.241 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.242 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.243 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.244 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.245 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.246 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.247 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_snap_create": { 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_snap_create", 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10", 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.248 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "mirror_snapshot_schedule": { 2023-07-13T22:25:53.249 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "mirror_snapshot_schedule", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.250 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "trash_purge_schedule": { 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "trash_purge_schedule", 2023-07-13T22:25:53.251 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.252 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "restful", 2023-07-13T22:25:53.253 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "enable_auth": { 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enable_auth", 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.254 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.255 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "key_file": { 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "key_file", 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.256 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.257 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.258 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.259 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.260 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.261 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.262 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.263 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.264 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.265 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.266 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "server_addr": { 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_addr", 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.267 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.268 INFO:teuthology.orchestra.run.smithi026.stdout: "server_port": { 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "server_port", 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.269 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.270 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rgw", 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.271 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.272 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.273 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.274 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.275 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.276 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.277 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.278 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.279 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.280 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rook", 2023-07-13T22:25:53.281 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "drive_group_interval": { 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "drive_group_interval", 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.282 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "300.0", 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.283 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.284 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.285 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.286 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.288 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.289 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.290 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.291 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.292 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "storage_class": { 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "storage_class", 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "local", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "selftest", 2023-07-13T22:25:53.295 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.296 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.297 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.298 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.299 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.300 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.301 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.302 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.303 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.304 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "roption1": { 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption1", 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.305 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.306 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "roption2": { 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "roption2", 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.307 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "xyz", 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.308 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption1": { 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption1", 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.309 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.310 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption2": { 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption2", 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.311 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption3": { 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption3", 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "float", 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.313 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.314 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption4": { 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption4", 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.315 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption5": { 2023-07-13T22:25:53.316 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption5", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption6": { 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption6", 2023-07-13T22:25:53.318 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.319 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "rwoption7": { 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rwoption7", 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.320 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "1", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "42", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.321 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "testkey": { 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testkey", 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.322 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.323 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "testlkey": { 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testlkey", 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.324 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "testnewline": { 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "testnewline", 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.326 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snap_schedule", 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "allow_m_granularity": { 2023-07-13T22:25:53.328 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "allow_m_granularity", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.329 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "allow minute scheduled snapshots", 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "dump_on_update": { 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "dump_on_update", 2023-07-13T22:25:53.330 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.331 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "dump database to debug log on update", 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.332 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.333 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.334 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.335 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.336 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.337 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.338 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.339 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.340 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.341 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.342 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "stats", 2023-07-13T22:25:53.342 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.343 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.344 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.345 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.345 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.345 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.345 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.345 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.346 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.347 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.348 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.349 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.350 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.351 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.352 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "status", 2023-07-13T22:25:53.353 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.354 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.356 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.357 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.358 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.359 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.360 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.361 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.362 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.363 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telegraf", 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.364 INFO:teuthology.orchestra.run.smithi026.stdout: "address": { 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "address", 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.365 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "15", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.367 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.368 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.369 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.370 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.371 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.372 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.373 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.374 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.375 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.376 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.377 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "telemetry", 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.378 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_basic": { 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_basic", 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.380 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_crash": { 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_crash", 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.381 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_device": { 2023-07-13T22:25:53.382 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_device", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "True", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.383 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_ident": { 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_ident", 2023-07-13T22:25:53.384 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-13T22:25:53.385 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "channel_perf": { 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "channel_perf", 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.386 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.387 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "contact": { 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "contact", 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.388 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.389 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "description": { 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "description", 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.391 INFO:teuthology.orchestra.run.smithi026.stdout: "device_url": { 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "device_url", 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.392 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.393 INFO:teuthology.orchestra.run.smithi026.stdout: "enabled": { 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "enabled", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.394 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:53.395 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "24", 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "8", 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.396 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "last_opt_revision": { 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "last_opt_revision", 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.397 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "1", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.398 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard": { 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard", 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.399 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.400 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "leaderboard_description": { 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "leaderboard_description", 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.401 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.402 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.403 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.404 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.405 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.406 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.407 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.408 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.409 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.410 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.411 INFO:teuthology.orchestra.run.smithi026.stdout: "organization": { 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "organization", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.412 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "proxy": { 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "proxy", 2023-07-13T22:25:53.413 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.414 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "url": { 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "url", 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.415 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.416 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "test_orchestrator", 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.417 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.418 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.419 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.420 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.421 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.422 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.423 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.424 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.425 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.426 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-07-13T22:25:53.427 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.428 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.429 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.431 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.432 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.433 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.434 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.435 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.436 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "max_concurrent_clones": { 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "max_concurrent_clones", 2023-07-13T22:25:53.437 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "4", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-13T22:25:53.438 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot_clone_delay": { 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "snapshot_clone_delay", 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.439 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "0", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.440 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix", 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "can_run": true, 2023-07-13T22:25:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: "error_string": "", 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "module_options": { 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "discovery_interval": { 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "discovery_interval", 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "uint", 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "100", 2023-07-13T22:25:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "identifier": { 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "identifier", 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.444 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.445 INFO:teuthology.orchestra.run.smithi026.stdout: "interval": { 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "interval", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "secs", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "60", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.446 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "log_level": { 2023-07-13T22:25:53.447 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_level", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.448 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.449 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster": { 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster", 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.450 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.451 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_cluster_level": { 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_cluster_level", 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.452 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "info", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [ 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "critical", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "debug", 2023-07-13T22:25:53.453 INFO:teuthology.orchestra.run.smithi026.stdout: "error", 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "info", 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "warning" 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.454 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "log_to_file": { 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "log_to_file", 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "bool", 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 1, 2023-07-13T22:25:53.455 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "False", 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.456 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_host": { 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_host", 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "", 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.457 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.458 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_port": { 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_port", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "int", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "10051", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.459 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "zabbix_sender": { 2023-07-13T22:25:53.460 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "zabbix_sender", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "str", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "level": "advanced", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "flags": 0, 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "min": "", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "max": "", 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "enum_allowed": [], 2023-07-13T22:25:53.461 INFO:teuthology.orchestra.run.smithi026.stdout: "desc": "", 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "long_desc": "", 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "tags": [], 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "see_also": [] 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.462 INFO:teuthology.orchestra.run.smithi026.stdout: "services": { 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "dashboard": "https://172.21.15.26:8443/" 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "always_on_modules": { 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "octopus": [ 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-07-13T22:25:53.463 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-07-13T22:25:53.464 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "pacific": [ 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-07-13T22:25:53.465 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "quincy": [ 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-07-13T22:25:53.466 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-07-13T22:25:53.467 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "reef": [ 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "balancer", 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "crash", 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "devicehealth", 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "orchestrator", 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "pg_autoscaler", 2023-07-13T22:25:53.468 INFO:teuthology.orchestra.run.smithi026.stdout: "progress", 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "rbd_support", 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "status", 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "telemetry", 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "volumes" 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.469 INFO:teuthology.orchestra.run.smithi026.stdout: "last_failure_osd_epoch": 3, 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "active_clients": [ 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "devicehealth", 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-07-13T22:25:53.470 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3478184650 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "libcephsqlite", 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-07-13T22:25:53.471 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3406498384 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.472 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "rbd_support", 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-07-13T22:25:53.473 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1918471153 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "volumes", 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: "addrvec": [ 2023-07-13T22:25:53.474 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v2", 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "172.21.15.26:0", 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 935946693 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-07-13T22:25:53.475 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:53.477 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:50 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1726853690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.477 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:50 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/940699281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.477 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:51 smithi121 ceph-mon[129187]: pgmap v674: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 156 MiB/s wr, 1.04k op/s 2023-07-13T22:25:53.477 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:51 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2759648860' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-13T22:25:53.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:50 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1726853690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:50 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/940699281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:51 smithi117 ceph-mon[129074]: pgmap v674: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 156 MiB/s wr, 1.04k op/s 2023-07-13T22:25:53.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:51 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2759648860' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 msgr2=0x7f94040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 0x7f94040a36d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9400009c60 tx=0x7f9400009260 comp rx=0 tx=0).stop 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 msgr2=0x7f9418197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9408005740 tx=0x7f9408005770 comp rx=0 tx=0).stop 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 shutdown_connections 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f94040a1220 0x7f94040a36d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:53.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94180ffb00 0x7f9418197a70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9418101680 0x7f941819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 --2- 172.21.15.26:0/2759648860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9418100cf0 0x7f9418197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:25:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 >> 172.21.15.26:0/2759648860 conn(0x7f94180fb0e0 msgr2=0x7f94181065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:25:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 shutdown_connections 2023-07-13T22:25:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:25:50.920+0000 7f941e3ff700 1 -- 172.21.15.26:0/2759648860 wait complete. 2023-07-13T22:25:53.488 DEBUG:tasks.check_counter:Getting stats from a 2023-07-13T22:25:53.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-13T22:25:53.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:50 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1726853690' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:50 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/940699281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:25:53.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:51 smithi026 ceph-mon[95647]: pgmap v674: 97 pgs: 97 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 156 MiB/s wr, 1.04k op/s 2023-07-13T22:25:53.490 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:51 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2759648860' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-13T22:25:54.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:53 smithi117 ceph-mon[129074]: pgmap v675: 97 pgs: 97 active+clean; 50 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 101 MiB/s wr, 882 op/s 2023-07-13T22:25:54.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:53 smithi026 ceph-mon[95647]: pgmap v675: 97 pgs: 97 active+clean; 50 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 101 MiB/s wr, 882 op/s 2023-07-13T22:25:54.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:53 smithi121 ceph-mon[129187]: pgmap v675: 97 pgs: 97 active+clean; 50 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 101 MiB/s wr, 882 op/s 2023-07-13T22:25:54.236 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:25:55.929 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:55.929 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 126, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 130, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 54719, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 420828, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 51, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 6, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.063153813, 2023-07-13T22:25:55.930 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.007102404, 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.042044782, 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000365267, 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 125, 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.078123866, 2023-07-13T22:25:55.931 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000624990 2023-07-13T22:25:55.932 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.932 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:55.932 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 165, 2023-07-13T22:25:55.932 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000071207, 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000431 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:25:55.933 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 743, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 893, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 688239, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 292508, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 7, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2023-07-13T22:25:55.934 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.150656717, 2023-07-13T22:25:55.935 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.056188092, 2023-07-13T22:25:55.935 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.083079681, 2023-07-13T22:25:55.935 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.017145174, 2023-07-13T22:25:55.936 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:55.936 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 896, 2023-07-13T22:25:55.936 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.502146100, 2023-07-13T22:25:55.936 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000560430 2023-07-13T22:25:55.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.937 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:55.937 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1071, 2023-07-13T22:25:55.938 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001801991, 2023-07-13T22:25:55.938 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000001682 2023-07-13T22:25:55.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.938 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 540144, 2023-07-13T22:25:55.939 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 132048 2023-07-13T22:25:55.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.939 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:25:55.940 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 74, 2023-07-13T22:25:55.940 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 242, 2023-07-13T22:25:55.940 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 44223, 2023-07-13T22:25:55.941 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 417567, 2023-07-13T22:25:55.941 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 8, 2023-07-13T22:25:55.941 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 5, 2023-07-13T22:25:55.941 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.032321365, 2023-07-13T22:25:55.942 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.019771113, 2023-07-13T22:25:55.942 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.011121432, 2023-07-13T22:25:55.942 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000204332, 2023-07-13T22:25:55.942 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:55.942 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 240, 2023-07-13T22:25:55.943 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.133906454, 2023-07-13T22:25:55.943 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000557943 2023-07-13T22:25:55.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.943 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:55.944 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 122, 2023-07-13T22:25:55.944 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000033669, 2023-07-13T22:25:55.944 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000275 2023-07-13T22:25:55.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.945 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:25:55.945 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 367088 2023-07-13T22:25:55.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-07-13T22:25:55.946 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 35, 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001889574, 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000053987 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-07-13T22:25:55.947 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 2, 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000027896, 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000013948 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-07-13T22:25:55.948 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-07-13T22:25:55.949 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2023-07-13T22:25:55.950 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2023-07-13T22:25:55.950 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-07-13T22:25:55.950 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 11, 2023-07-13T22:25:55.950 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2023-07-13T22:25:55.951 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-07-13T22:25:55.951 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 10, 2023-07-13T22:25:55.951 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 0, 2023-07-13T22:25:55.951 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 10, 2023-07-13T22:25:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2023-07-13T22:25:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 10, 2023-07-13T22:25:55.952 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2023-07-13T22:25:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2023-07-13T22:25:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2023-07-13T22:25:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2023-07-13T22:25:55.953 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2023-07-13T22:25:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2023-07-13T22:25:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2023-07-13T22:25:55.954 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2023-07-13T22:25:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:25:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2023-07-13T22:25:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2023-07-13T22:25:55.955 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 1000, 2023-07-13T22:25:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 1, 2023-07-13T22:25:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2023-07-13T22:25:55.956 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2023-07-13T22:25:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2023-07-13T22:25:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2023-07-13T22:25:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-07-13T22:25:55.957 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:25:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2023-07-13T22:25:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 21, 2023-07-13T22:25:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 10738140632, 2023-07-13T22:25:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2023-07-13T22:25:55.958 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:25:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-07-13T22:25:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:25:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:25:55.959 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:25:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:25:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:25:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2023-07-13T22:25:55.960 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:25:55.961 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2023-07-13T22:25:55.962 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 1, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2023-07-13T22:25:55.963 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-07-13T22:25:55.964 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-07-13T22:25:55.965 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 1, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 1, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 1, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-07-13T22:25:55.966 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 1, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 4194844, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 0, 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-07-13T22:25:55.967 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1, 2023-07-13T22:25:55.968 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001648972, 2023-07-13T22:25:55.968 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.001648972 2023-07-13T22:25:55.968 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.968 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 0 2023-07-13T22:25:55.969 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.969 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-07-13T22:25:55.969 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 13, 2023-07-13T22:25:55.969 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 13, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 0, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 11, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 12, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 1, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 10, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 10, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 0, 2023-07-13T22:25:55.970 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 48768, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 198964 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2023-07-13T22:25:55.971 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.972 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.973 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.974 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.975 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-07-13T22:25:55.976 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.977 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.978 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.979 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-07-13T22:25:55.980 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.981 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.982 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.983 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.984 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.985 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-07-13T22:25:55.986 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.986 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.986 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.986 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.987 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-07-13T22:25:55.987 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.987 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.987 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.988 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-07-13T22:25:55.988 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.988 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.989 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.989 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.989 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-07-13T22:25:55.989 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.990 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.990 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.990 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.990 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-07-13T22:25:55.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.991 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.991 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.992 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.992 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-07-13T22:25:55.992 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.992 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.993 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.993 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-07-13T22:25:55.993 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.994 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.994 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.994 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.994 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-07-13T22:25:55.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.995 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.995 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.995 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.996 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-07-13T22:25:55.996 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.996 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.997 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.997 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.997 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:25:55.997 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:25:55.998 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-07-13T22:25:55.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:55.998 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:55.998 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:55.999 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:55.999 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:55.999 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-07-13T22:25:55.999 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2023-07-13T22:25:56.000 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 0 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:25:56.001 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:25:56.002 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:25:56.003 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:25:56.004 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:25:56.005 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 229943, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 73, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-07-13T22:25:56.006 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7528, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 158, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-07-13T22:25:56.007 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 50040, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 198, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-07-13T22:25:56.008 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 35, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 10662, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 27, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 35, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 35, 2023-07-13T22:25:56.009 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.218996399, 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.006257039 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 35, 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 57 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.010 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 35, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 0, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 35, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 0, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 10, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 0, 2023-07-13T22:25:56.011 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 1, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 5, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 6, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-07-13T22:25:56.012 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 20, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-07-13T22:25:56.013 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-07-13T22:25:56.014 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-07-13T22:25:56.015 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 85, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-07-13T22:25:56.016 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 2, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 9, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 9, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 14, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 0, 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 1 2023-07-13T22:25:56.017 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 1, 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-07-13T22:25:56.018 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2023-07-13T22:25:56.019 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 939, 2023-07-13T22:25:56.020 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 686144, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 939, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 939, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 686144, 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:56.021 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:25:56.022 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 35, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 10662, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 35, 2023-07-13T22:25:56.023 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 10662, 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-07-13T22:25:56.024 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 35, 2023-07-13T22:25:56.025 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 35, 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 35, 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 35, 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.026 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:56.027 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:56.028 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x558ab3c2c000": { 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:56.029 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 540, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 540, 2023-07-13T22:25:56.030 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:56.031 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:56.032 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:55 smithi026 ceph-mon[95647]: pgmap v676: 97 pgs: 97 active+clean; 6.5 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 69 MiB/s wr, 702 op/s 2023-07-13T22:25:56.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:55 smithi117 ceph-mon[129074]: pgmap v676: 97 pgs: 97 active+clean; 6.5 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 69 MiB/s wr, 702 op/s 2023-07-13T22:25:56.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:55 smithi121 ceph-mon[129187]: pgmap v676: 97 pgs: 97 active+clean; 6.5 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 69 MiB/s wr, 702 op/s 2023-07-13T22:25:56.437 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-07-13T22:25:56.437 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-07-13T22:25:56.438 DEBUG:tasks.check_counter:Getting stats from d 2023-07-13T22:25:56.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-13T22:25:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:25:58.116 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:57 smithi026 ceph-mon[95647]: pgmap v677: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 286 op/s 2023-07-13T22:25:58.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:57 smithi117 ceph-mon[129074]: pgmap v677: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 286 op/s 2023-07-13T22:25:58.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:57 smithi121 ceph-mon[129187]: pgmap v677: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 MiB/s wr, 286 op/s 2023-07-13T22:25:58.754 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 2944, 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2023-07-13T22:25:58.755 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.004040268, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000080435, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001224001, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2023-07-13T22:25:58.756 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001217651, 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000405883 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 230, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 227, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 474768, 2023-07-13T22:25:58.758 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 129392, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.053173434, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.022764931, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.021683487, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.008454027, 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:58.759 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 226, 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.197592565, 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000874303 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:58.760 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 474768, 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 129392 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1, 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 173, 2023-07-13T22:25:58.761 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 160, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 106016, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 3, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.022005018, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.017116709, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001787169, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-13T22:25:58.762 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 171, 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.056000805, 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000327490 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:58.763 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 106016 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-07-13T22:25:58.764 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:25:58.765 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:25:58.766 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:25:58.767 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-07-13T22:25:58.768 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148984, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 30, 2023-07-13T22:25:58.769 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:25:58.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-07-13T22:25:58.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:25:58.772 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 231, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 442245, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 231, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:25:58.773 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 231, 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 442245, 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:25:58.774 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:25:59.276 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-13T22:25:59.276 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-13T22:25:59.276 DEBUG:tasks.check_counter:Getting stats from g 2023-07-13T22:25:59.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-13T22:26:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:26:00.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:25:59 smithi117 ceph-mon[129074]: pgmap v678: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-13T22:26:00.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:25:59 smithi026 ceph-mon[95647]: pgmap v678: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-13T22:26:00.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:25:59 smithi121 ceph-mon[129187]: pgmap v678: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 MiB/s wr, 217 op/s 2023-07-13T22:26:01.585 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:26:01.586 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:01.586 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 806, 2023-07-13T22:26:01.586 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 808, 2023-07-13T22:26:01.586 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 270761, 2023-07-13T22:26:01.586 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 250667, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 6, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.189266700, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.060501138, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.054298422, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.058903269, 2023-07-13T22:26:01.587 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 806, 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.368681068, 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000457420 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.588 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 0 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:01.589 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1044, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 1038, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 720006, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 379834, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.183659919, 2023-07-13T22:26:01.590 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.057032132, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.065804060, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.049030608, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1037, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.474979308, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000458032 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.591 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 508000, 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 128800 2023-07-13T22:26:01.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 8, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 176, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 4452, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 368832, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:01.593 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.022981699, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.017398337, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.003212021, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000068468, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 175, 2023-07-13T22:26:01.594 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.089360946, 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000510633 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.595 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 367888 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 1, 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-07-13T22:26:01.596 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-MDSRank": { 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3225, 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.124221207, 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000038518 2023-07-13T22:26:01.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "queue_len": 0, 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "complete_latency": { 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": { 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "request": 0, 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "reply": 0, 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "reply_latency": { 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.599 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_reply": 0, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "forward": 0, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:01.600 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_commit": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_split": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_merge": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_top": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_bottom": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:01.601 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_pinned": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_expired": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "subtrees": 1, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_hit": 0, 2023-07-13T22:26:01.602 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_forward": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_discover": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "traverse_lock": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "load_cent": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "q": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "exported": 0, 2023-07-13T22:26:01.603 INFO:teuthology.orchestra.run.smithi026.stdout: "exported_inodes": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "imported": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "imported_inodes": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rfiles": 0, 2023-07-13T22:26:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rbytes": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "root_rsnaps": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:01.605 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:01.606 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps": 0, 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "process_request_cap_release": 0 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_cache": { 2023-07-13T22:26:01.607 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_try_discover": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_send_discover": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:01.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_created": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_enqueued": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "strays_migrated": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:01.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_started": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "recovery_completed": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_flush": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:01.610 INFO:teuthology.orchestra.run.smithi026.stdout: "ireq_inodestats": 0 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_log": { 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "evadd": 0, 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "evex": 0, 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "evtrm": 0, 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "ev": 0, 2023-07-13T22:26:01.611 INFO:teuthology.orchestra.run.smithi026.stdout: "evexg": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "evexd": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "segadd": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "segex": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "segtrm": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "seg": 1, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "segexg": 0, 2023-07-13T22:26:01.612 INFO:teuthology.orchestra.run.smithi026.stdout: "segexd": 0, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "expos": 4194304, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "wrpos": 0, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "rdpos": 4194304, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "jlat": { 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "replayed": 1 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_mem": { 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "ino": 3, 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "ino+": 3, 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "ino-": 0, 2023-07-13T22:26:01.614 INFO:teuthology.orchestra.run.smithi026.stdout: "dir": 1, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "dir+": 1, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "dir-": 0, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "dn": 0, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "dn+": 0, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "dn-": 0, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "cap": 0, 2023-07-13T22:26:01.615 INFO:teuthology.orchestra.run.smithi026.stdout: "cap+": 0, 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "cap-": 0, 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "rss": 47744, 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "heap": 207156 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_server": { 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:01.616 INFO:teuthology.orchestra.run.smithi026.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_request": 0, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_client_session": 0, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "handle_peer_request": 0, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "req_create_latency": { 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getattr_latency": { 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:01.618 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "req_link_latency": { 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookup_latency": { 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:01.620 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupino_latency": { 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.621 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupname_latency": { 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.622 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.623 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: "req_lssnap_latency": { 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.624 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mkdir_latency": { 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mknod_latency": { 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.625 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: "req_mksnap_latency": { 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.626 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "req_open_latency": { 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "req_readdir_latency": { 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.627 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rename_latency": { 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.628 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:01.629 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmdir_latency": { 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.630 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:01.631 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setattr_latency": { 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.632 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.633 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setlayout_latency": { 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.634 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: "req_setxattr_latency": { 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.635 INFO:teuthology.orchestra.run.smithi026.stdout: "req_symlink_latency": { 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "req_unlink_latency": { 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.636 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.637 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_sessions": { 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "session_count": 0, 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "session_add": 0, 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "session_remove": 0, 2023-07-13T22:26:01.638 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_open": 0, 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "sessions_stale": 0, 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "total_load": 0, 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "average_load": 0, 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "avg_session_uptime": 0 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-07-13T22:26:01.639 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:01.640 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:01.641 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:01.642 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:01.643 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:01.644 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 197463, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 54, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:01.645 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 158, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:01.646 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 8320, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 13, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-07-13T22:26:01.647 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "objecter": { 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_active": 0, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_laggy": 0, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send": 1616, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_send_bytes": 21, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_resend": 3, 2023-07-13T22:26:01.648 INFO:teuthology.orchestra.run.smithi026.stdout: "op_reply": 1616, 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "op_latency": { 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1616, 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 7.513876793, 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.004649676 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "op_inflight": 0, 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "oplen_avg": { 2023-07-13T22:26:01.649 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 1616, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 1617 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "op": 1616, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "op_r": 1616, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "op_w": 0, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "op_rmw": 0, 2023-07-13T22:26:01.650 INFO:teuthology.orchestra.run.smithi026.stdout: "op_pg": 0, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_stat": 805, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_create": 0, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_read": 809, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_write": 0, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writefull": 0, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_writesame": 0, 2023-07-13T22:26:01.651 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_append": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_zero": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_truncate": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_delete": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_mapext": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:01.652 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_getxattr": 1, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_setxattr": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_call": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_watch": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_notify": 0, 2023-07-13T22:26:01.653 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "osdop_other": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_active": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_send": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_resend": 0, 2023-07-13T22:26:01.654 INFO:teuthology.orchestra.run.smithi026.stdout: "linger_ping": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_active": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_send": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolop_resend": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_active": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_send": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "poolstat_resend": 0, 2023-07-13T22:26:01.655 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_active": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_send": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "statfs_resend": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "command_active": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "command_send": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "command_resend": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "map_epoch": 85, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "map_full": 0, 2023-07-13T22:26:01.656 INFO:teuthology.orchestra.run.smithi026.stdout: "map_inc": 2, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_sessions": 6, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_open": 6, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_session_close": 0, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_laggy": 0, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_wr": 0, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_rd": 2, 2023-07-13T22:26:01.657 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_del": 0 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: "oft": { 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_objs": 0, 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_updates": 0, 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: "omap_total_removes": 0 2023-07-13T22:26:01.658 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "osd-slow-ops": { 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "slow_ops_count": 0 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "purge_queue": { 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing": 0, 2023-07-13T22:26:01.659 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_executed": 0, 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:26:01.660 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 1854, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 813225, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 1854, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:26:01.661 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1854, 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 813225, 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.662 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:01.663 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1616, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 545, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1616, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 545, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.664 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1024, 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:01.665 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 1616, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 1616, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 1616, 2023-07-13T22:26:01.666 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 1616, 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.667 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:01.668 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.669 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-write_buf_throttle-0x55d36ae98500": { 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 3758096384, 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:01.670 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 0, 2023-07-13T22:26:01.671 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 0, 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:01.672 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:26:01.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:01 smithi026 ceph-mon[95647]: pgmap v679: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 105 op/s 2023-07-13T22:26:02.108 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-07-13T22:26:02.109 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-07-13T22:26:02.109 DEBUG:tasks.check_counter:Getting stats from j 2023-07-13T22:26:02.110 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-13T22:26:02.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:01 smithi117 ceph-mon[129074]: pgmap v679: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 105 op/s 2023-07-13T22:26:02.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:01 smithi121 ceph-mon[129187]: pgmap v679: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 105 op/s 2023-07-13T22:26:02.847 INFO:teuthology.orchestra.run.smithi026.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.a/config 2023-07-13T22:26:04.166 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:03 smithi026 ceph-mon[95647]: pgmap v680: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 42 op/s 2023-07-13T22:26:04.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:03 smithi117 ceph-mon[129074]: pgmap v680: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 42 op/s 2023-07-13T22:26:04.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:03 smithi121 ceph-mon[129187]: pgmap v680: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 42 op/s 2023-07-13T22:26:04.461 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-07-13T22:26:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 4, 2023-07-13T22:26:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 4, 2023-07-13T22:26:04.461 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 2944, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 448, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 4, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 0, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.003824421, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.000064624, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001063754, 2023-07-13T22:26:04.462 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 3, 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.001336766, 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000445588 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:04.463 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:04.464 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 1, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 174, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 160, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 106192, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 3, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.021173047, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.017126213, 2023-07-13T22:26:04.465 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.001439643, 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 172, 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.058144569, 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000338049 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:04.466 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:04.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 106192 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_messages": 232, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages": 229, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_bytes": 475312, 2023-07-13T22:26:04.468 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_bytes": 130576, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_created_connections": 2, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_active_connections": 1, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_total_time": 0.059515746, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_send_time": 0.024315651, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_recv_time": 0.024718561, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_running_fast_dispatch_time": 0.009590121, 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:04.469 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 228, 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.099902071, 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000438166 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:04.470 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_recv_encrypted_bytes": 475312, 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "msgr_send_encrypted_bytes": 130576 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "cct": { 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "total_workers": 0, 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: "unhealthy_workers": 0 2023-07-13T22:26:04.471 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "mempool": { 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:04.472 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:04.473 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:04.474 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:04.475 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_items": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_bytes": 148967, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_anon_items": 29, 2023-07-13T22:26:04.476 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_bytes": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_items": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:04.477 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_bytes": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_items": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap_items": 0, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_bytes": 928, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_co_items": 2, 2023-07-13T22:26:04.478 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_1_items": 0, 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "unittest_2_items": 0 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "val": 0, 2023-07-13T22:26:04.479 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 104857600, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_started": 0, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get": 233, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_sum": 442505, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "get_or_fail_success": 233, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "take": 0, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "take_sum": 0, 2023-07-13T22:26:04.480 INFO:teuthology.orchestra.run.smithi026.stdout: "put": 233, 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "put_sum": 442505, 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "wait": { 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgcount": 0, 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "sum": 0.000000000, 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: "avgtime": 0.000000000 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-07-13T22:26:04.481 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-07-13T22:26:04.946 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-13T22:26:04.946 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-13T22:26:04.946 DEBUG:tasks.check_counter:Getting stats from b 2023-07-13T22:26:04.946 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-13T22:26:05.521 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:26:05.995 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:05 smithi117 ceph-mon[129074]: pgmap v681: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 10 op/s 2023-07-13T22:26:06.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:05 smithi026 ceph-mon[95647]: pgmap v681: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 10 op/s 2023-07-13T22:26:06.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:05 smithi121 ceph-mon[129187]: pgmap v681: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 0 B/s wr, 10 op/s 2023-07-13T22:26:06.638 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 4544, 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3756, 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1404046, 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 7461020, 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 16, 2023-07-13T22:26:06.639 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.383621158, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.116431671, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.170529638, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.099507931, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3752, 2023-07-13T22:26:06.640 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.719253318, 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000191698 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3829, 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000821653, 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000214 2023-07-13T22:26:06.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 55392, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 29264 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 3062, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 3403, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1327116, 2023-07-13T22:26:06.642 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1098732, 2023-07-13T22:26:06.643 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 11, 2023-07-13T22:26:06.643 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2023-07-13T22:26:06.643 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.225806932, 2023-07-13T22:26:06.643 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.079255572, 2023-07-13T22:26:06.643 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.082504295, 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.059088632, 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3403, 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.903549999, 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000265515 2023-07-13T22:26:06.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2090, 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000635729, 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000304 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 555504, 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 154960 2023-07-13T22:26:06.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2162, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2237, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 605683, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 2265775, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 152, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 5, 2023-07-13T22:26:06.646 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.219577922, 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.055634677, 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.105594430, 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.047517526, 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2237, 2023-07-13T22:26:06.647 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.164949970, 2023-07-13T22:26:06.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000073737 2023-07-13T22:26:06.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.648 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:06.648 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2524, 2023-07-13T22:26:06.649 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000519752, 2023-07-13T22:26:06.649 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000205 2023-07-13T22:26:06.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.650 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:06.650 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 376672 2023-07-13T22:26:06.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.650 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2023-07-13T22:26:06.651 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2023-07-13T22:26:06.651 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2023-07-13T22:26:06.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.651 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1230, 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.845606714, 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000687485 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2463, 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.073606176, 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000029884 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 84, 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 83, 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 83, 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.263484979, 2023-07-13T22:26:06.654 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.003174517 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 1, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 12, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2023-07-13T22:26:06.655 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 27, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 1, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 26, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 25, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 2, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2023-07-13T22:26:06.656 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 2, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 2, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 126, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 112, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:06.657 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 1, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 2, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 1, 2023-07-13T22:26:06.658 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 503, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 246, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 503, 2023-07-13T22:26:06.659 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 189, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 18, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 171, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 314, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 3, 2023-07-13T22:26:06.660 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 17, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 159, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 1344, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 1313, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 7, 2023-07-13T22:26:06.661 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 45 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2023-07-13T22:26:06.662 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 7, 2023-07-13T22:26:06.663 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 2, 2023-07-13T22:26:06.663 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:06.663 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 20, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 18, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:06.664 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 18, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 1, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:06.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 1695, 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 1695, 2023-07-13T22:26:06.666 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 3, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 3, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2023-07-13T22:26:06.667 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 10378657, 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 538, 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 3.686446186, 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.006852130 2023-07-13T22:26:06.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 25, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 63, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 38, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 20, 2023-07-13T22:26:06.669 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 22, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 2, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 27, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 65, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 38, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 2, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 35, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 33, 2023-07-13T22:26:06.670 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 57136, 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 207156 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 130, 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 2, 2023-07-13T22:26:06.671 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 84, 2023-07-13T22:26:06.672 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 17, 2023-07-13T22:26:06.672 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 6, 2023-07-13T22:26:06.672 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2023-07-13T22:26:06.672 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 22, 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.015655310, 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000711605 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 16, 2023-07-13T22:26:06.673 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.003405418, 2023-07-13T22:26:06.674 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000212838 2023-07-13T22:26:06.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.674 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:06.674 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.675 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.675 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.675 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2023-07-13T22:26:06.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.676 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.676 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.677 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2023-07-13T22:26:06.677 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.677 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.678 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.678 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:06.679 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.679 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.679 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.680 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2023-07-13T22:26:06.680 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:06.680 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000315063, 2023-07-13T22:26:06.680 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000315063 2023-07-13T22:26:06.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.681 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2023-07-13T22:26:06.681 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.681 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.683 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 10, 2023-07-13T22:26:06.684 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.026009913, 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002600991 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2023-07-13T22:26:06.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:06.687 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000377592, 2023-07-13T22:26:06.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000377592 2023-07-13T22:26:06.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.687 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2023-07-13T22:26:06.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4, 2023-07-13T22:26:06.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.009736588, 2023-07-13T22:26:06.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002434147 2023-07-13T22:26:06.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.689 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2023-07-13T22:26:06.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2023-07-13T22:26:06.689 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.002623728, 2023-07-13T22:26:06.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001311864 2023-07-13T22:26:06.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.690 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:06.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.691 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.691 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:06.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4, 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.159087952, 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.039771988 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:06.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001308055, 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001308055 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6, 2023-07-13T22:26:06.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.016912578, 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002818763 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.698 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 16, 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.028052782, 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001753298 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:06.699 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2023-07-13T22:26:06.700 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 22, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 22, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 726 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:06.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:06.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:06.704 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:06.705 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 298023, 2023-07-13T22:26:06.706 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 116, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:06.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 103754, 2023-07-13T22:26:06.708 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 921, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2023-07-13T22:26:06.709 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 4284, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 6210255, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 31, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 4284, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4284, 2023-07-13T22:26:06.710 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 37.272222140, 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008700331 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4284, 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4326 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.711 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 4284, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 546, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 3738, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 11, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 25, 2023-07-13T22:26:06.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 545, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 59, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 3091, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2023-07-13T22:26:06.713 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 521, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 22, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:06.714 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2023-07-13T22:26:06.715 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2023-07-13T22:26:06.716 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2023-07-13T22:26:06.717 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 3, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2023-07-13T22:26:06.718 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 42, 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 10 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2023-07-13T22:26:06.719 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 137 2023-07-13T22:26:06.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 513, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 6, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 18, 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:06.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.722 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:06.722 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:06.722 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:06.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 9764, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 2433461, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 9764, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:06.723 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 9764, 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2433461, 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.724 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:06.725 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 4284, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 6212073, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 4284, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 6212073, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:06.727 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 4284, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 4284, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 4284, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 4284, 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:06.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:06.729 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:06.730 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 18, 2023-07-13T22:26:06.730 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1818, 2023-07-13T22:26:06.730 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:06.730 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 18, 2023-07-13T22:26:06.730 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:06.731 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:06.731 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 7, 2023-07-13T22:26:06.731 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1818, 2023-07-13T22:26:06.731 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:06.732 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.732 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.732 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.732 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:06.733 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55d5856fa0a0": { 2023-07-13T22:26:06.733 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:06.733 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:06.734 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:06.734 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1695, 2023-07-13T22:26:06.734 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 6184353, 2023-07-13T22:26:06.734 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:06.735 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1695, 2023-07-13T22:26:06.735 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:06.735 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:06.735 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 538, 2023-07-13T22:26:06.736 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 6184353, 2023-07-13T22:26:06.736 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:06.736 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:06.736 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:06.737 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:06.737 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.737 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:06.737 INFO:teuthology.orchestra.run.smithi117.stdout:} 2023-07-13T22:26:06.964 INFO:tasks.check_counter:Daemon mds.b mds.exported=1 2023-07-13T22:26:06.964 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2023-07-13T22:26:06.964 DEBUG:tasks.check_counter:Getting stats from e 2023-07-13T22:26:06.965 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-13T22:26:07.512 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:26:08.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:07 smithi117 ceph-mon[129074]: pgmap v682: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2023-07-13T22:26:08.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:07 smithi026 ceph-mon[95647]: pgmap v682: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2023-07-13T22:26:08.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:07 smithi121 ceph-mon[129187]: pgmap v682: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2023-07-13T22:26:08.690 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2023-07-13T22:26:08.690 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1651, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1653, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 494653, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 513468, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:08.691 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.129726838, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.042948745, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.031019150, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.043338220, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1651, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.048528432, 2023-07-13T22:26:08.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000029393 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-13T22:26:08.693 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 7, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 178, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3660, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 372204, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2023-07-13T22:26:08.694 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.009208022, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.006211554, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001397705, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000066873, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 177, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.008711019, 2023-07-13T22:26:08.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000049214 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.696 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 371520 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 248, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 237, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 532332, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 132055, 2023-07-13T22:26:08.697 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.034115740, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.012336059, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.015827315, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.005969180, 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:08.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 236, 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.009780759, 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000041443 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 532064, 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 131744 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.700 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3301, 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.081716346, 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000024755 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.701 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.702 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2023-07-13T22:26:08.704 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2023-07-13T22:26:08.705 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2023-07-13T22:26:08.706 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:08.707 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:08.708 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:08.709 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2023-07-13T22:26:08.710 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:08.711 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2023-07-13T22:26:08.712 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.713 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2023-07-13T22:26:08.714 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 1, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 0, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 4194304, 2023-07-13T22:26:08.715 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 1 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2023-07-13T22:26:08.716 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 3, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 3, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 1, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 1, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 0, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 0, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 0, 2023-07-13T22:26:08.717 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 48944, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 207156 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2023-07-13T22:26:08.718 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.720 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.724 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2023-07-13T22:26:08.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.728 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.729 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:08.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:08.732 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.733 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:08.734 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2023-07-13T22:26:08.736 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:08.738 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 0, 2023-07-13T22:26:08.739 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 0 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.740 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:08.741 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:08.742 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:08.743 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:08.744 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 181281, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 46, 2023-07-13T22:26:08.745 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:08.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 8320, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 13, 2023-07-13T22:26:08.747 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2023-07-13T22:26:08.748 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 1654, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 1654, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1654, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.759984112, 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000459482 2023-07-13T22:26:08.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1654, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1655 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 1654, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 1654, 2023-07-13T22:26:08.750 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 824, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 828, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2023-07-13T22:26:08.751 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:08.752 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2023-07-13T22:26:08.753 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2023-07-13T22:26:08.754 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2023-07-13T22:26:08.755 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 4, 2023-07-13T22:26:08.756 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 4, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2023-07-13T22:26:08.757 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.758 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.759 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1902, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 843852, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:08.760 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1902, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1902, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 843852, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.761 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:08.762 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1654, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 545, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1654, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 545, 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:08.763 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2023-07-13T22:26:08.764 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 1654, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1654, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1654, 2023-07-13T22:26:08.765 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1654, 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:08.766 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:08.767 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.768 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55fee3344460": { 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:08.769 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:08.770 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:08.771 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:08.771 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.771 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:08.771 INFO:teuthology.orchestra.run.smithi117.stdout:} 2023-07-13T22:26:09.054 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-13T22:26:09.054 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-13T22:26:09.055 DEBUG:tasks.check_counter:Getting stats from h 2023-07-13T22:26:09.055 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-13T22:26:09.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:08 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:26:09.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:08 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:26:09.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:08 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-13T22:26:09.603 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:26:10.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:09 smithi117 ceph-mon[129074]: pgmap v683: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-07-13T22:26:10.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:26:10.070 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:26:10.070 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:09 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:26:10.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:09 smithi026 ceph-mon[95647]: pgmap v683: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-07-13T22:26:10.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:26:10.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:26:10.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:09 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:26:10.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:09 smithi121 ceph-mon[129187]: pgmap v683: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2023-07-13T22:26:10.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-13T22:26:10.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-13T22:26:10.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:09 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' 2023-07-13T22:26:10.680 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2023-07-13T22:26:10.680 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 140, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 146, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 60835, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 168771, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 19, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.021845845, 2023-07-13T22:26:10.681 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.004438139, 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.012504027, 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.001278216, 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 145, 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.026555611, 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000183142 2023-07-13T22:26:10.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 145, 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000039975, 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000275 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:26:10.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 117, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 289, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 51617, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 706587, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 39, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2023-07-13T22:26:10.684 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.027472091, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.009053797, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.015969475, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000528130, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 287, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.031836936, 2023-07-13T22:26:10.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000110930 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 151, 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000027309, 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000180 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:10.686 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 372048 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1668, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1299, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 926482, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 522185, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 9, 2023-07-13T22:26:10.687 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.131678956, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.057169274, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.071827494, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.011236594, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1298, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.056839825, 2023-07-13T22:26:10.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000043790 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2199, 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000517991, 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000235 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 543392, 2023-07-13T22:26:10.689 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 147776 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:10.690 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 100, 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.008249929, 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000082499 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:10.691 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 17, 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000553856, 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000032579 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2023-07-13T22:26:10.692 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 4, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 4, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.004991490, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001247872 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2023-07-13T22:26:10.693 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 15, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2023-07-13T22:26:10.694 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 15, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 13, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 8, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 3, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 10, 2023-07-13T22:26:10.695 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 6, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 1, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2023-07-13T22:26:10.696 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 1, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 1, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 2, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 28, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 21, 2023-07-13T22:26:10.697 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 10738140632, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 28, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:10.698 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 28, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 1, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 3, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:10.699 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 1, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 2, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 4 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:10.700 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 2, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 2, 2023-07-13T22:26:10.701 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 2, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2023-07-13T22:26:10.702 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 1, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2023-07-13T22:26:10.703 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 15, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 15, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 1, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2023-07-13T22:26:10.704 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 1, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4220979, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2023-07-13T22:26:10.705 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11, 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.032295602, 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002935963 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 19, 2023-07-13T22:26:10.706 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 25, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 6, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 18, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 23, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 5, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 15, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 27, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 12, 2023-07-13T22:26:10.707 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 5, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 5, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 47776, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 198964 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 7, 2023-07-13T22:26:10.708 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 1, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 4, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 9, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 4, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:10.710 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000440179, 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000440179 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.712 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.713 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.714 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2023-07-13T22:26:10.715 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2023-07-13T22:26:10.717 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000372379, 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000372379 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.719 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.720 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:10.722 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.724 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.725 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2023-07-13T22:26:10.726 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.004178932, 2023-07-13T22:26:10.727 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002089466 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 1, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2023-07-13T22:26:10.729 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 3, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 3, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 2756 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:10.730 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:10.731 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:10.732 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:10.733 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:10.734 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 275115, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 95, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2023-07-13T22:26:10.735 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:10.736 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 75565, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 315, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2023-07-13T22:26:10.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 90, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 38546, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 33, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 90, 2023-07-13T22:26:10.738 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 90, 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.456990459, 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005077671 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 90, 2023-07-13T22:26:10.739 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 114 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 90, 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 29, 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 61, 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2023-07-13T22:26:10.740 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 1, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 12, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 10, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2023-07-13T22:26:10.741 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 14, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 28, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2023-07-13T22:26:10.742 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2023-07-13T22:26:10.743 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2023-07-13T22:26:10.744 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2023-07-13T22:26:10.745 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 9, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 9, 2023-07-13T22:26:10.746 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 17, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 2 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2023-07-13T22:26:10.747 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 6 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2023-07-13T22:26:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 2, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 2, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 2, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1921, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 847887, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1921, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:10.750 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1921, 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 847887, 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:10.752 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 90, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 38748, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 90, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 38748, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.753 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:10.754 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 90, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 90, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 90, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 90, 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:10.755 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:10.756 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 202, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1, 2023-07-13T22:26:10.757 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 202, 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x557be21e00a0": { 2023-07-13T22:26:10.758 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 15, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 26675, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 15, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:10.759 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 11, 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 26675, 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:10.760 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.761 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:10.761 INFO:teuthology.orchestra.run.smithi117.stdout:} 2023-07-13T22:26:11.041 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2023-07-13T22:26:11.041 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2023-07-13T22:26:11.042 DEBUG:tasks.check_counter:Getting stats from k 2023-07-13T22:26:11.042 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-13T22:26:11.630 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.b/config 2023-07-13T22:26:12.151 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:11 smithi117 ceph-mon[129074]: pgmap v684: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:12.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:11 smithi026 ceph-mon[95647]: pgmap v684: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:12.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:11 smithi121 ceph-mon[129187]: pgmap v684: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 479, 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 654, 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 149154, 2023-07-13T22:26:12.935 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 156404, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 10, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.060312862, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.025230315, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.026411492, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.004234674, 2023-07-13T22:26:12.936 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 653, 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.042001569, 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000064320 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1087, 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000224450, 2023-07-13T22:26:12.937 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000206 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 116, 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 292, 2023-07-13T22:26:12.938 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 50207, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 795093, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 49, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.032201858, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.009181855, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.020830272, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000184090, 2023-07-13T22:26:12.939 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 290, 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.059316245, 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000204538 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 147, 2023-07-13T22:26:12.940 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000030248, 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000205 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 373152 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:12.941 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 358, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 350, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 571133, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 187344, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:12.942 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.042140331, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.014722395, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.021677286, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.006980899, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 349, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.036240036, 2023-07-13T22:26:12.943 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000103839 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 108, 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000032444, 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000300 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 524928, 2023-07-13T22:26:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 138320 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2023-07-13T22:26:12.945 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 44, 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001256256, 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000028551 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:12.946 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2, 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000009287, 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000004643 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2023-07-13T22:26:12.947 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2023-07-13T22:26:12.948 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2023-07-13T22:26:12.948 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2023-07-13T22:26:12.948 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.948 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.948 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 11, 2023-07-13T22:26:12.949 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 10, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 10, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 10, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2023-07-13T22:26:12.950 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:12.951 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2023-07-13T22:26:12.952 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 21, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 10738140632, 2023-07-13T22:26:12.953 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:12.954 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:12.955 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2023-07-13T22:26:12.956 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 1, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:12.957 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:12.958 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0 2023-07-13T22:26:12.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 1, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 1, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2023-07-13T22:26:12.960 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 1, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 1, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2023-07-13T22:26:12.961 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4194844, 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001827763, 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001827763 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2023-07-13T22:26:12.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 13, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 13, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 11, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 12, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 1, 2023-07-13T22:26:12.963 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 10, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 10, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 50376, 2023-07-13T22:26:12.964 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 198964 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2023-07-13T22:26:12.965 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.966 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2023-07-13T22:26:12.967 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.968 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.969 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:12.969 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.971 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.972 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.974 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.976 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.977 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:12.978 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.978 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.978 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2023-07-13T22:26:12.979 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.980 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2023-07-13T22:26:12.981 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.983 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.985 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:12.986 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 0, 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 0, 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 0, 2023-07-13T22:26:12.987 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2023-07-13T22:26:12.988 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2023-07-13T22:26:12.988 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2023-07-13T22:26:12.988 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 0 2023-07-13T22:26:12.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:12.989 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:12.990 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:12.991 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:12.992 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 262135, 2023-07-13T22:26:12.993 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 85, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:12.994 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 158, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 50040, 2023-07-13T22:26:12.995 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 198, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2023-07-13T22:26:12.996 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 35, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 10662, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 33, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 35, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35, 2023-07-13T22:26:12.997 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.292993889, 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008371253 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 35, 2023-07-13T22:26:12.998 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 57 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 35, 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 0, 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 35, 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2023-07-13T22:26:12.999 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 0, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 10, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 0, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 1, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 5, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2023-07-13T22:26:13.000 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 6, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 0, 2023-07-13T22:26:13.001 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 20, 2023-07-13T22:26:13.002 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:13.002 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:13.002 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:13.002 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2023-07-13T22:26:13.002 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2023-07-13T22:26:13.003 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2023-07-13T22:26:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2023-07-13T22:26:13.005 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 2, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 9, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 9, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 14, 2023-07-13T22:26:13.006 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 0, 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 1 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2023-07-13T22:26:13.007 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: "osd-slow-ops": { 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2023-07-13T22:26:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2023-07-13T22:26:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 949, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 668068, 2023-07-13T22:26:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 949, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 949, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 668068, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2023-07-13T22:26:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 35, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 10662, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 35, 2023-07-13T22:26:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 10662, 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:13.014 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:13.015 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 35, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 35, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 35, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 35, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:13.016 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2023-07-13T22:26:13.017 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x560314b0e0a0": { 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2023-07-13T22:26:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 540, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2023-07-13T22:26:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 1, 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 540, 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: } 2023-07-13T22:26:13.021 INFO:teuthology.orchestra.run.smithi117.stdout:} 2023-07-13T22:26:13.273 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-07-13T22:26:13.273 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-07-13T22:26:13.273 DEBUG:tasks.check_counter:Getting stats from c 2023-07-13T22:26:13.274 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-13T22:26:13.846 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:26:14.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:13 smithi117 ceph-mon[129074]: pgmap v685: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:14.187 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:13 smithi121 ceph-mon[129187]: pgmap v685: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:14.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:13 smithi026 ceph-mon[95647]: pgmap v685: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:15.012 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-13T22:26:15.012 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:15.012 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 249, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 238, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 533170, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 132653, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 1, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.034813158, 2023-07-13T22:26:15.013 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.012155222, 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.015245847, 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.006253772, 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 237, 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.009909190, 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000041810 2023-07-13T22:26:15.014 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.015 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 532336, 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 132336 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 647, 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 819, 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 161058, 2023-07-13T22:26:15.016 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 572915, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.065585646, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.017884930, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.014541435, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.026978937, 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 818, 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.030208436, 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000036929 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 373248 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1654, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1656, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 520790, 2023-07-13T22:26:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 514452, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 4, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.137458154, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.038963363, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.035200024, 2023-07-13T22:26:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.050236547, 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1654, 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.051231789, 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000030974 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2023-07-13T22:26:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2023-07-13T22:26:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2023-07-13T22:26:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3934, 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.104717035, 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000026618 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2023-07-13T22:26:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2023-07-13T22:26:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2023-07-13T22:26:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2023-07-13T22:26:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2023-07-13T22:26:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 3, 2023-07-13T22:26:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2023-07-13T22:26:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 0, 2023-07-13T22:26:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 3, 2023-07-13T22:26:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 3, 2023-07-13T22:26:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2023-07-13T22:26:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2023-07-13T22:26:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 1, 2023-07-13T22:26:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2023-07-13T22:26:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2023-07-13T22:26:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2023-07-13T22:26:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2023-07-13T22:26:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2023-07-13T22:26:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2023-07-13T22:26:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2023-07-13T22:26:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2023-07-13T22:26:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2023-07-13T22:26:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2023-07-13T22:26:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2023-07-13T22:26:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2023-07-13T22:26:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2023-07-13T22:26:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 1, 2023-07-13T22:26:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 0, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 4219328, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 15 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 4, 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 16, 2023-07-13T22:26:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 12, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 2, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 13, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 11, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 3, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 15, 2023-07-13T22:26:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 12, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 45920, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 207156 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2023-07-13T22:26:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2023-07-13T22:26:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2023-07-13T22:26:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2023-07-13T22:26:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2023-07-13T22:26:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2023-07-13T22:26:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2023-07-13T22:26:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2023-07-13T22:26:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.069 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.070 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.071 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 1, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 0, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2023-07-13T22:26:15.072 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2540 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:15.073 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:15.074 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:15.075 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:15.076 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:15.077 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 190185, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 53, 2023-07-13T22:26:15.078 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:15.079 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 158, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 13904, 2023-07-13T22:26:15.080 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 40, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2023-07-13T22:26:15.081 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 2297, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 2297, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2297, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 9.460871523, 2023-07-13T22:26:15.082 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004118794 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2297, 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2298 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.083 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 2297, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 2297, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 1462, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 833, 2023-07-13T22:26:15.084 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2023-07-13T22:26:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:15.086 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:15.087 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2023-07-13T22:26:15.088 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2023-07-13T22:26:15.089 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 85, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 5, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 5, 2023-07-13T22:26:15.090 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2023-07-13T22:26:15.091 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.093 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 2546, 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 970213, 2023-07-13T22:26:15.094 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 2546, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2546, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 970213, 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:15.095 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:15.096 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2297, 2023-07-13T22:26:15.097 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 26680, 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2297, 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 26680, 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.098 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:15.099 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2297, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 2297, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2297, 2023-07-13T22:26:15.100 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 2297, 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:15.101 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:15.102 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.103 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x55c3a3606460": { 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:15.104 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:15.105 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:15.105 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:15.105 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:15.105 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:15.106 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:15.106 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:15.106 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:15.106 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:15.107 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:15.107 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.107 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:15.107 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-13T22:26:15.389 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-13T22:26:15.390 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-13T22:26:15.390 DEBUG:tasks.check_counter:Getting stats from f 2023-07-13T22:26:15.390 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-13T22:26:15.941 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:26:16.139 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:15 smithi121 ceph-mon[129187]: pgmap v686: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:16.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:15 smithi117 ceph-mon[129074]: pgmap v686: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:16.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:15 smithi026 ceph-mon[95647]: pgmap v686: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:17.131 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-13T22:26:17.132 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:17.132 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 385, 2023-07-13T22:26:17.132 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 378, 2023-07-13T22:26:17.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 585012, 2023-07-13T22:26:17.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 212406, 2023-07-13T22:26:17.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 13, 2023-07-13T22:26:17.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:17.134 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.053759691, 2023-07-13T22:26:17.134 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.017344786, 2023-07-13T22:26:17.134 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.026813648, 2023-07-13T22:26:17.135 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.006640323, 2023-07-13T22:26:17.135 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:17.135 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 377, 2023-07-13T22:26:17.135 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.028086510, 2023-07-13T22:26:17.136 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000074500 2023-07-13T22:26:17.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.136 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:17.137 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 153, 2023-07-13T22:26:17.137 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000029652, 2023-07-13T22:26:17.137 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000193 2023-07-13T22:26:17.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.138 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 535664, 2023-07-13T22:26:17.138 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 139856 2023-07-13T22:26:17.138 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.138 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:17.139 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 124, 2023-07-13T22:26:17.139 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 297, 2023-07-13T22:26:17.139 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 54427, 2023-07-13T22:26:17.140 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 775333, 2023-07-13T22:26:17.140 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 46, 2023-07-13T22:26:17.140 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2023-07-13T22:26:17.140 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.029838730, 2023-07-13T22:26:17.141 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.009213149, 2023-07-13T22:26:17.141 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.018330317, 2023-07-13T22:26:17.141 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000319668, 2023-07-13T22:26:17.142 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:17.142 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 296, 2023-07-13T22:26:17.142 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.023369761, 2023-07-13T22:26:17.142 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000078951 2023-07-13T22:26:17.143 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.143 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:17.143 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 156, 2023-07-13T22:26:17.143 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000044805, 2023-07-13T22:26:17.144 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000287 2023-07-13T22:26:17.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.144 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-07-13T22:26:17.145 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 374976 2023-07-13T22:26:17.145 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.145 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:17.145 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 860, 2023-07-13T22:26:17.146 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 786, 2023-07-13T22:26:17.146 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 247086, 2023-07-13T22:26:17.146 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 334826, 2023-07-13T22:26:17.146 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2023-07-13T22:26:17.147 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:17.147 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.065339685, 2023-07-13T22:26:17.147 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.024280212, 2023-07-13T22:26:17.147 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.039551346, 2023-07-13T22:26:17.148 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.004991836, 2023-07-13T22:26:17.148 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:17.148 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 784, 2023-07-13T22:26:17.149 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.047564233, 2023-07-13T22:26:17.149 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000060668 2023-07-13T22:26:17.149 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.149 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:17.150 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1578, 2023-07-13T22:26:17.150 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000344915, 2023-07-13T22:26:17.150 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000218 2023-07-13T22:26:17.150 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.151 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-13T22:26:17.151 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2023-07-13T22:26:17.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.151 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2023-07-13T22:26:17.152 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2023-07-13T22:26:17.152 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2023-07-13T22:26:17.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.152 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2023-07-13T22:26:17.153 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:17.153 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:17.153 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 36, 2023-07-13T22:26:17.153 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.001649057, 2023-07-13T22:26:17.154 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000045807 2023-07-13T22:26:17.154 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.154 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:17.155 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:17.155 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:17.155 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2, 2023-07-13T22:26:17.156 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000012857, 2023-07-13T22:26:17.156 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000006428 2023-07-13T22:26:17.156 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.156 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2023-07-13T22:26:17.157 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 6, 2023-07-13T22:26:17.157 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 3, 2023-07-13T22:26:17.157 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2023-07-13T22:26:17.158 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3, 2023-07-13T22:26:17.158 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.001373530, 2023-07-13T22:26:17.158 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000457843 2023-07-13T22:26:17.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.159 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2023-07-13T22:26:17.159 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 3, 2023-07-13T22:26:17.159 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:17.159 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:17.160 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 11, 2023-07-13T22:26:17.160 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2023-07-13T22:26:17.160 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2023-07-13T22:26:17.161 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 14, 2023-07-13T22:26:17.161 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2023-07-13T22:26:17.161 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 14, 2023-07-13T22:26:17.161 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 14, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 2, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 2, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 2, 2023-07-13T22:26:17.162 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 8, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 3, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 2, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2023-07-13T22:26:17.163 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 3000, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 3, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:17.164 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 21, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 10738140632, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:17.165 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:17.166 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 2, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 3, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 2, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:17.167 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 3 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 3, 2023-07-13T22:26:17.168 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2023-07-13T22:26:17.169 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:17.170 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:17.170 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:17.170 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2023-07-13T22:26:17.170 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2023-07-13T22:26:17.170 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:17.171 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:17.171 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2023-07-13T22:26:17.171 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:17.171 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:17.172 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2023-07-13T22:26:17.172 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.172 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2023-07-13T22:26:17.173 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 4, 2023-07-13T22:26:17.173 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2023-07-13T22:26:17.173 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2023-07-13T22:26:17.173 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 4, 2023-07-13T22:26:17.174 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2023-07-13T22:26:17.174 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2023-07-13T22:26:17.174 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 1, 2023-07-13T22:26:17.174 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2023-07-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2023-07-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 1, 2023-07-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2023-07-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2023-07-13T22:26:17.175 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2023-07-13T22:26:17.176 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 4195691, 2023-07-13T22:26:17.176 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2023-07-13T22:26:17.176 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4, 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.008023189, 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.002005797 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2023-07-13T22:26:17.177 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 17, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 17, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 0, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 15, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 16, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 1, 2023-07-13T22:26:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 14, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 14, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 0, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 2, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 3, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 1, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 48004, 2023-07-13T22:26:17.179 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 198964 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 8, 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 6, 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 14, 2023-07-13T22:26:17.180 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2023-07-13T22:26:17.181 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2, 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.001168512, 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000584256 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.182 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.183 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.184 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000205018, 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000205018 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.185 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:17.186 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.187 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.188 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.189 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.190 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.191 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.192 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:17.193 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.194 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:17.195 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.197 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.198 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.199 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.200 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2023-07-13T22:26:17.202 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 2, 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 2, 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 736 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2023-07-13T22:26:17.203 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:17.204 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:17.205 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:17.205 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:17.205 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:17.205 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:17.205 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:17.206 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:17.207 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:17.207 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:17.207 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:17.207 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:17.208 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:17.208 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:17.208 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:17.208 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:17.209 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:17.209 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:17.209 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:17.209 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:17.210 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2023-07-13T22:26:17.210 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:17.210 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:17.210 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:17.211 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:17.211 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 263713, 2023-07-13T22:26:17.211 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 96, 2023-07-13T22:26:17.212 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:17.212 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:17.212 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2023-07-13T22:26:17.212 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2023-07-13T22:26:17.212 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 158, 2023-07-13T22:26:17.213 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:17.214 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:17.214 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:17.214 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2023-07-13T22:26:17.214 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 67341, 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 293, 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2023-07-13T22:26:17.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 41, 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 12010, 2023-07-13T22:26:17.216 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 33, 2023-07-13T22:26:17.217 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 41, 2023-07-13T22:26:17.217 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2023-07-13T22:26:17.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 41, 2023-07-13T22:26:17.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.223996865, 2023-07-13T22:26:17.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005463338 2023-07-13T22:26:17.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.218 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2023-07-13T22:26:17.218 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2023-07-13T22:26:17.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 41, 2023-07-13T22:26:17.219 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 65 2023-07-13T22:26:17.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.219 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 41, 2023-07-13T22:26:17.220 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 0, 2023-07-13T22:26:17.220 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 41, 2023-07-13T22:26:17.220 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2023-07-13T22:26:17.221 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2023-07-13T22:26:17.221 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 0, 2023-07-13T22:26:17.221 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 10, 2023-07-13T22:26:17.221 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 0, 2023-07-13T22:26:17.222 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 4, 2023-07-13T22:26:17.222 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 6, 2023-07-13T22:26:17.222 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2023-07-13T22:26:17.222 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2023-07-13T22:26:17.223 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2023-07-13T22:26:17.223 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2023-07-13T22:26:17.223 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 6, 2023-07-13T22:26:17.223 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2023-07-13T22:26:17.224 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:17.224 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:17.224 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 0, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 20, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2023-07-13T22:26:17.225 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2023-07-13T22:26:17.226 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2023-07-13T22:26:17.227 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2023-07-13T22:26:17.228 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2023-07-13T22:26:17.228 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2023-07-13T22:26:17.228 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2023-07-13T22:26:17.228 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2023-07-13T22:26:17.228 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2023-07-13T22:26:17.229 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 85, 2023-07-13T22:26:17.229 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2023-07-13T22:26:17.229 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2023-07-13T22:26:17.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 8, 2023-07-13T22:26:17.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 8, 2023-07-13T22:26:17.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2023-07-13T22:26:17.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2023-07-13T22:26:17.230 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 18, 2023-07-13T22:26:17.231 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 0, 2023-07-13T22:26:17.231 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 1 2023-07-13T22:26:17.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.231 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2023-07-13T22:26:17.231 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2023-07-13T22:26:17.232 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:17.232 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2023-07-13T22:26:17.232 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2023-07-13T22:26:17.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.232 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2023-07-13T22:26:17.233 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 6 2023-07-13T22:26:17.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.233 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2023-07-13T22:26:17.233 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:17.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:17.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2023-07-13T22:26:17.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:17.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2023-07-13T22:26:17.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:17.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.235 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:17.235 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:17.235 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:17.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:17.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 1365, 2023-07-13T22:26:17.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 746282, 2023-07-13T22:26:17.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:17.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 1365, 2023-07-13T22:26:17.237 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:17.237 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:17.237 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 1365, 2023-07-13T22:26:17.237 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 746282, 2023-07-13T22:26:17.237 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:17.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.238 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.238 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.239 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:17.239 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:17.239 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:17.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:17.240 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 41, 2023-07-13T22:26:17.241 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 12010, 2023-07-13T22:26:17.241 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 41, 2023-07-13T22:26:17.241 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 12010, 2023-07-13T22:26:17.241 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:17.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.242 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.242 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.243 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:17.243 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:17.243 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2023-07-13T22:26:17.243 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 41, 2023-07-13T22:26:17.244 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 41, 2023-07-13T22:26:17.245 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 41, 2023-07-13T22:26:17.245 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 41, 2023-07-13T22:26:17.245 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:17.245 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.245 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:17.246 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:17.247 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x5589619e80a0": { 2023-07-13T22:26:17.248 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 4, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1387, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 4, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:17.249 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 4, 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1387, 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.250 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:17.251 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-13T22:26:17.483 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-13T22:26:17.483 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-13T22:26:17.483 DEBUG:tasks.check_counter:Getting stats from i 2023-07-13T22:26:17.484 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-13T22:26:18.035 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:26:18.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:17 smithi117 ceph-mon[129074]: pgmap v687: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:18.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:17 smithi026 ceph-mon[95647]: pgmap v687: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:18.205 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:17 smithi121 ceph-mon[129187]: pgmap v687: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:19.123 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-13T22:26:19.123 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:19.123 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1788, 2023-07-13T22:26:19.123 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1777, 2023-07-13T22:26:19.123 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 5197664, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 611129, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 1, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.162637148, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.048536710, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.050461535, 2023-07-13T22:26:19.124 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.051373751, 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1776, 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.056691210, 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000031920 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:19.125 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 532496, 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 132496 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.126 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 252, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 426, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 62578, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 452479, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.029902088, 2023-07-13T22:26:19.127 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.012894030, 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.005660252, 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.008183602, 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 424, 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.018236965, 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000043011 2023-07-13T22:26:19.128 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:19.129 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 374352 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 876, 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 876, 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 2211723, 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 271647, 2023-07-13T22:26:19.130 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 4, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 3, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.070725079, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.024358797, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.016002369, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.024386194, 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:19.131 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 875, 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.029262941, 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000033443 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.132 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2023-07-13T22:26:19.133 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4444, 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.111146374, 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000025010 2023-07-13T22:26:19.134 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.135 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2023-07-13T22:26:19.136 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:19.137 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 52, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 0, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 52, 2023-07-13T22:26:19.138 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 52, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 2, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2023-07-13T22:26:19.139 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2023-07-13T22:26:19.140 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:19.141 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 4, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 246, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:19.142 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:19.143 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.144 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2023-07-13T22:26:19.145 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:19.146 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2023-07-13T22:26:19.147 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2023-07-13T22:26:19.148 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 3, 2023-07-13T22:26:19.149 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 0, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 10377519, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.150 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 1695 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 16, 2023-07-13T22:26:19.151 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 36, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 20, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 11, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 13, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 2, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 52, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 55, 2023-07-13T22:26:19.152 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 3, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 49516, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 207156 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.153 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.154 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2023-07-13T22:26:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.157 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.158 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.159 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.160 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.161 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.162 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.163 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.164 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2023-07-13T22:26:19.165 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.166 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.167 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.169 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.170 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.171 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.172 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.173 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:19.174 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2023-07-13T22:26:19.175 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2023-07-13T22:26:19.176 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2544 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:19.177 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:19.178 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:19.179 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:19.180 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:19.181 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 189468, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 53, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:19.182 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 158, 2023-07-13T22:26:19.183 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 86111, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 701, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:19.184 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2023-07-13T22:26:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 2662, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 26, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 4, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 2662, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2662, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1.329981842, 2023-07-13T22:26:19.186 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000499617 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2662, 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2663 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 2662, 2023-07-13T22:26:19.187 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 2662, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 1655, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 1004, 2023-07-13T22:26:19.188 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2023-07-13T22:26:19.189 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 2, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:19.190 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2023-07-13T22:26:19.191 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2023-07-13T22:26:19.192 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2023-07-13T22:26:19.193 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 85, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 5, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 5, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2023-07-13T22:26:19.194 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2023-07-13T22:26:19.195 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.196 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.197 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 2912, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 7194174, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:19.198 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 2912, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2912, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 7194174, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.199 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:19.200 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2662, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 6191595, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2662, 2023-07-13T22:26:19.201 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 6191595, 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:19.203 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2662, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 2662, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2662, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 2662, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.204 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:19.205 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:19.206 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:19.207 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x565048730460": { 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:19.208 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:19.209 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:19.210 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:19.210 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.210 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:19.210 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-13T22:26:19.479 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-07-13T22:26:19.479 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-07-13T22:26:19.479 DEBUG:tasks.check_counter:Getting stats from l 2023-07-13T22:26:19.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3fc0089cc1082313a98853d48b892acdc1385b8b shell --fsid 23b26990-21c9-11ee-9b32-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-13T22:26:20.031 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/mon.c/config 2023-07-13T22:26:20.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:19 smithi117 ceph-mon[129074]: pgmap v688: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:26:20.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:19 smithi026 ceph-mon[95647]: pgmap v688: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:26:20.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:19 smithi121 ceph-mon[129187]: pgmap v688: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2023-07-13T22:26:21.278 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2023-07-13T22:26:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2023-07-13T22:26:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 866, 2023-07-13T22:26:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 866, 2023-07-13T22:26:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 291437, 2023-07-13T22:26:21.279 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 268531, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.086250419, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.025037603, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.021865760, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.030087507, 2023-07-13T22:26:21.280 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 865, 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.042411750, 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000049030 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.281 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 753, 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 931, 2023-07-13T22:26:21.282 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 185804, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 608996, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.068859348, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.026391187, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.012901770, 2023-07-13T22:26:21.283 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.022882417, 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 929, 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.027663684, 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000029777 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.284 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 375056 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2023-07-13T22:26:21.285 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1060, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1053, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 724571, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 387517, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 3, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.089448079, 2023-07-13T22:26:21.286 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.030331867, 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.028195396, 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.024925086, 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1052, 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.031989563, 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000030408 2023-07-13T22:26:21.287 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.288 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 510992, 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 135296 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.289 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4044, 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.106644962, 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000026371 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.290 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.291 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.292 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2023-07-13T22:26:21.293 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2023-07-13T22:26:21.294 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 1, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2023-07-13T22:26:21.295 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2023-07-13T22:26:21.296 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2023-07-13T22:26:21.297 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2023-07-13T22:26:21.298 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2023-07-13T22:26:21.299 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2023-07-13T22:26:21.300 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2023-07-13T22:26:21.301 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2023-07-13T22:26:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 0, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2023-07-13T22:26:21.303 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 1, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2023-07-13T22:26:21.304 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 0, 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 4195375, 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.305 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 4 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 3, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 3, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 0, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 1, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 1, 2023-07-13T22:26:21.306 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2023-07-13T22:26:21.307 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 45884, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 207156 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2023-07-13T22:26:21.308 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2023-07-13T22:26:21.309 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.310 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.312 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.313 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.314 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.315 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.316 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.317 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.318 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.319 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.320 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.321 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.322 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.323 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2023-07-13T22:26:21.324 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.325 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.326 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.327 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.328 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2023-07-13T22:26:21.329 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2545 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2023-07-13T22:26:21.330 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-13T22:26:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2023-07-13T22:26:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2023-07-13T22:26:21.333 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2023-07-13T22:26:21.334 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 189383, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 50, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2023-07-13T22:26:21.335 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7528, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 158, 2023-07-13T22:26:21.336 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 8320, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 13, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2023-07-13T22:26:21.337 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 2425, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2023-07-13T22:26:21.338 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 2425, 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2425, 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1.330981745, 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000548858 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.339 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2425, 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2426 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 2425, 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 2425, 2023-07-13T22:26:21.340 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 1583, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 840, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2023-07-13T22:26:21.341 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2023-07-13T22:26:21.342 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2023-07-13T22:26:21.343 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2023-07-13T22:26:21.344 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2023-07-13T22:26:21.345 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 85, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2023-07-13T22:26:21.346 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 5, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 5, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2023-07-13T22:26:21.348 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2023-07-13T22:26:21.349 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 2675, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 945356, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:21.350 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 2675, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2675, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 945356, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.351 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:21.352 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2425, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 1392, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2425, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1392, 2023-07-13T22:26:21.353 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2023-07-13T22:26:21.354 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:21.355 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 2425, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 2425, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2425, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 2425, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.356 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:21.357 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.358 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x5629b6e02460": { 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2023-07-13T22:26:21.359 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2023-07-13T22:26:21.360 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout: } 2023-07-13T22:26:21.361 INFO:teuthology.orchestra.run.smithi121.stdout:} 2023-07-13T22:26:21.584 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-07-13T22:26:21.584 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-07-13T22:26:21.585 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-13T22:26:21.600 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-13T22:26:21.600 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:26:21.601 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:26:21.651 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-07-13T22:26:21.652 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-13T22:26:21.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:26:21.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.968+0000 7f63e27ff700 1 -- 172.21.15.26:0/560802036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc105810 msgr2=0x7f63dc107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:21.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.968+0000 7f63e27ff700 1 --2- 172.21.15.26:0/560802036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc105810 0x7f63dc107bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f63d400b0a0 tx=0x7f63d400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 -- 172.21.15.26:0/560802036 shutdown_connections 2023-07-13T22:26:21.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 --2- 172.21.15.26:0/560802036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc105810 0x7f63dc107bf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 --2- 172.21.15.26:0/560802036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 --2- 172.21.15.26:0/560802036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc102ef0 0x7f63dc1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 -- 172.21.15.26:0/560802036 >> 172.21.15.26:0/560802036 conn(0x7f63dc0f9fd0 msgr2=0x7f63dc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 -- 172.21.15.26:0/560802036 shutdown_connections 2023-07-13T22:26:21.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.969+0000 7f63e27ff700 1 -- 172.21.15.26:0/560802036 wait complete. 2023-07-13T22:26:21.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 Processor -- start 2023-07-13T22:26:21.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 -- start start 2023-07-13T22:26:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 0x7f63dc195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:21.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 0x7f63dc19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19d470 con 0x7f63dc102ef0 2023-07-13T22:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19d5b0 con 0x7f63dc1005d0 2023-07-13T22:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.970+0000 7f63e27ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63dc19d740 con 0x7f63dc105810 2023-07-13T22:26:21.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63db7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 0x7f63dc195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:58420/0 (socket says 172.21.15.26:58420) 2023-07-13T22:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 -- 172.21.15.26:0/593868210 learned_addr learned my addr 172.21.15.26:0/593868210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:21.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63e0d9c700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 0x7f63dc19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 -- 172.21.15.26:0/593868210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 msgr2=0x7f63dc19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 0x7f63dc19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 -- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 msgr2=0x7f63dc195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 0x7f63dc195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 -- 172.21.15.26:0/593868210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4009d00 con 0x7f63dc1005d0 2023-07-13T22:26:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63e0d9c700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 0x7f63dc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63db7fe700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 0x7f63dc195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63dbfff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00d950 tx=0x7f63cc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.971+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009940 con 0x7f63dc1005d0 2023-07-13T22:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.972+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63cc010460 con 0x7f63dc1005d0 2023-07-13T22:26:21.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.972+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63dc19da20 con 0x7f63dc1005d0 2023-07-13T22:26:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.972+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00f620 con 0x7f63dc1005d0 2023-07-13T22:26:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.972+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63dc19df80 con 0x7f63dc1005d0 2023-07-13T22:26:21.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.973+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63dc0fbb40 con 0x7f63dc1005d0 2023-07-13T22:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.975+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f63cc0108d0 con 0x7f63dc1005d0 2023-07-13T22:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.976+0000 7f63d97fa700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 0x7f63c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:21.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.977+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f63cc0cc760 con 0x7f63dc1005d0 2023-07-13T22:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.977+0000 7f63db7fe700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 0x7f63c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.977+0000 7f63db7fe700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 0x7f63c40a36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011490 tx=0x7f63d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:21.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:21.979+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f63cc095e60 con 0x7f63dc1005d0 2023-07-13T22:26:22.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.188+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63dc02c850 con 0x7f63dc1005d0 2023-07-13T22:26:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.191+0000 7f63d97fa700 1 -- 172.21.15.26:0/593868210 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7f63cc020090 con 0x7f63dc1005d0 2023-07-13T22:26:22.172 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:26:22.172 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:26:22.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 msgr2=0x7f63c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 0x7f63c40a36d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011490 tx=0x7f63d0009260 comp rx=0 tx=0).stop 2023-07-13T22:26:22.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 msgr2=0x7f63dc1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00d950 tx=0x7f63cc00dc60 comp rx=0 tx=0).stop 2023-07-13T22:26:22.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 shutdown_connections 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63dc105810 0x7f63dc19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63dc1005d0 0x7f63dc1958f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f63c40a1220 0x7f63c40a36d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 --2- 172.21.15.26:0/593868210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63dc102ef0 0x7f63dc195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 >> 172.21.15.26:0/593868210 conn(0x7f63dc0f9fd0 msgr2=0x7f63dc101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.192+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 shutdown_connections 2023-07-13T22:26:22.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.193+0000 7f63e27ff700 1 -- 172.21.15.26:0/593868210 wait complete. 2023-07-13T22:26:22.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:21 smithi117 ceph-mon[129074]: pgmap v689: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:22.190 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:21 smithi026 ceph-mon[95647]: pgmap v689: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:22.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-13T22:08:09.955297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-13T22:12:16.912382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-13T22:12:17.951760+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-13T22:26:22.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-13T22:26:22.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:21 smithi121 ceph-mon[129187]: pgmap v689: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2023-07-13T22:26:22.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 -- 172.21.15.26:0/4278781580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 msgr2=0x7fe9740733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/4278781580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe9740733e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe9640098f0 tx=0x7fe964009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:22.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 -- 172.21.15.26:0/4278781580 shutdown_connections 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/4278781580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe9740729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/4278781580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 0x7fe97410e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/4278781580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe9740733e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.419+0000 7fe97c5ff700 1 -- 172.21.15.26:0/4278781580 >> 172.21.15.26:0/4278781580 conn(0x7fe9740fb0e0 msgr2=0x7fe9740fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.420+0000 7fe97c5ff700 1 -- 172.21.15.26:0/4278781580 shutdown_connections 2023-07-13T22:26:22.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.420+0000 7fe97c5ff700 1 -- 172.21.15.26:0/4278781580 wait complete. 2023-07-13T22:26:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.421+0000 7fe97c5ff700 1 Processor -- start 2023-07-13T22:26:22.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.421+0000 7fe97c5ff700 1 -- start start 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe97419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 0x7fe9741a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a3ad0 con 0x7fe974074550 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a3c10 con 0x7fe97410c0b0 2023-07-13T22:26:22.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97c5ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741a3da0 con 0x7fe974072f70 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe979b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe97419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe979b9a700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe97419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56354/0 (socket says 172.21.15.26:56354) 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:53424/0 (socket says 172.21.15.26:53424) 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe979b9a700 1 -- 172.21.15.26:0/233075210 learned_addr learned my addr 172.21.15.26:0/233075210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:22.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97ab9c700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 0x7fe9741a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 -- 172.21.15.26:0/233075210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 msgr2=0x7fe9741a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 0x7fe9741a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 -- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 msgr2=0x7fe97419c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe97419c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.422+0000 7fe97a39b700 1 -- 172.21.15.26:0/233075210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe970009520 con 0x7fe974072f70 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe97a39b700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe97000e9c0 tx=0x7fe97000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97000cb70 con 0x7fe974072f70 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe964009520 con 0x7fe974072f70 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97000ee60 con 0x7fe974072f70 2023-07-13T22:26:22.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970003c60 con 0x7fe974072f70 2023-07-13T22:26:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.423+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9741a4450 con 0x7fe974072f70 2023-07-13T22:26:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.424+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe974101c00 con 0x7fe974072f70 2023-07-13T22:26:22.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.426+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe970005030 con 0x7fe974072f70 2023-07-13T22:26:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.428+0000 7fe96b7fe700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 0x7fe9600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.428+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe970014070 con 0x7fe974072f70 2023-07-13T22:26:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.428+0000 7fe979b9a700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 0x7fe9600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.429+0000 7fe979b9a700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 0x7fe9600a3680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c010 tx=0x7fe964019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.430+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe9700968c0 con 0x7fe974072f70 2023-07-13T22:26:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.639+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe97402cac0 con 0x7fe974072f70 2023-07-13T22:26:22.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.640+0000 7fe96b7fe700 1 -- 172.21.15.26:0/233075210 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe970099ee0 con 0x7fe974072f70 2023-07-13T22:26:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-07-13T22:26:22.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 msgr2=0x7fe9600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 0x7fe9600a3680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c010 tx=0x7fe964019040 comp rx=0 tx=0).stop 2023-07-13T22:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 msgr2=0x7fe97419bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe97000e9c0 tx=0x7fe97000ecd0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 shutdown_connections 2023-07-13T22:26:22.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe97410c0b0 0x7fe9741a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974072f70 0x7fe97419bf50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fe9600a11d0 0x7fe9600a3680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 --2- 172.21.15.26:0/233075210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe974074550 0x7fe97419c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 >> 172.21.15.26:0/233075210 conn(0x7fe9740fb0e0 msgr2=0x7fe9740fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 shutdown_connections 2023-07-13T22:26:22.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.643+0000 7fe97c5ff700 1 -- 172.21.15.26:0/233075210 wait complete. 2023-07-13T22:26:22.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-13T22:26:22.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 -- 172.21.15.26:0/112017149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 msgr2=0x7f3fa00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/112017149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00ffef0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009990 tx=0x7f3f90009ca0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 -- 172.21.15.26:0/112017149 shutdown_connections 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/112017149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0101660 0x7f3fa0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/112017149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00ffef0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/112017149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 -- 172.21.15.26:0/112017149 >> 172.21.15.26:0/112017149 conn(0x7f3fa00fb0e0 msgr2=0x7f3fa00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 -- 172.21.15.26:0/112017149 shutdown_connections 2023-07-13T22:26:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.870+0000 7f3fa5fff700 1 -- 172.21.15.26:0/112017149 wait complete. 2023-07-13T22:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.871+0000 7f3fa5fff700 1 Processor -- start 2023-07-13T22:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.871+0000 7f3fa5fff700 1 -- start start 2023-07-13T22:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0101660 0x7f3fa0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa00735e0 con 0x7f3fa0100cd0 2023-07-13T22:26:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa0073750 con 0x7f3fa00ffae0 2023-07-13T22:26:22.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3fa5fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa019f8f0 con 0x7f3fa0101660 2023-07-13T22:26:22.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56364/0 (socket says 172.21.15.26:56364) 2023-07-13T22:26:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 -- 172.21.15.26:0/3114502848 learned_addr learned my addr 172.21.15.26:0/3114502848 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9ffff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0101660 0x7f3fa0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 -- 172.21.15.26:0/3114502848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0101660 msgr2=0x7f3fa0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0101660 0x7f3fa0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 -- 172.21.15.26:0/3114502848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 msgr2=0x7f3fa00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.872+0000 7f3f9effd700 1 -- 172.21.15.26:0/3114502848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900095f0 con 0x7f3fa0100cd0 2023-07-13T22:26:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3f9f7fe700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3f9effd700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e9a0 tx=0x7f3f9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94009790 con 0x7f3fa0100cd0 2023-07-13T22:26:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa019fbd0 con 0x7f3fa0100cd0 2023-07-13T22:26:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f94010430 con 0x7f3fa0100cd0 2023-07-13T22:26:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f94018680 con 0x7f3fa0100cd0 2023-07-13T22:26:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.873+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa01a0130 con 0x7f3fa0100cd0 2023-07-13T22:26:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.874+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa0106a60 con 0x7f3fa0100cd0 2023-07-13T22:26:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.876+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3f940105a0 con 0x7f3fa0100cd0 2023-07-13T22:26:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.877+0000 7f3f9cff9700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 0x7f3f880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.878+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3f94014070 con 0x7f3fa0100cd0 2023-07-13T22:26:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.878+0000 7f3f9f7fe700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 0x7f3f880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.878+0000 7f3f9f7fe700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 0x7f3f880a3600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b2e0 tx=0x7f3f90005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:22.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:22.882+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f94051790 con 0x7f3fa0100cd0 2023-07-13T22:26:22.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/593868210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:22.943 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:22 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/233075210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:26:23.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.097+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fa00611c0 con 0x7f3fa0100cd0 2023-07-13T22:26:23.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.097+0000 7f3f9cff9700 1 -- 172.21.15.26:0/3114502848 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f94096210 con 0x7f3fa0100cd0 2023-07-13T22:26:23.079 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-07-13T22:26:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.099+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 msgr2=0x7f3f880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.099+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 0x7f3f880a3600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000b2e0 tx=0x7f3f90005fb0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.099+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 msgr2=0x7f3fa0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.099+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e9a0 tx=0x7f3f9400ecb0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 shutdown_connections 2023-07-13T22:26:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fa0101660 0x7f3fa0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fa00ffae0 0x7f3fa00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f3f880a1150 0x7f3f880a3600 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 --2- 172.21.15.26:0/3114502848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa0100cd0 0x7f3fa0072900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 >> 172.21.15.26:0/3114502848 conn(0x7f3fa00fb0e0 msgr2=0x7f3fa00fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 shutdown_connections 2023-07-13T22:26:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.100+0000 7f3fa5fff700 1 -- 172.21.15.26:0/3114502848 wait complete. 2023-07-13T22:26:23.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-13T22:26:23.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/593868210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:23.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:22 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/233075210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:26:23.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/593868210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:23.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:22 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/233075210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-13T22:26:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.325+0000 7f70aefff700 1 -- 172.21.15.26:0/1120747584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a80ffb10 msgr2=0x7f70a80fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.325+0000 7f70aefff700 1 --2- 172.21.15.26:0/1120747584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a80ffb10 0x7f70a80fff20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098f0 tx=0x7f709c009c00 comp rx=0 tx=0).stop 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 -- 172.21.15.26:0/1120747584 shutdown_connections 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 --2- 172.21.15.26:0/1120747584 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8101690 0x7f70a8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 --2- 172.21.15.26:0/1120747584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a80ffb10 0x7f70a80fff20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 --2- 172.21.15.26:0/1120747584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a8100d00 0x7f70a8101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 -- 172.21.15.26:0/1120747584 >> 172.21.15.26:0/1120747584 conn(0x7f70a80fb130 msgr2=0x7f70a80fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 -- 172.21.15.26:0/1120747584 shutdown_connections 2023-07-13T22:26:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.326+0000 7f70aefff700 1 -- 172.21.15.26:0/1120747584 wait complete. 2023-07-13T22:26:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 Processor -- start 2023-07-13T22:26:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 -- start start 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a80ffb10 0x7f70a80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 0x7f70a8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80735e0 con 0x7f70a80ffb10 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8073750 con 0x7f70a8100d00 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.327+0000 7f70aefff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a819f890 con 0x7f70a8101690 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70acd9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a80ffb10 0x7f70a80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52356/0 (socket says 172.21.15.26:52356) 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 -- 172.21.15.26:0/1041005333 learned_addr learned my addr 172.21.15.26:0/1041005333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70ad59c700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 0x7f70a8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 -- 172.21.15.26:0/1041005333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 msgr2=0x7f70a8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 0x7f70a8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 -- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a80ffb10 msgr2=0x7f70a80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a80ffb10 0x7f70a80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 -- 172.21.15.26:0/1041005333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940095f0 con 0x7f70a8100d00 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70ad59c700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 0x7f70a8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.328+0000 7f70a7fff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f709400ec10 tx=0x7f709400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.329+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709400cd40 con 0x7f70a8100d00 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.329+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70940044e0 con 0x7f70a8100d00 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.329+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c009520 con 0x7f70a8100d00 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.329+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094010690 con 0x7f70a8100d00 2023-07-13T22:26:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.329+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a819ff40 con 0x7f70a8100d00 2023-07-13T22:26:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.330+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a810b280 con 0x7f70a8100d00 2023-07-13T22:26:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.332+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7094003710 con 0x7f70a8100d00 2023-07-13T22:26:23.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.333+0000 7f70a5ffb700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 0x7f70980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.334+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f7094014070 con 0x7f70a8100d00 2023-07-13T22:26:23.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.334+0000 7f70acd9b700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 0x7f70980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.334+0000 7f70acd9b700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 0x7f70980a3680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098c0 tx=0x7f709c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:23.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.336+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7094096b50 con 0x7f70a8100d00 2023-07-13T22:26:23.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.547+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70a8106a30 con 0x7f70a8100d00 2023-07-13T22:26:23.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.547+0000 7f70a5ffb700 1 -- 172.21.15.26:0/1041005333 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f709409a170 con 0x7f70a8100d00 2023-07-13T22:26:23.529 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-07-13T22:26:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 msgr2=0x7f70980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 0x7f70980a3680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098c0 tx=0x7f709c019040 comp rx=0 tx=0).stop 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 msgr2=0x7f70a8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f709400ec10 tx=0x7f709400ef20 comp rx=0 tx=0).stop 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 shutdown_connections 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f70980a11d0 0x7f70980a3680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a80ffb10 0x7f70a80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a8101690 0x7f70a8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 --2- 172.21.15.26:0/1041005333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a8100d00 0x7f70a8072900 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 >> 172.21.15.26:0/1041005333 conn(0x7f70a80fb130 msgr2=0x7f70a80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 shutdown_connections 2023-07-13T22:26:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.550+0000 7f70aefff700 1 -- 172.21.15.26:0/1041005333 wait complete. 2023-07-13T22:26:23.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-13T22:26:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 -- 172.21.15.26:0/1287102159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7041079a0 msgr2=0x7fd704109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/1287102159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7041079a0 0x7fd704109e20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00b0a0 tx=0x7fd6fc00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 -- 172.21.15.26:0/1287102159 shutdown_connections 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/1287102159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7041079a0 0x7fd704109e20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/1287102159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704074550 0x7fd7040729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/1287102159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd704072f70 0x7fd7040733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 -- 172.21.15.26:0/1287102159 >> 172.21.15.26:0/1287102159 conn(0x7fd7040fb0e0 msgr2=0x7fd7040fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 -- 172.21.15.26:0/1287102159 shutdown_connections 2023-07-13T22:26:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.774+0000 7fd70a9ff700 1 -- 172.21.15.26:0/1287102159 wait complete. 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.776+0000 7fd70a9ff700 1 Processor -- start 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.776+0000 7fd70a9ff700 1 -- start start 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.776+0000 7fd70a9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd70a9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 0x7fd7041980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd70a9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7041079a0 0x7fd70419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd70a9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419f6e0 con 0x7fd704074550 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd70a9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419f820 con 0x7fd704072f70 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd70a9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70419f9b0 con 0x7fd7041079a0 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd7037fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 0x7fd7041980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.26:52364/0 (socket says 172.21.15.26:52364) 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 -- 172.21.15.26:0/33255831 learned_addr learned my addr 172.21.15.26:0/33255831 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd708f9c700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7041079a0 0x7fd70419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 -- 172.21.15.26:0/33255831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7041079a0 msgr2=0x7fd70419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7041079a0 0x7fd70419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 -- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 msgr2=0x7fd7041980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 0x7fd7041980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd703fff700 1 -- 172.21.15.26:0/33255831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc009d00 con 0x7fd704072f70 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.777+0000 7fd7037fe700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 0x7fd7041980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd703fff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e970 tx=0x7fd6f400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4009790 con 0x7fd704072f70 2023-07-13T22:26:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70419fc90 con 0x7fd704072f70 2023-07-13T22:26:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6f400c7e0 con 0x7fd704072f70 2023-07-13T22:26:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40054e0 con 0x7fd704072f70 2023-07-13T22:26:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.778+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7041a01a0 con 0x7fd704072f70 2023-07-13T22:26:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.779+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd704106050 con 0x7fd704072f70 2023-07-13T22:26:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.781+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd6f400c950 con 0x7fd704072f70 2023-07-13T22:26:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.782+0000 7fd7017fa700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 0x7fd6ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.783+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd6f4014070 con 0x7fd704072f70 2023-07-13T22:26:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.783+0000 7fd7037fe700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 0x7fd6ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.783+0000 7fd7037fe700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 0x7fd6ec0a3600 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd704074270 tx=0x7fd6f8007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:23.784+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd6f4096470 con 0x7fd704072f70 2023-07-13T22:26:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.011+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd70402c850 con 0x7fd704072f70 2023-07-13T22:26:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.012+0000 7fd7017fa700 1 -- 172.21.15.26:0/33255831 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+591 (secure 0 0 0) 0x7fd6f4099a90 con 0x7fd704072f70 2023-07-13T22:26:23.994 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1422542320 2023-07-14T22:05:51.339234+0000 2023-07-13T22:26:23.994 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/3258657934 2023-07-14T22:05:51.339234+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2685973797 2023-07-14T22:05:24.923026+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/551567635 2023-07-14T22:05:24.923026+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/2707453820 2023-07-14T22:05:51.339234+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/2707453820 2023-07-14T22:05:51.339234+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/2272047376 2023-07-14T22:05:51.339234+0000 2023-07-13T22:26:23.995 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6800/3558776409 2023-07-14T22:05:24.923026+0000 2023-07-13T22:26:23.996 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6801/3558776409 2023-07-14T22:05:24.923026+0000 2023-07-13T22:26:23.996 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:0/1729376159 2023-07-14T22:05:24.923026+0000 2023-07-13T22:26:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 msgr2=0x7fd6ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 0x7fd6ec0a3600 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd704074270 tx=0x7fd6f8007400 comp rx=0 tx=0).stop 2023-07-13T22:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 msgr2=0x7fd704197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e970 tx=0x7fd6f400ec80 comp rx=0 tx=0).stop 2023-07-13T22:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 shutdown_connections 2023-07-13T22:26:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7041079a0 0x7fd70419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd704072f70 0x7fd704197b60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fd6ec0a1150 0x7fd6ec0a3600 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 --2- 172.21.15.26:0/33255831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd704074550 0x7fd7041980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 >> 172.21.15.26:0/33255831 conn(0x7fd7040fb0e0 msgr2=0x7fd7040fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.014+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 shutdown_connections 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.015+0000 7fd70a9ff700 1 -- 172.21.15.26:0/33255831 wait complete. 2023-07-13T22:26:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2023-07-13T22:26:24.015 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-13T22:26:24.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:26:24.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-13T22:26:24.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:23 smithi026 ceph-mon[95647]: pgmap v690: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:24.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3114502848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.150 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:23 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1041005333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T22:26:24.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:26:24.151 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-13T22:26:24.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:23 smithi117 ceph-mon[129074]: pgmap v690: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:24.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3114502848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:23 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1041005333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:26:24.205 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-07-13T22:26:24.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:23 smithi121 ceph-mon[129187]: pgmap v690: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-07-13T22:26:24.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3114502848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:23 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1041005333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-13T22:26:24.249 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-13T22:26:24.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:26:24.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-13T22:26:24.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-13T22:26:24.360 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-07-13T22:26:24.474 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-13T22:26:24.486 INFO:tasks.cephadm:Teardown begin 2023-07-13T22:26:24.486 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-13T22:26:24.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:26:24.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 -- 172.21.15.26:0/2961113229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7300ffb00 msgr2=0x7fc7300fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:24.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 --2- 172.21.15.26:0/2961113229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7300ffb00 0x7fc7300fff10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc720009990 tx=0x7fc720009ca0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 -- 172.21.15.26:0/2961113229 shutdown_connections 2023-07-13T22:26:24.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 --2- 172.21.15.26:0/2961113229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7300ffb00 0x7fc7300fff10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 --2- 172.21.15.26:0/2961113229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc730101680 0x7fc730107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 --2- 172.21.15.26:0/2961113229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 0x7fc730101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 -- 172.21.15.26:0/2961113229 >> 172.21.15.26:0/2961113229 conn(0x7fc7300fb0e0 msgr2=0x7fc7300fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 -- 172.21.15.26:0/2961113229 shutdown_connections 2023-07-13T22:26:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.715+0000 7fc7365ff700 1 -- 172.21.15.26:0/2961113229 wait complete. 2023-07-13T22:26:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.716+0000 7fc7365ff700 1 Processor -- start 2023-07-13T22:26:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.716+0000 7fc7365ff700 1 -- start start 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.716+0000 7fc7365ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 0x7fc730197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc7365ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 0x7fc730197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc7365ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc7365ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019f590 con 0x7fc730101680 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc7365ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019f6d0 con 0x7fc730100cf0 2023-07-13T22:26:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc7365ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73019f860 con 0x7fc7300ffb00 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35324/0 (socket says 172.21.15.26:35324) 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 -- 172.21.15.26:0/258643739 learned_addr learned my addr 172.21.15.26:0/258643739 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc72f7fe700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 0x7fc730197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc72ffff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 0x7fc730197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 -- 172.21.15.26:0/258643739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 msgr2=0x7fc730197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:24.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 0x7fc730197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 -- 172.21.15.26:0/258643739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 msgr2=0x7fc730197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 0x7fc730197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc734b9c700 1 -- 172.21.15.26:0/258643739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7200095f0 con 0x7fc730101680 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.717+0000 7fc72ffff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 0x7fc730197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc734b9c700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc72800ef30 tx=0x7fc72800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728018070 con 0x7fc730101680 2023-07-13T22:26:24.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc72800f040 con 0x7fc730101680 2023-07-13T22:26:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73019fae0 con 0x7fc730101680 2023-07-13T22:26:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800cd00 con 0x7fc730101680 2023-07-13T22:26:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.718+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73019fff0 con 0x7fc730101680 2023-07-13T22:26:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.719+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73010b320 con 0x7fc730101680 2023-07-13T22:26:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.721+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc728003a00 con 0x7fc730101680 2023-07-13T22:26:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.723+0000 7fc72d7fa700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 0x7fc7180a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.723+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc7280cc480 con 0x7fc730101680 2023-07-13T22:26:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.723+0000 7fc72ffff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 0x7fc7180a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.724+0000 7fc72ffff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 0x7fc7180a3650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc72000b2e0 tx=0x7fc720005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.726+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc728095c10 con 0x7fc730101680 2023-07-13T22:26:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.984+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc73010b4b0 con 0x7fc730101680 2023-07-13T22:26:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.985+0000 7fc72d7fa700 1 -- 172.21.15.26:0/258643739 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10892 (secure 0 0 0) 0x7fc728099230 con 0x7fc730101680 2023-07-13T22:26:24.967 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:26:24.967 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:25:15.677742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:26:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 msgr2=0x7fc7180a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 0x7fc7180a3650 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc72000b2e0 tx=0x7fc720005fb0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 msgr2=0x7fc73019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:24.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc72800ef30 tx=0x7fc72800c3a0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 shutdown_connections 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fc7180a11a0 0x7fc7180a3650 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc730101680 0x7fc73019efc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7300ffb00 0x7fc730197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 --2- 172.21.15.26:0/258643739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc730100cf0 0x7fc730197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.987+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 >> 172.21.15.26:0/258643739 conn(0x7fc7300fb0e0 msgr2=0x7fc730106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.988+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 shutdown_connections 2023-07-13T22:26:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:24.988+0000 7fc7365ff700 1 -- 172.21.15.26:0/258643739 wait complete. 2023-07-13T22:26:24.971 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2023-07-13T22:26:24.989 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-13T22:26:25.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:24 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/33255831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-13T22:26:25.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:24 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/33255831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-13T22:26:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.212+0000 7f5503fff700 1 -- 172.21.15.26:0/1751528109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc100cf0 msgr2=0x7f54fc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.212+0000 7f5503fff700 1 --2- 172.21.15.26:0/1751528109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc100cf0 0x7f54fc101140 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009990 tx=0x7f54ec009ca0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 -- 172.21.15.26:0/1751528109 shutdown_connections 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 --2- 172.21.15.26:0/1751528109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 0x7f54fc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 --2- 172.21.15.26:0/1751528109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc0ffb00 0x7f54fc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 --2- 172.21.15.26:0/1751528109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc100cf0 0x7f54fc101140 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 -- 172.21.15.26:0/1751528109 >> 172.21.15.26:0/1751528109 conn(0x7f54fc0fb0e0 msgr2=0x7f54fc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 -- 172.21.15.26:0/1751528109 shutdown_connections 2023-07-13T22:26:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.213+0000 7f5503fff700 1 -- 172.21.15.26:0/1751528109 wait complete. 2023-07-13T22:26:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.214+0000 7f5503fff700 1 Processor -- start 2023-07-13T22:26:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.214+0000 7f5503fff700 1 -- start start 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 0x7f54fc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 0x7f54fc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f590 con 0x7f54fc0ffb00 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f6d0 con 0x7f54fc101680 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5503fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f860 con 0x7f54fc100cf0 2023-07-13T22:26:25.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5501d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 0x7f54fc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5501d9b700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35346/0 (socket says 172.21.15.26:35346) 2023-07-13T22:26:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5501d9b700 1 -- 172.21.15.26:0/2865074724 learned_addr learned my addr 172.21.15.26:0/2865074724 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550259c700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 0x7f54fc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 -- 172.21.15.26:0/2865074724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 msgr2=0x7f54fc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 0x7f54fc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 -- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 msgr2=0x7f54fc197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f550159a700 1 -- 172.21.15.26:0/2865074724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0095f0 con 0x7f54fc100cf0 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.215+0000 7f5501d9b700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f550159a700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 0x7f54fc197f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0098e0 tx=0x7f54ec005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec01d070 con 0x7f54fc100cf0 2023-07-13T22:26:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc19fae0 con 0x7f54fc100cf0 2023-07-13T22:26:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ec021410 con 0x7f54fc100cf0 2023-07-13T22:26:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec017650 con 0x7f54fc100cf0 2023-07-13T22:26:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.216+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc19ff00 con 0x7f54fc100cf0 2023-07-13T22:26:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.217+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc10b390 con 0x7f54fc100cf0 2023-07-13T22:26:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.219+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f54ec021580 con 0x7f54fc100cf0 2023-07-13T22:26:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.221+0000 7f54f2ffd700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 0x7f54e80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.221+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f54ec0ce360 con 0x7f54fc100cf0 2023-07-13T22:26:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.221+0000 7f5501d9b700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 0x7f54e80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.222+0000 7f5501d9b700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 0x7f54e80a3650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005950 tx=0x7f54f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.223+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f54ec097af0 con 0x7f54fc100cf0 2023-07-13T22:26:25.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:24 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/33255831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-13T22:26:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.475+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54fc1a04b0 con 0x7f54fc100cf0 2023-07-13T22:26:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.476+0000 7f54f2ffd700 1 -- 172.21.15.26:0/2865074724 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10892 (secure 0 0 0) 0x7f54ec09b110 con 0x7f54fc100cf0 2023-07-13T22:26:25.458 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:26:25.458 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14466,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/956191234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":956191234},{"type":"v1","addr":"172.21.15.26:6835","nonce":956191234}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14478,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6837/3391570568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6836","nonce":3391570568},{"type":"v1","addr":"172.21.15.26:6837","nonce":3391570568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-13T22:12:18.516293+0000","modified":"2023-07-13T22:25:15.677742+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24575,"mds_1":24563,"mds_2":24530,"mds_3":24493,"mds_4":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.26:6839/1019342380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6838","nonce":1019342380},{"type":"v1","addr":"172.21.15.26:6839","nonce":1019342380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6833/930793918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":930793918},{"type":"v1","addr":"172.21.15.121:6833","nonce":930793918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24467":{"gid":24467,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.117:6833/2196716047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2196716047},{"type":"v1","addr":"172.21.15.117:6833","nonce":2196716047}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.121:6835/59384395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":59384395},{"type":"v1","addr":"172.21.15.121:6835","nonce":59384395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/1505729647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1505729647},{"type":"v1","addr":"172.21.15.121:6837","nonce":1505729647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/1862924316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1862924316},{"type":"v1","addr":"172.21.15.117:6835","nonce":1862924316}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6839/3259819874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":3259819874},{"type":"v1","addr":"172.21.15.121:6839","nonce":3259819874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6837/1128080760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1128080760},{"type":"v1","addr":"172.21.15.117:6837","nonce":1128080760}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.26:6841/2542800089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6840","nonce":2542800089},{"type":"v1","addr":"172.21.15.26:6841","nonce":2542800089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24575":{"gid":24575,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.117:6839/1698058160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":1698058160},{"type":"v1","addr":"172.21.15.117:6839","nonce":1698058160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-13T22:26:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 msgr2=0x7f54e80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 0x7f54e80a3650 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f54f8005950 tx=0x7f54f8009260 comp rx=0 tx=0).stop 2023-07-13T22:26:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 msgr2=0x7f54fc197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 0x7f54fc197f50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54ec0098e0 tx=0x7f54ec005740 comp rx=0 tx=0).stop 2023-07-13T22:26:25.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 shutdown_connections 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f54e80a11a0 0x7f54e80a3650 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54fc0ffb00 0x7f54fc197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54fc101680 0x7f54fc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 --2- 172.21.15.26:0/2865074724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54fc100cf0 0x7f54fc197f50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.478+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 >> 172.21.15.26:0/2865074724 conn(0x7f54fc0fb0e0 msgr2=0x7f54fc106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.479+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 shutdown_connections 2023-07-13T22:26:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.479+0000 7f5503fff700 1 -- 172.21.15.26:0/2865074724 wait complete. 2023-07-13T22:26:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 30 2023-07-13T22:26:25.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-13T22:26:25.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 -- 172.21.15.26:0/1191932757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 msgr2=0x7fef18107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/1191932757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef18107f40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fef10009f90 tx=0x7fef1000b330 comp rx=0 tx=0).stop 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 -- 172.21.15.26:0/1191932757 shutdown_connections 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/1191932757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef180fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/1191932757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef18107f40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/1191932757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 0x7fef18101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 -- 172.21.15.26:0/1191932757 >> 172.21.15.26:0/1191932757 conn(0x7fef180fb0e0 msgr2=0x7fef180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 -- 172.21.15.26:0/1191932757 shutdown_connections 2023-07-13T22:26:25.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.705+0000 7fef1c9ff700 1 -- 172.21.15.26:0/1191932757 wait complete. 2023-07-13T22:26:25.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 Processor -- start 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 -- start start 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 0x7fef18197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef1819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819f450 con 0x7fef180ffb00 2023-07-13T22:26:25.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819f590 con 0x7fef18100cf0 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.706+0000 7fef1c9ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1819f720 con 0x7fef18101680 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35374/0 (socket says 172.21.15.26:35374) 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 -- 172.21.15.26:0/3432885750 learned_addr learned my addr 172.21.15.26:0/3432885750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef15d9b700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 0x7fef18197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef16d9d700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef1819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 -- 172.21.15.26:0/3432885750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 msgr2=0x7fef1819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef1819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 -- 172.21.15.26:0/3432885750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 msgr2=0x7fef18197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 0x7fef18197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 -- 172.21.15.26:0/3432885750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009c40 con 0x7fef180ffb00 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.707+0000 7fef1659c700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.708+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08004220 con 0x7fef180ffb00 2023-07-13T22:26:25.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.708+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef08004380 con 0x7fef180ffb00 2023-07-13T22:26:25.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.708+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1819fa00 con 0x7fef180ffb00 2023-07-13T22:26:25.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.708+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08003c00 con 0x7fef180ffb00 2023-07-13T22:26:25.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.708+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1819ff60 con 0x7fef180ffb00 2023-07-13T22:26:25.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.709+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1810b270 con 0x7fef180ffb00 2023-07-13T22:26:25.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.711+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fef0800c7c0 con 0x7fef180ffb00 2023-07-13T22:26:25.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.712+0000 7fef077fe700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 0x7fef000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:25.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.712+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fef08014070 con 0x7fef180ffb00 2023-07-13T22:26:25.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.712+0000 7fef15d9b700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 0x7fef000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:25.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.713+0000 7fef15d9b700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 0x7fef000a36d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009c60 tx=0x7fef0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:25.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.716+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fef08096380 con 0x7fef180ffb00 2023-07-13T22:26:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.928+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef1810b490 con 0x7fef180ffb00 2023-07-13T22:26:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.929+0000 7fef077fe700 1 -- 172.21.15.26:0/3432885750 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22529 (secure 0 0 0) 0x7fef080999a0 con 0x7fef180ffb00 2023-07-13T22:26:25.911 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-07-13T22:26:25.911 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":85,"fsid":"23b26990-21c9-11ee-9b32-001a4aab830c","created":"2023-07-13T22:04:53.734470+0000","modified":"2023-07-13T22:12:52.529361+0000","last_up_change":"2023-07-13T22:11:31.437743+0000","last_in_change":"2023-07-13T22:11:13.852430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-13T22:08:09.955297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-13T22:12:16.912382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-13T22:12:17.951760+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5c5f34fc-c74d-49f7-9f97-f1ab1cb4c40b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6803","nonce":1617573741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6805","nonce":1617573741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6809","nonce":1617573741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":1617573741},{"type":"v1","addr":"172.21.15.26:6807","nonce":1617573741}]},"public_addr":"172.21.15.26:6803/1617573741","cluster_addr":"172.21.15.26:6805/1617573741","heartbeat_back_addr":"172.21.15.26:6809/1617573741","heartbeat_front_addr":"172.21.15.26:6807/1617573741","state":["exists","up"]},{"osd":1,"uuid":"b5fb060a-61be-4007-809f-69c2e19ed034","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6801","nonce":2207798137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6803","nonce":2207798137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6807","nonce":2207798137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2207798137},{"type":"v1","addr":"172.21.15.117:6805","nonce":2207798137}]},"public_addr":"172.21.15.117:6801/2207798137","cluster_addr":"172.21.15.117:6803/2207798137","heartbeat_back_addr":"172.21.15.117:6807/2207798137","heartbeat_front_addr":"172.21.15.117:6805/2207798137","state":["exists","up"]},{"osd":2,"uuid":"79f28811-85d8-4f49-a4eb-c8a30a9b606b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6801","nonce":1910431229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6803","nonce":1910431229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6807","nonce":1910431229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":1910431229},{"type":"v1","addr":"172.21.15.121:6805","nonce":1910431229}]},"public_addr":"172.21.15.121:6801/1910431229","cluster_addr":"172.21.15.121:6803/1910431229","heartbeat_back_addr":"172.21.15.121:6807/1910431229","heartbeat_front_addr":"172.21.15.121:6805/1910431229","state":["exists","up"]},{"osd":3,"uuid":"609be474-a324-4303-b916-e112222f2ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6811","nonce":3949990107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6813","nonce":3949990107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6817","nonce":3949990107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":3949990107},{"type":"v1","addr":"172.21.15.26:6815","nonce":3949990107}]},"public_addr":"172.21.15.26:6811/3949990107","cluster_addr":"172.21.15.26:6813/3949990107","heartbeat_back_addr":"172.21.15.26:6817/3949990107","heartbeat_front_addr":"172.21.15.26:6815/3949990107","state":["exists","up"]},{"osd":4,"uuid":"244086b9-04cf-4892-a23a-ca2d9050394a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6809","nonce":4231906293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6811","nonce":4231906293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6815","nonce":4231906293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":4231906293},{"type":"v1","addr":"172.21.15.117:6813","nonce":4231906293}]},"public_addr":"172.21.15.117:6809/4231906293","cluster_addr":"172.21.15.117:6811/4231906293","heartbeat_back_addr":"172.21.15.117:6815/4231906293","heartbeat_front_addr":"172.21.15.117:6813/4231906293","state":["exists","up"]},{"osd":5,"uuid":"a1497c31-c3f2-4912-b954-a9996a18cfd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6809","nonce":37316284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6811","nonce":37316284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6815","nonce":37316284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":37316284},{"type":"v1","addr":"172.21.15.121:6813","nonce":37316284}]},"public_addr":"172.21.15.121:6809/37316284","cluster_addr":"172.21.15.121:6811/37316284","heartbeat_back_addr":"172.21.15.121:6815/37316284","heartbeat_front_addr":"172.21.15.121:6813/37316284","state":["exists","up"]},{"osd":6,"uuid":"a6b07e8a-bab9-4cf6-8af9-86655ea5b994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6819","nonce":38495361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6821","nonce":38495361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6825","nonce":38495361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38495361},{"type":"v1","addr":"172.21.15.26:6823","nonce":38495361}]},"public_addr":"172.21.15.26:6819/38495361","cluster_addr":"172.21.15.26:6821/38495361","heartbeat_back_addr":"172.21.15.26:6825/38495361","heartbeat_front_addr":"172.21.15.26:6823/38495361","state":["exists","up"]},{"osd":7,"uuid":"102fca76-132a-4dfd-86d3-f8adfc6caf04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6817","nonce":2078940606}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6819","nonce":2078940606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6823","nonce":2078940606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2078940606},{"type":"v1","addr":"172.21.15.117:6821","nonce":2078940606}]},"public_addr":"172.21.15.117:6817/2078940606","cluster_addr":"172.21.15.117:6819/2078940606","heartbeat_back_addr":"172.21.15.117:6823/2078940606","heartbeat_front_addr":"172.21.15.117:6821/2078940606","state":["exists","up"]},{"osd":8,"uuid":"dac2076e-cc3e-47d2-8d8c-267556ea9a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6817","nonce":4171528294}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6819","nonce":4171528294}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6823","nonce":4171528294}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":4171528294},{"type":"v1","addr":"172.21.15.121:6821","nonce":4171528294}]},"public_addr":"172.21.15.121:6817/4171528294","cluster_addr":"172.21.15.121:6819/4171528294","heartbeat_back_addr":"172.21.15.121:6823/4171528294","heartbeat_front_addr":"172.21.15.121:6821/4171528294","state":["exists","up"]},{"osd":9,"uuid":"9a2b112d-b9df-4867-bbfe-7d3455f384ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6827","nonce":411448379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6829","nonce":411448379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6833","nonce":411448379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":411448379},{"type":"v1","addr":"172.21.15.26:6831","nonce":411448379}]},"public_addr":"172.21.15.26:6827/411448379","cluster_addr":"172.21.15.26:6829/411448379","heartbeat_back_addr":"172.21.15.26:6833/411448379","heartbeat_front_addr":"172.21.15.26:6831/411448379","state":["exists","up"]},{"osd":10,"uuid":"a52c43f1-5b23-401c-bea3-05c5bacc3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6825","nonce":1101018481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6827","nonce":1101018481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6831","nonce":1101018481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":1101018481},{"type":"v1","addr":"172.21.15.117:6829","nonce":1101018481}]},"public_addr":"172.21.15.117:6825/1101018481","cluster_addr":"172.21.15.117:6827/1101018481","heartbeat_back_addr":"172.21.15.117:6831/1101018481","heartbeat_front_addr":"172.21.15.117:6829/1101018481","state":["exists","up"]},{"osd":11,"uuid":"43cbee7d-bb99-429b-a99a-102d7497f251","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6825","nonce":1646753631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6827","nonce":1646753631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6831","nonce":1646753631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1646753631},{"type":"v1","addr":"172.21.15.121:6829","nonce":1646753631}]},"public_addr":"172.21.15.121:6825/1646753631","cluster_addr":"172.21.15.121:6827/1646753631","heartbeat_back_addr":"172.21.15.121:6831/1646753631","heartbeat_front_addr":"172.21.15.121:6829/1646753631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:24.973174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:07:46.470072+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:07.450278+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:33.283587+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:08:55.024747+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:15.077566+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:09:40.237304+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:02.264324+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:22.656206+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:10:47.731235+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:09.000434+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-13T22:11:29.793994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:0/1422542320":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/3258657934":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2685973797":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/551567635":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/2707453820":"2023-07-14T22:05:51.339234+0000","172.21.15.26:0/2272047376":"2023-07-14T22:05:51.339234+0000","172.21.15.26:6800/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:6801/3558776409":"2023-07-14T22:05:24.923026+0000","172.21.15.26:0/1729376159":"2023-07-14T22:05:24.923026+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-13T22:26:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.931+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 msgr2=0x7fef000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.931+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 0x7fef000a36d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009c60 tx=0x7fef0c009260 comp rx=0 tx=0).stop 2023-07-13T22:26:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.931+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 msgr2=0x7fef181978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.931+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800ed00 comp rx=0 tx=0).stop 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 shutdown_connections 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7fef000a1220 0x7fef000a36d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef180ffb00 0x7fef181978d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef18101680 0x7fef1819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 --2- 172.21.15.26:0/3432885750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef18100cf0 0x7fef18197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 >> 172.21.15.26:0/3432885750 conn(0x7fef180fb0e0 msgr2=0x7fef18106500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 shutdown_connections 2023-07-13T22:26:25.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:25.932+0000 7fef1c9ff700 1 -- 172.21.15.26:0/3432885750 wait complete. 2023-07-13T22:26:25.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-13T22:26:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/3728644244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 msgr2=0x7f2ff4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/3728644244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff4107f30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b0a0 tx=0x7f2fec00b3b0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/3728644244 shutdown_connections 2023-07-13T22:26:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/3728644244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff4107f30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/3728644244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/3728644244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 0x7f2ff4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/3728644244 >> 172.21.15.26:0/3728644244 conn(0x7f2ff40fb130 msgr2=0x7f2ff40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/3728644244 shutdown_connections 2023-07-13T22:26:26.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.158+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/3728644244 wait complete. 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.159+0000 7f2ffa3ff700 1 Processor -- start 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.159+0000 7f2ffa3ff700 1 -- start start 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.159+0000 7f2ffa3ff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ffa3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 0x7f2ff4195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ffa3ff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ffa3ff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419d510 con 0x7f2ff4101670 2023-07-13T22:26:26.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ffa3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419d650 con 0x7f2ff4100ce0 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ffa3ff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff419d7e0 con 0x7f2ff40ffaf0 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35384/0 (socket says 172.21.15.26:35384) 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff3fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:54914/0 (socket says 172.21.15.26:54914) 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff3fff700 1 -- 172.21.15.26:0/2777627318 learned_addr learned my addr 172.21.15.26:0/2777627318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:26.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff37fe700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 0x7f2ff4195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 -- 172.21.15.26:0/2777627318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 msgr2=0x7f2ff4195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 -- 172.21.15.26:0/2777627318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 msgr2=0x7f2ff4195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 0x7f2ff4195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff899c700 1 -- 172.21.15.26:0/2777627318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec009d00 con 0x7f2ff4101670 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.160+0000 7f2ff3fff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:26.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ff899c700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b070 tx=0x7f2fec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00e040 con 0x7f2ff4101670 2023-07-13T22:26:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fec009480 con 0x7f2ff4101670 2023-07-13T22:26:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff419da60 con 0x7f2ff4101670 2023-07-13T22:26:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0052a0 con 0x7f2ff4101670 2023-07-13T22:26:26.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.161+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff419de80 con 0x7f2ff4101670 2023-07-13T22:26:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.162+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff4106fd0 con 0x7f2ff4101670 2023-07-13T22:26:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.164+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2fec01d480 con 0x7f2ff4101670 2023-07-13T22:26:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.165+0000 7f2ff17fa700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 0x7f2fdc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.165+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2fec0cdb20 con 0x7f2ff4101670 2023-07-13T22:26:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.166+0000 7f2ff3fff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 0x7f2fdc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.166+0000 7f2ff3fff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 0x7f2fdc0a3680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4005950 tx=0x7f2fe4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:26.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.169+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2fec0972b0 con 0x7f2ff4101670 2023-07-13T22:26:26.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:25 smithi117 ceph-mon[129074]: pgmap v691: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:26.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:25 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/258643739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:25 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2865074724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:25 smithi026 ceph-mon[95647]: pgmap v691: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:26.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:25 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/258643739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:25 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2865074724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:25 smithi121 ceph-mon[129187]: pgmap v691: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-07-13T22:26:26.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:25 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/258643739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:25 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2865074724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-13T22:26:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.429+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2ff402cc70 con 0x7f2ff4101670 2023-07-13T22:26:26.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.436+0000 7f2ff17fa700 1 -- 172.21.15.26:0/2777627318 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2fec09a8d0 con 0x7f2ff4101670 2023-07-13T22:26:26.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 msgr2=0x7f2fdc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 0x7f2fdc0a3680 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4005950 tx=0x7f2fe4009260 comp rx=0 tx=0).stop 2023-07-13T22:26:26.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 msgr2=0x7f2ff419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b070 tx=0x7f2fec00bc80 comp rx=0 tx=0).stop 2023-07-13T22:26:26.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 shutdown_connections 2023-07-13T22:26:26.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f2fdc0a11d0 0x7f2fdc0a3680 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4101670 0x7f2ff419cf40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ff40ffaf0 0x7f2ff4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 --2- 172.21.15.26:0/2777627318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ff4100ce0 0x7f2ff4195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 >> 172.21.15.26:0/2777627318 conn(0x7f2ff40fb130 msgr2=0x7f2ff4104110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 shutdown_connections 2023-07-13T22:26:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.440+0000 7f2ffa3ff700 1 -- 172.21.15.26:0/2777627318 wait complete. 2023-07-13T22:26:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-13T22:26:26.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-13T22:26:26.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 -- 172.21.15.26:0/4040317981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc100cd0 msgr2=0x7f33bc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 --2- 172.21.15.26:0/4040317981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc100cd0 0x7f33bc101120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009990 tx=0x7f33b0009ca0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 -- 172.21.15.26:0/4040317981 shutdown_connections 2023-07-13T22:26:26.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 --2- 172.21.15.26:0/4040317981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc101660 0x7f33bc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 --2- 172.21.15.26:0/4040317981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 --2- 172.21.15.26:0/4040317981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc100cd0 0x7f33bc101120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 -- 172.21.15.26:0/4040317981 >> 172.21.15.26:0/4040317981 conn(0x7f33bc0fb0e0 msgr2=0x7f33bc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 -- 172.21.15.26:0/4040317981 shutdown_connections 2023-07-13T22:26:26.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.679+0000 7f33c0dff700 1 -- 172.21.15.26:0/4040317981 wait complete. 2023-07-13T22:26:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.680+0000 7f33c0dff700 1 Processor -- start 2023-07-13T22:26:26.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 -- start start 2023-07-13T22:26:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 0x7f33bc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 0x7f33bc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0735e0 con 0x7f33bc101660 2023-07-13T22:26:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc073750 con 0x7f33bc0ffae0 2023-07-13T22:26:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33c0dff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc19f8e0 con 0x7f33bc100cd0 2023-07-13T22:26:26.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 0x7f33bc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33b9d9b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 0x7f33bc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33bad9d700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 0x7f33bc072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35428/0 (socket says 172.21.15.26:35428) 2023-07-13T22:26:26.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33bad9d700 1 -- 172.21.15.26:0/3825677369 learned_addr learned my addr 172.21.15.26:0/3825677369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.681+0000 7f33ba59c700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 -- 172.21.15.26:0/3825677369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 msgr2=0x7f33bc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 -- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 msgr2=0x7f33bc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 0x7f33bc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 -- 172.21.15.26:0/3825677369 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b00095f0 con 0x7f33bc100cd0 2023-07-13T22:26:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33ba59c700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:26.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33b9d9b700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 0x7f33bc072900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b620 tx=0x7f33b0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b001d070 con 0x7f33bc100cd0 2023-07-13T22:26:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc19fb60 con 0x7f33bc100cd0 2023-07-13T22:26:26.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33b0021410 con 0x7f33bc100cd0 2023-07-13T22:26:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0017650 con 0x7f33bc100cd0 2023-07-13T22:26:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.682+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc19ffe0 con 0x7f33bc100cd0 2023-07-13T22:26:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.683+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc10b300 con 0x7f33bc100cd0 2023-07-13T22:26:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.685+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f33b00217f0 con 0x7f33bc100cd0 2023-07-13T22:26:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.687+0000 7f33ab7fe700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 0x7f33a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.687+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f33b00cdfe0 con 0x7f33bc100cd0 2023-07-13T22:26:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.687+0000 7f33ba59c700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 0x7f33a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.688+0000 7f33ba59c700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 0x7f33a40a3600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f33bc100b30 tx=0x7f33ac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:26.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.688+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f33b00975e0 con 0x7f33bc100cd0 2023-07-13T22:26:26.693 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a[95623]: 2023-07-13T22:26:26.430+0000 7feef351e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:26:26.694 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.694 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.694 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mds[130661]: -- 172.21.15.26:0/786135665 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f18e971e0 con 0x557f18e6d000 2023-07-13T22:26:26.694 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: starting mds.a at 2023-07-13T22:26:26.694 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.695 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.695 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mds[130247]: -- 172.21.15.26:0/1999323279 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564fa480f1e0 con 0x564fa47e4800 2023-07-13T22:26:26.695 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: starting mds.g at 2023-07-13T22:26:26.736 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.736 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.736 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mds[155278]: -- 172.21.15.121:0/3525743309 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b088bc71e0 con 0x55b088b9cc00 2023-07-13T22:26:26.736 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: starting mds.i at 2023-07-13T22:26:26.737 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.737 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.737 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mds[154260]: -- 172.21.15.121:0/3879613807 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a106e511e0 con 0x55a106e26800 2023-07-13T22:26:26.737 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: starting mds.l at 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mds[154937]: -- 172.21.15.121:0/332685199 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565076e191e0 con 0x565076deec00 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: starting mds.c at 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.738 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.739 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mds[154598]: -- 172.21.15.121:0/2769180098 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56150a9f51e0 con 0x56150a9cac00 2023-07-13T22:26:26.739 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: starting mds.f at 2023-07-13T22:26:26.890 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.890 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.890 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mds[153297]: -- 172.21.15.117:0/3020452948 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e3b06f1e0 con 0x556e3b045000 2023-07-13T22:26:26.891 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: starting mds.k at 2023-07-13T22:26:26.891 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.891 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.891 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mds[154319]: -- 172.21.15.117:0/237370962 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bf696431e0 con 0x55bf69619000 2023-07-13T22:26:26.891 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: starting mds.b at 2023-07-13T22:26:26.892 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.892 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.892 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mds[153637]: -- 172.21.15.117:0/1945436876 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f7668b1e0 con 0x559f76660800 2023-07-13T22:26:26.892 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: starting mds.e at 2023-07-13T22:26:26.893 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: ignoring --setuser ceph since I am not root 2023-07-13T22:26:26.893 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: ignoring --setgroup ceph since I am not root 2023-07-13T22:26:26.893 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mds[153978]: -- 172.21.15.117:0/1741128275 <== mon.1 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5575e07031e0 con 0x5575e06d8800 2023-07-13T22:26:26.893 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: starting mds.h at 2023-07-13T22:26:26.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:26.949+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f33bc1069e0 con 0x7f33bc100cd0 2023-07-13T22:26:27.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3432885750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:27.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-13T22:26:27.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-13T22:26:27.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:26:27.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: osdmap e86: 12 total, 12 up, 12 in 2023-07-13T22:26:27.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-13T22:26:27.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:26 smithi117 ceph-mon[129074]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-13T22:26:27.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3432885750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:27.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-13T22:26:27.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-13T22:26:27.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:26:27.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: osdmap e86: 12 total, 12 up, 12 in 2023-07-13T22:26:27.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-13T22:26:27.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:26 smithi026 ceph-mon[95647]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-13T22:26:27.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3432885750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-13T22:26:27.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-13T22:26:27.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-13T22:26:27.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-13T22:26:27.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: osdmap e86: 12 total, 12 up, 12 in 2023-07-13T22:26:27.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/2777627318' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-13T22:26:27.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:26 smithi121 ceph-mon[129187]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-07-13T22:26:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.443+0000 7f33ab7fe700 1 -- 172.21.15.26:0/3825677369 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33b009ac00 con 0x7f33bc100cd0 2023-07-13T22:26:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 msgr2=0x7f33a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 0x7f33a40a3600 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f33bc100b30 tx=0x7f33ac009260 comp rx=0 tx=0).stop 2023-07-13T22:26:27.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 msgr2=0x7f33bc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:27.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 0x7f33bc072900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f33b000b620 tx=0x7f33b0004730 comp rx=0 tx=0).stop 2023-07-13T22:26:27.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 shutdown_connections 2023-07-13T22:26:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f33a40a1150 0x7f33a40a3600 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33bc101660 0x7f33bc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33bc0ffae0 0x7f33bc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 --2- 172.21.15.26:0/3825677369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f33bc100cd0 0x7f33bc072900 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.448+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 >> 172.21.15.26:0/3825677369 conn(0x7f33bc0fb0e0 msgr2=0x7f33bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:27.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.449+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 shutdown_connections 2023-07-13T22:26:27.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.449+0000 7f33c0dff700 1 -- 172.21.15.26:0/3825677369 wait complete. 2023-07-13T22:26:27.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-13T22:26:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 -- 172.21.15.26:0/2846975993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 msgr2=0x7f47800fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 --2- 172.21.15.26:0/2846975993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f47800fff00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4770009990 tx=0x7f4770009ca0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 -- 172.21.15.26:0/2846975993 shutdown_connections 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 --2- 172.21.15.26:0/2846975993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f47800fff00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 --2- 172.21.15.26:0/2846975993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 0x7f4780107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 --2- 172.21.15.26:0/2846975993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 0x7f4780101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 -- 172.21.15.26:0/2846975993 >> 172.21.15.26:0/2846975993 conn(0x7f47800fb130 msgr2=0x7f47800fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 -- 172.21.15.26:0/2846975993 shutdown_connections 2023-07-13T22:26:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.683+0000 7f4785fff700 1 -- 172.21.15.26:0/2846975993 wait complete. 2023-07-13T22:26:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 Processor -- start 2023-07-13T22:26:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 -- start start 2023-07-13T22:26:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 0x7f4780197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 0x7f478019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f5e0 con 0x7f47800ffaf0 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f720 con 0x7f4780101670 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.684+0000 7f4785fff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478019f8b0 con 0x7f4780100ce0 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35446/0 (socket says 172.21.15.26:35446) 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 -- 172.21.15.26:0/393102062 learned_addr learned my addr 172.21.15.26:0/393102062 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:27.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f4777fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 0x7f4780197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477ffff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 0x7f478019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 -- 172.21.15.26:0/393102062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 msgr2=0x7f4780197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 0x7f4780197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 -- 172.21.15.26:0/393102062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 msgr2=0x7f478019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 0x7f478019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 -- 172.21.15.26:0/393102062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700095f0 con 0x7f47800ffaf0 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.685+0000 7f477f7fe700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4770005740 tx=0x7f4770005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.686+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477001d070 con 0x7f47800ffaf0 2023-07-13T22:26:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.686+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4770021410 con 0x7f47800ffaf0 2023-07-13T22:26:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.686+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478019fb30 con 0x7f47800ffaf0 2023-07-13T22:26:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.686+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770017650 con 0x7f47800ffaf0 2023-07-13T22:26:27.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.686+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478019ffb0 con 0x7f47800ffaf0 2023-07-13T22:26:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.687+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47801921d0 con 0x7f47800ffaf0 2023-07-13T22:26:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.689+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4770021580 con 0x7f47800ffaf0 2023-07-13T22:26:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.690+0000 7f477d7fa700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 0x7f476c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.690+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f47700cdbb0 con 0x7f47800ffaf0 2023-07-13T22:26:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.691+0000 7f4777fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 0x7f476c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.691+0000 7f4777fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 0x7f476c0a3680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4768009c60 tx=0x7f4768009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:27.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.694+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4770097190 con 0x7f47800ffaf0 2023-07-13T22:26:27.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:27.908+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f478002c850 con 0x7f47800ffaf0 2023-07-13T22:26:28.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: pgmap v693: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-13T22:26:28.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/3825677369' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: Cluster is now healthy 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6833/3515042893,v1:172.21.15.117:6837/3515042893] up:boot 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6835/3121719560,v1:172.21.15.117:6838/3121719560] up:boot 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6834/1477733024,v1:172.21.15.117:6839/1477733024] up:boot 2023-07-13T22:26:28.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6832/1404782973,v1:172.21.15.121:6833/1404782973] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.117:6832/2648411999,v1:172.21.15.117:6836/2648411999] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6838/2255780632,v1:172.21.15.121:6839/2255780632] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6834/3650572946,v1:172.21.15.121:6835/3650572946] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.121:6836/481698673,v1:172.21.15.121:6837/481698673] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6838/2106971735,v1:172.21.15.26:6840/2106971735] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: mds.? [v2:172.21.15.26:6839/2377805305,v1:172.21.15.26:6841/2377805305] up:boot 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:26:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: fsmap 12 up:standby 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:26:28.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:26:28.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:26:28.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:26:28.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:27 smithi117 ceph-mon[129074]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:26:28.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: pgmap v693: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-13T22:26:28.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/3825677369' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-13T22:26:28.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: Cluster is now healthy 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6833/3515042893,v1:172.21.15.117:6837/3515042893] up:boot 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6835/3121719560,v1:172.21.15.117:6838/3121719560] up:boot 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6834/1477733024,v1:172.21.15.117:6839/1477733024] up:boot 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6832/1404782973,v1:172.21.15.121:6833/1404782973] up:boot 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.117:6832/2648411999,v1:172.21.15.117:6836/2648411999] up:boot 2023-07-13T22:26:28.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6838/2255780632,v1:172.21.15.121:6839/2255780632] up:boot 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6834/3650572946,v1:172.21.15.121:6835/3650572946] up:boot 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.121:6836/481698673,v1:172.21.15.121:6837/481698673] up:boot 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6838/2106971735,v1:172.21.15.26:6840/2106971735] up:boot 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: mds.? [v2:172.21.15.26:6839/2377805305,v1:172.21.15.26:6841/2377805305] up:boot 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: fsmap 12 up:standby 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:26:28.195 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:27 smithi026 ceph-mon[95647]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:26:28.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: pgmap v693: 97 pgs: 97 active+clean; 6.5 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2023-07-13T22:26:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/3825677369' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-13T22:26:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-13T22:26:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-13T22:26:28.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: Cluster is now healthy 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6833/3515042893,v1:172.21.15.117:6837/3515042893] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6835/3121719560,v1:172.21.15.117:6838/3121719560] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6834/1477733024,v1:172.21.15.117:6839/1477733024] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6832/1404782973,v1:172.21.15.121:6833/1404782973] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.117:6832/2648411999,v1:172.21.15.117:6836/2648411999] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6838/2255780632,v1:172.21.15.121:6839/2255780632] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6834/3650572946,v1:172.21.15.121:6835/3650572946] up:boot 2023-07-13T22:26:28.237 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.121:6836/481698673,v1:172.21.15.121:6837/481698673] up:boot 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6838/2106971735,v1:172.21.15.26:6840/2106971735] up:boot 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: mds.? [v2:172.21.15.26:6839/2377805305,v1:172.21.15.26:6841/2377805305] up:boot 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: fsmap 12 up:standby 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-13T22:26:28.238 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-13T22:26:28.239 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:27 smithi121 ceph-mon[129187]: from='mgr.14148 172.21.15.26:0/1616727425' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-13T22:26:28.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.444+0000 7f477d7fa700 1 -- 172.21.15.26:0/393102062 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7f477009a7b0 con 0x7f47800ffaf0 2023-07-13T22:26:28.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.448+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 msgr2=0x7f476c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:28.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.448+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 0x7f476c0a3680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4768009c60 tx=0x7f4768009260 comp rx=0 tx=0).stop 2023-07-13T22:26:28.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.448+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 msgr2=0x7f4780197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:28.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.448+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4770005740 tx=0x7f4770005770 comp rx=0 tx=0).stop 2023-07-13T22:26:28.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 shutdown_connections 2023-07-13T22:26:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f476c0a11d0 0x7f476c0a3680 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4768009c60 tx=0x7f4768009260 comp rx=0 tx=0).stop 2023-07-13T22:26:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47800ffaf0 0x7f4780197a60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4780101670 0x7f478019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 --2- 172.21.15.26:0/393102062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4780100ce0 0x7f4780197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 >> 172.21.15.26:0/393102062 conn(0x7f47800fb130 msgr2=0x7f47801065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 shutdown_connections 2023-07-13T22:26:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.449+0000 7f4785fff700 1 -- 172.21.15.26:0/393102062 wait complete. 2023-07-13T22:26:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-07-13T22:26:28.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-13T22:26:28.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 -- 172.21.15.26:0/4117994133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c101660 msgr2=0x7f843c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:28.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 --2- 172.21.15.26:0/4117994133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c101660 0x7f843c107f20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f843400b0a0 tx=0x7f843400b3b0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 -- 172.21.15.26:0/4117994133 shutdown_connections 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 --2- 172.21.15.26:0/4117994133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c101660 0x7f843c107f20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 --2- 172.21.15.26:0/4117994133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c0ffae0 0x7f843c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 --2- 172.21.15.26:0/4117994133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c100cd0 0x7f843c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.679+0000 7f8442bff700 1 -- 172.21.15.26:0/4117994133 >> 172.21.15.26:0/4117994133 conn(0x7f843c0fb0e0 msgr2=0x7f843c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.680+0000 7f8442bff700 1 -- 172.21.15.26:0/4117994133 shutdown_connections 2023-07-13T22:26:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.680+0000 7f8442bff700 1 -- 172.21.15.26:0/4117994133 wait complete. 2023-07-13T22:26:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 Processor -- start 2023-07-13T22:26:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 -- start start 2023-07-13T22:26:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c101660 0x7f843c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0735e0 con 0x7f843c100cd0 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c073750 con 0x7f843c101660 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f8442bff700 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c19f9c0 con 0x7f843c0ffae0 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f844099b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f843bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f844099b700 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.26:55030/0 (socket says 172.21.15.26:55030) 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f844099b700 1 -- 172.21.15.26:0/1989476943 learned_addr learned my addr 172.21.15.26:0/1989476943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.681+0000 7f843bfff700 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35464/0 (socket says 172.21.15.26:35464) 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844119c700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c101660 0x7f843c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 -- 172.21.15.26:0/1989476943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c101660 msgr2=0x7f843c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c101660 0x7f843c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 -- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 msgr2=0x7f843c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 -- 172.21.15.26:0/1989476943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434009d00 con 0x7f843c0ffae0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f843bfff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f844099b700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f843000e970 tx=0x7f843000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.682+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430009790 con 0x7f843c0ffae0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.683+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c19fca0 con 0x7f843c0ffae0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.683+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843000c7e0 con 0x7f843c0ffae0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.683+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430003b60 con 0x7f843c0ffae0 2023-07-13T22:26:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.683+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c1a01d0 con 0x7f843c0ffae0 2023-07-13T22:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.684+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c10b310 con 0x7f843c0ffae0 2023-07-13T22:26:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.686+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8430003d00 con 0x7f843c0ffae0 2023-07-13T22:26:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.687+0000 7f8439ffb700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 0x7f842c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-13T22:26:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.687+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8116+0+0 (secure 0 0 0) 0x7f8430014070 con 0x7f843c0ffae0 2023-07-13T22:26:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.687+0000 7f843bfff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 0x7f842c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-13T22:26:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.688+0000 7f843bfff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 0x7f842c0a3680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8428005950 tx=0x7f842800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-13T22:26:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.690+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f84300962e0 con 0x7f843c0ffae0 2023-07-13T22:26:28.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:28.899+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f843c106990 con 0x7f843c0ffae0 2023-07-13T22:26:29.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:28 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:28 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:29.185 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:28 smithi117 ceph-mon[129074]: osdmap e87: 12 total, 12 up, 12 in 2023-07-13T22:26:29.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:28 smithi117 ceph-mon[129074]: from='client.? 172.21.15.26:0/1989476943' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.186 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:28 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:29.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:28 smithi026 ceph-mon[95647]: osdmap e87: 12 total, 12 up, 12 in 2023-07-13T22:26:29.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:28 smithi026 ceph-mon[95647]: from='client.? 172.21.15.26:0/1989476943' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.194 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:28 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.235 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:28 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:28 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/393102062' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:29.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:28 smithi121 ceph-mon[129187]: osdmap e87: 12 total, 12 up, 12 in 2023-07-13T22:26:29.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:28 smithi121 ceph-mon[129187]: from='client.? 172.21.15.26:0/1989476943' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.236 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:28 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-13T22:26:29.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.454+0000 7f8439ffb700 1 -- 172.21.15.26:0/1989476943 <== mon.1 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8430099900 con 0x7f843c0ffae0 2023-07-13T22:26:29.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.459+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 msgr2=0x7f842c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.459+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 0x7f842c0a3680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8428005950 tx=0x7f842800b410 comp rx=0 tx=0).stop 2023-07-13T22:26:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.459+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 msgr2=0x7f843c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-13T22:26:29.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.459+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f843000e970 tx=0x7f843000ec80 comp rx=0 tx=0).stop 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 shutdown_connections 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f843c101660 0x7f843c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f843c0ffae0 0x7f843c0742c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:6800/4053809718,v1:172.21.15.26:6801/4053809718] conn(0x7f842c0a11d0 0x7f842c0a3680 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 --2- 172.21.15.26:0/1989476943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843c100cd0 0x7f843c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 >> 172.21.15.26:0/1989476943 conn(0x7f843c0fb0e0 msgr2=0x7f843c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-13T22:26:29.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 shutdown_connections 2023-07-13T22:26:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-07-13T22:26:29.460+0000 7f8442bff700 1 -- 172.21.15.26:0/1989476943 wait complete. 2023-07-13T22:26:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-07-13T22:26:29.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:26:29.553 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:26:29.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:26:29.617 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-13T22:26:29.618 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-13T22:26:29.664 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-13T22:26:29.681 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-13T22:26:29.697 INFO:tasks.cephadm:Stopping all daemons... 2023-07-13T22:26:29.698 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-13T22:26:29.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a 2023-07-13T22:26:29.903 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:29 smithi117 ceph-mon[129074]: pgmap v695: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:26:29.904 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:29 smithi117 ceph-mon[129074]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:29.904 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:29 smithi117 ceph-mon[129074]: osdmap e88: 12 total, 12 up, 12 in 2023-07-13T22:26:29.904 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:29 smithi121 ceph-mon[129187]: pgmap v695: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:26:29.905 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:29 smithi121 ceph-mon[129187]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:29.905 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:29 smithi121 ceph-mon[129187]: osdmap e88: 12 total, 12 up, 12 in 2023-07-13T22:26:30.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:29 smithi026 systemd[1]: Stopping Ceph mon.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:30.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:29 smithi026 ceph-mon[95647]: pgmap v695: 65 pgs: 65 active+clean; 577 KiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-13T22:26:30.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:29 smithi026 ceph-mon[95647]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-13T22:26:30.193 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:29 smithi026 ceph-mon[95647]: osdmap e88: 12 total, 12 up, 12 in 2023-07-13T22:26:30.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a[95623]: 2023-07-13T22:26:30.286+0000 7feef8d29700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:30.567 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a[95623]: 2023-07-13T22:26:30.286+0000 7feef8d29700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-13T22:26:30.837 INFO:journalctl@ceph.mon.a.smithi026.stdout:Jul 13 22:26:30 smithi026 bash[160168]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-a 2023-07-13T22:26:31.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.a.service' 2023-07-13T22:26:31.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:31.243 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-13T22:26:31.244 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-13T22:26:31.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.b 2023-07-13T22:26:31.590 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:31 smithi117 systemd[1]: Stopping Ceph mon.b for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:31.853 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-b[129050]: 2023-07-13T22:26:31.608+0000 7fbfb6000700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:31.853 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-b[129050]: 2023-07-13T22:26:31.608+0000 7fbfb6000700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-13T22:26:31.853 INFO:journalctl@ceph.mon.b.smithi117.stdout:Jul 13 22:26:31 smithi117 bash[159186]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-b 2023-07-13T22:26:32.135 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.b.service' 2023-07-13T22:26:32.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:32.269 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-13T22:26:32.270 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-13T22:26:32.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.c 2023-07-13T22:26:32.627 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:32 smithi121 systemd[1]: Stopping Ceph mon.c for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:32.985 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-c[129161]: 2023-07-13T22:26:32.645+0000 7faf33562700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:32.986 INFO:journalctl@ceph.mon.c.smithi121.stdout:Jul 13 22:26:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mon-c[129161]: 2023-07-13T22:26:32.645+0000 7faf33562700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-13T22:26:33.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mon.c.service' 2023-07-13T22:26:33.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:33.246 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-13T22:26:33.247 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-13T22:26:33.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x 2023-07-13T22:26:33.693 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:26:33 smithi026 systemd[1]: Stopping Ceph mgr.x for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:34.013 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:26:33 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x[95897]: 2023-07-13T22:26:33.778+0000 7fb1fed74700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:34.443 INFO:journalctl@ceph.mgr.x.smithi026.stdout:Jul 13 22:26:34 smithi026 bash[160307]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-x 2023-07-13T22:26:34.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.x.service' 2023-07-13T22:26:34.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:34.573 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-13T22:26:34.573 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-13T22:26:34.573 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.y 2023-07-13T22:26:34.934 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:26:34 smithi117 systemd[1]: Stopping Ceph mgr.y for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:35.356 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:26:35 smithi117 bash[159340]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-y 2023-07-13T22:26:35.356 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Jul 13 22:26:35 smithi117 systemd[1]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-13T22:26:35.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.y.service' 2023-07-13T22:26:35.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:35.400 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-13T22:26:35.400 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-13T22:26:35.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.z 2023-07-13T22:26:35.736 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:26:35 smithi121 systemd[1]: Stopping Ceph mgr.z for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:36.107 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:26:36 smithi121 bash[160463]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mgr-z 2023-07-13T22:26:36.108 INFO:journalctl@ceph.mgr.z.smithi121.stdout:Jul 13 22:26:36 smithi121 systemd[1]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-13T22:26:36.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mgr.z.service' 2023-07-13T22:26:36.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:36.326 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-13T22:26:36.327 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-13T22:26:36.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.0 2023-07-13T22:26:36.693 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:26:36 smithi026 systemd[1]: Stopping Ceph osd.0 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:37.193 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:26:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:26:36.868+0000 7f139d933700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:37.193 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:26:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:26:36.868+0000 7f139d933700 -1 osd.0 88 *** Got signal Terminated *** 2023-07-13T22:26:37.193 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:26:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0[108574]: 2023-07-13T22:26:36.868+0000 7f139d933700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:26:42.943 INFO:journalctl@ceph.osd.0.smithi026.stdout:Jul 13 22:26:42 smithi026 bash[160453]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-0 2023-07-13T22:26:44.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.0.service' 2023-07-13T22:26:44.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:44.714 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-13T22:26:44.714 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-13T22:26:44.715 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.1 2023-07-13T22:26:45.087 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:26:44 smithi117 systemd[1]: Stopping Ceph osd.1 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:45.435 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:26:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:26:45.105+0000 7f7c714cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:45.435 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:26:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:26:45.105+0000 7f7c714cb700 -1 osd.1 88 *** Got signal Terminated *** 2023-07-13T22:26:45.435 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:26:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1[134136]: 2023-07-13T22:26:45.105+0000 7f7c714cb700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:26:50.935 INFO:journalctl@ceph.osd.1.smithi117.stdout:Jul 13 22:26:50 smithi117 bash[159484]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-1 2023-07-13T22:26:52.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.1.service' 2023-07-13T22:26:52.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:52.152 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-13T22:26:52.153 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-13T22:26:52.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.2 2023-07-13T22:26:52.485 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:26:52 smithi121 systemd[1]: Stopping Ceph osd.2 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:26:52.985 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:26:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:26:52.554+0000 7f530beba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:26:52.985 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:26:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:26:52.554+0000 7f530beba700 -1 osd.2 88 *** Got signal Terminated *** 2023-07-13T22:26:52.986 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:26:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2[134973]: 2023-07-13T22:26:52.554+0000 7f530beba700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:26:58.485 INFO:journalctl@ceph.osd.2.smithi121.stdout:Jul 13 22:26:58 smithi121 bash[160611]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-2 2023-07-13T22:26:59.552 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.2.service' 2023-07-13T22:26:59.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:26:59.594 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-13T22:26:59.595 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-13T22:26:59.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.3 2023-07-13T22:26:59.943 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:26:59 smithi026 systemd[1]: Stopping Ceph osd.3 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:00.443 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:27:00 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:27:00.139+0000 7fa0d6646700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:00.443 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:27:00 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:27:00.139+0000 7fa0d6646700 -1 osd.3 88 *** Got signal Terminated *** 2023-07-13T22:27:00.443 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:27:00 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:27:00.139+0000 7fa0d6646700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:03.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:02 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:02.888+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:04.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:03 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:03.924+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:04.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:04.172+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:04.692 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:27:04 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3[114183]: 2023-07-13T22:27:04.247+0000 7fa0d1c3a700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:41.468496+0000 front 2023-07-13T22:26:41.468560+0000 (oldest deadline 2023-07-13T22:27:03.768642+0000) 2023-07-13T22:27:04.933 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:04 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:04.597+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:05.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:04 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:04.953+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:05.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:05.239+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:05.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:05.176+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:05.901 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:05 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:05.581+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:06.007 INFO:journalctl@ceph.osd.3.smithi026.stdout:Jul 13 22:27:05 smithi026 bash[160732]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-3 2023-07-13T22:27:06.007 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:05 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:05.810+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:06.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:05 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:05.920+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:06.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:06.227+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:06.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:06.155+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:06.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:06.438+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:06.913 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:06 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:06.548+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:07.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:06 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:06.933+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:07.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:06 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:06.766+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:07.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:07 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:07.276+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:07.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:07 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:07.191+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:07.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:07 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:07.402+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:07.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.3.service' 2023-07-13T22:27:07.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:07.856 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-13T22:27:07.857 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-13T22:27:07.859 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.4 2023-07-13T22:27:07.892 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:07 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:07.562+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:08.184 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:07 smithi117 systemd[1]: Stopping Ceph osd.4 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:08.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:07 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:07.972+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:08.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:07 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:07.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:08.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:08 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:08.252+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:08.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:08 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:08.174+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:08.547 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:08 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:08.224+0000 7fefc10e3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:08.547 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:08 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:08.224+0000 7fefc10e3700 -1 osd.4 88 *** Got signal Terminated *** 2023-07-13T22:27:08.547 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:08 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:08.224+0000 7fefc10e3700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:08.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:08.382+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:08.934 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:08 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:08.567+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:09.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:08 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:08.801+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:09.434 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:09 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:09.019+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:09.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:09 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:09.207+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:09.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:09 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:09.169+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:09.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:09.395+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:09.934 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:09 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:09.568+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:10.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:09.766+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:10.434 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:09 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:09.993+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:10.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:10 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:10.164+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:10.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:10 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:10.172+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:10.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:10 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:10.442+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:10.934 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:10 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:10.531+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:11.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:10 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:10.719+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:11.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:11 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:11.146+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:11.435 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:10 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:10.965+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:11.435 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:10 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:10.965+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:11.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:11 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:11.169+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:11.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:11 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:11.204+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:11.739 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:11 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:11.487+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:11.934 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:11 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:11.486+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:12.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:11 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:11.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:12.421 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:12 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:12.104+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:12.422 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:11 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:11.978+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:12.422 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:11 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:11.978+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:12.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:12 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:12.123+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:12.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:12 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:12.149+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:12.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:12 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:12.242+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:12.684 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:12 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4[139612]: 2023-07-13T22:27:12.441+0000 7fefbdeda700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.932353+0000 front 2023-07-13T22:26:40.932398+0000 (oldest deadline 2023-07-13T22:27:04.432510+0000) 2023-07-13T22:27:12.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:12 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:12.492+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:12.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:12 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:12.737+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:13.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:13 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:13.154+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:13.435 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:12 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:12.986+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:13.435 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:12 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:12.986+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:13.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:13 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:13.167+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:13.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:13 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:13.131+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:13.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:13 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:13.233+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:13.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:13 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:13.233+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:13.675 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:13 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:13.443+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:13.939 INFO:journalctl@ceph.osd.4.smithi117.stdout:Jul 13 22:27:13 smithi117 bash[159749]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-4 2023-07-13T22:27:13.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:13 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:13.695+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:14.433 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:14 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:14.117+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:14.434 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:13 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:13.957+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:14.434 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:13 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:13.957+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:14.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:14.163+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:14.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:14.185+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:14.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:14.185+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:14.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:14.191+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:14.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:14 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:14.192+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:14.682 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:14 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:14.402+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:14.683 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:14 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:14.402+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:14.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:14 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:14.702+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:15.147 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:14 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:14.940+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:15.148 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:14 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:14.940+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:15.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.4.service' 2023-07-13T22:27:15.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:15.240 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-13T22:27:15.241 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-13T22:27:15.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.5 2023-07-13T22:27:15.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:15 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:15.167+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:15.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:15.181+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:15.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:15.177+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:15.486 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:15.177+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:15.486 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 systemd[1]: Stopping Ceph osd.5 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:15.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:15.223+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:15.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:15.223+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:15.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:15 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:15.435+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:15.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:15 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:15.435+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:15.985 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:15.622+0000 7f2920dfc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:15.985 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:15.622+0000 7f2920dfc700 -1 osd.5 88 *** Got signal Terminated *** 2023-07-13T22:27:15.985 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:15 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:15.622+0000 7f2920dfc700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:16.184 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:16 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:16.149+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:16.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:15 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:15.930+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:16.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:15 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:15.930+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:16.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:15 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:15.715+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:16.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:16 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:16.188+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:16.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:16 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:16.214+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:16.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:16 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:16.214+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:16.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:16 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:16.245+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:16.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:16 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:16.245+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:16.702 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:16.463+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:16.702 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:16.463+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:17.184 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:17 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:17.111+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:17.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:16 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:16.935+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:17.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:16 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:16.935+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:17.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:16.722+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:17.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:16 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:16.722+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:17.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:17 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:17.227+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:17.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:17 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:17.214+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:17.486 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:17 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:17.214+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:17.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:17 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:17.254+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:17.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:17 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:17.254+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:17.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:17.443+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:17.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:17.443+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:18.184 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:18 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:18.148+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:18.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:17 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:17.929+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:18.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:17 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:17.929+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:18.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:17.727+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:18.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:17 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:17.727+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:18.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:18 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:18.198+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:18.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:18 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:18.254+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:18.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:18 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:18.254+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:18.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:18 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:18.286+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:18.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:18 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:18.286+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:18.662 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:18 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:18.409+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:18.663 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:18 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:18.409+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:18.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:18 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:18.682+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:18.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:18 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:18.682+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:19.184 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:19 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:19.104+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:19.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:18 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:18.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:19.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:18 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:18.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:19.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:18 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:18.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:19.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:19.160+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:19.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:19.160+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:19.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:19.218+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:19.486 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:19.218+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:19.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:19.259+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:19.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:19.259+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:19.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:19 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:19.259+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:19.657 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:19.412+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:19.657 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:19.412+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:19.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:19.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:19.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:19.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:19.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:19.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:20.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:19 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:19.855+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:20.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:19 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:19.855+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:20.101 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:19 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:19.855+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:20.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:20 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:20.120+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:20.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:20.227+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:37.979829+0000 front 2023-07-13T22:26:37.979766+0000 (oldest deadline 2023-07-13T22:27:04.380594+0000) 2023-07-13T22:27:20.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:20.227+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.382147+0000 front 2023-07-13T22:26:48.382261+0000 (oldest deadline 2023-07-13T22:27:13.682133+0000) 2023-07-13T22:27:20.485 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5[140428]: 2023-07-13T22:27:20.227+0000 7f291c3f0700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.682612+0000 front 2023-07-13T22:26:53.682589+0000 (oldest deadline 2023-07-13T22:27:19.582716+0000) 2023-07-13T22:27:20.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:20.153+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:20.486 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:20.153+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:20.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:20.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:20.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:20.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:20.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:20 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:20.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:20.663 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:20.417+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:20.663 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:20.417+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:20.663 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:20.417+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:20.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:20.682+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:20.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:20.682+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:20.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:20 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:20.682+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:21.136 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:20 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:20.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:21.136 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:20 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:20.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:21.137 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:20 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:20.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:21.256 INFO:journalctl@ceph.osd.5.smithi121.stdout:Jul 13 22:27:21 smithi121 bash[160885]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-5 2023-07-13T22:27:21.257 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:21.155+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:21.257 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:21.155+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:21.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:21 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:21.156+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:21.636 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:21.379+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:21.636 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:21.379+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:21.636 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:21.379+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:21.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:21.275+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:21.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:21.275+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:21.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:21 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:21.275+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:21.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:21.656+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:21.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:21.656+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:21.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:21 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:21.656+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:22.160 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:21 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:21.851+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:22.160 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:21 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:21.851+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:22.160 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:21 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:21.851+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:22.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:22.179+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:22.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:22.179+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:22.436 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:22 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:22.116+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:22.436 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:22 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:22.116+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:22.436 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:22 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:22.276+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:22.436 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:22 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:22.276+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:22.436 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:22 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:22.276+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:22.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.5.service' 2023-07-13T22:27:22.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:22.483 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-13T22:27:22.483 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-13T22:27:22.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.6 2023-07-13T22:27:22.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:22.408+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:22.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:22.408+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:22.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:22.408+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:22.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:22 smithi026 systemd[1]: Stopping Ceph osd.6 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:22.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:22.644+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:22.694 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:22.644+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:22.694 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:22 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:22.644+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:23.174 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:22.836+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:23.174 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:22.836+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:23.174 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:22.836+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:23.433 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.044+0000 7fc15dbfb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:23.433 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.044+0000 7fc15dbfb700 -1 osd.6 88 *** Got signal Terminated *** 2023-07-13T22:27:23.433 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.044+0000 7fc15dbfb700 -1 osd.6 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:23.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:23 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:23.193+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:23.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:23 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:23.193+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:23.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:23.140+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:23.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:23.140+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:23.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:23.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:23.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:23.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:23.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:23 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:23.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:23.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.452+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:23.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.452+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:23.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:23.452+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:23.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:23.625+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:23.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:23.625+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:23.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:23 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:23.625+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:24.180 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:23 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:23.867+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:24.180 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:23 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:23.867+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:24.181 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:23 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:23.867+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:24.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:24 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:24.199+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:24.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:24 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:24.199+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:24.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:24 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:24.167+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:24.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:24 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:24.167+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:24.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:24 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:24.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:24.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:24 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:24.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:24.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:24 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:24.314+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:24.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:24.492+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:24.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:24.492+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:24.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:24.492+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:24.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:24.607+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:24.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:24.607+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:24.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:24 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:24.607+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:25.180 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:24 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:24.887+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:25.180 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:24 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:24.887+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:25.180 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:24 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:24.887+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:25.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:25 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:25.199+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:25.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:25 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:25.199+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:25.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:25 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:25.168+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:25.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:25 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:25.168+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:25.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:25 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:25.303+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:25.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:25 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:25.303+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:25.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:25 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:25.303+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:25.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:25.499+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:25.943 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:25.499+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:25.943 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:25.499+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:25.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:25.595+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:25.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:25.595+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:25.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:25 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:25.595+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:26.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:25 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:25.874+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:26.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:25 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:25.874+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:26.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:25 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:25.874+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:26.484 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:26.147+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:26.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:26.147+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:26.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:26.351+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:26.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:26.351+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:26.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:26 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:26.351+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:26.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:26.229+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:26.685 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:26.229+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:26.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:26.465+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:26.942 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:26.465+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:26.943 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:26.465+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:26.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:26.606+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:26.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:26.606+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:26.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:26 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:26.606+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:27.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:26.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:27.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:26.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:27.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:26.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:27.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:26 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:26.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:27.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:27 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:27.128+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:27.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:27 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:27.128+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:27.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:27 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:27.321+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:27.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:27 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:27.321+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:27.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:27 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:27.321+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:27.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:27.227+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:27.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:27.227+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:27.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:27.437+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.133744+0000 front 2023-07-13T22:26:40.133709+0000 (oldest deadline 2023-07-13T22:27:05.433894+0000) 2023-07-13T22:27:27.692 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:27.437+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.534994+0000 front 2023-07-13T22:26:49.535169+0000 (oldest deadline 2023-07-13T22:27:14.234924+0000) 2023-07-13T22:27:27.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:27.437+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.235515+0000 front 2023-07-13T22:26:54.235488+0000 (oldest deadline 2023-07-13T22:27:20.135442+0000) 2023-07-13T22:27:27.693 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6[119677]: 2023-07-13T22:27:27.437+0000 7fc1569ea700 -1 osd.6 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.836625+0000 front 2023-07-13T22:27:04.836654+0000 (oldest deadline 2023-07-13T22:27:27.136773+0000) 2023-07-13T22:27:27.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:27.565+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:27.693 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:27.565+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:27.694 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:27.565+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:27.694 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:27 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:27.565+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:28.183 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:27.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:28.183 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:27.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:28.183 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:27.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:28.183 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:27.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:28.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:28.202+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:28.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:28.202+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:28.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:28.135+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:28.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:28.135+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:28.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:28.135+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:28.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:28.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:28.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:28.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:28.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:28 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:28.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:28.824 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:28 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:28.589+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:28.824 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:28 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:28.589+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:28.824 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:28 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:28.589+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:28.824 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:28 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:28.589+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:29.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:28.783+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:29.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:28.783+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:29.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:28.783+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:29.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:28 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:28.783+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:29.192 INFO:journalctl@ceph.osd.6.smithi026.stdout:Jul 13 22:27:28 smithi026 bash[160997]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-6 2023-07-13T22:27:29.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:29.184+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:29.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:29.184+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:29.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:29.184+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:29.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:29.316+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:29.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:29.316+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:29.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:29 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:29.316+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:29.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:29.247+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:29.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:29.247+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:29.823 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:29.590+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:29.823 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:29.590+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:29.823 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:29.590+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:29.823 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:29 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:29.590+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:30.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:29.754+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:30.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:29.754+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:30.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:29.754+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:30.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:29 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:29.754+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:30.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:30.221+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:30.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:30.221+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:30.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:30.221+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:30.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:30.339+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:30.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:30.339+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:30.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:30 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:30.339+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:30.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:30.215+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:30.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:30.215+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:30.685 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:30.215+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:30.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.6.service' 2023-07-13T22:27:30.793 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:30.793 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-13T22:27:30.794 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-13T22:27:30.794 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.7 2023-07-13T22:27:30.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:30.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:30.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:30.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:30.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:30.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:30.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:30 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:30.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:31.175 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:30 smithi117 systemd[1]: Stopping Ceph osd.7 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:31.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:30.794+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:31.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:30.794+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:31.175 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:30.794+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:31.176 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:30 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:30.794+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:31.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.194+0000 7f1e32edb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:31.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.194+0000 7f1e32edb700 -1 osd.7 88 *** Got signal Terminated *** 2023-07-13T22:27:31.435 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.194+0000 7f1e32edb700 -1 osd.7 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:31.435 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.205+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:31.435 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.205+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:31.435 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:31.205+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:31.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:31.196+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:31.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:31.196+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:31.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:31.196+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:31.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:31.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:31.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:31.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:31.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:31 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:31.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:31.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:31 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:31.647+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:31.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:31 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:31.647+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:31.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:31 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:31.647+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:31.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:31 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:31.647+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:32.158 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:31.812+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:32.159 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:31.812+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:32.159 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:31.812+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:32.159 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:31.812+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:32.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:32.178+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:32.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:32.178+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:32.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:32.178+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:32.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:32.227+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:32.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:32.227+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:32.485 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:32.227+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:32.485 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:32.340+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:32.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:32.340+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:32.486 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:32 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:32.340+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:32.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:32 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:32.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:32.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:32 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:32.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:32.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:32 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:32.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:32.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:32 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:32.609+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:33.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:32.804+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:33.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:32.804+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:33.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:32.804+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:33.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:32 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:32.804+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:33.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:33.223+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:33.684 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:33.223+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:33.685 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:33.223+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:33.734 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:33.262+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:33.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:33.262+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:33.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:33.262+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:33.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:33.305+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:33.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:33.305+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:33.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:33 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:33.305+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:33.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:33 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:33.627+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:33.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:33 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:33.627+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:33.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:33 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:33.627+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:33.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:33 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:33.627+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:34.167 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:33.853+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:34.168 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:33.853+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:34.168 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:33.853+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:34.168 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:33 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:33.853+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:34.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:34.187+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:34.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:34.187+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:34.435 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:34.187+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:34.734 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:34.259+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:34.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:34.259+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:34.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:34.259+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:34.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:34.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:34.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:34.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:34.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:34 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:34.327+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:34.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:34.651+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:34.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:34.651+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:34.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:34.651+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:34.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:34 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:34.651+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:35.152 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:34.872+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:35.152 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:34.872+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:35.152 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:34.872+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:35.153 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:34 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:34.872+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:35.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:35.171+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:35.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:35.171+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:35.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:35.171+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:35.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:35.299+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:35.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:35.299+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:35.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:35.299+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:35.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:35.304+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:35.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:35.304+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:35.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:35 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:35.304+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:35.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:35 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:35.619+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:35.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:35 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:35.619+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:35.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:35 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:35.619+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:35.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:35 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:35.619+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:36.154 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:35.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:36.154 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:35.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:36.154 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:35.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:36.154 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:35.830+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:36.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:36.173+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:49.022481+0000 front 2023-07-13T22:26:49.022609+0000 (oldest deadline 2023-07-13T22:27:10.623371+0000) 2023-07-13T22:27:36.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:36.173+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:55.324731+0000 front 2023-07-13T22:26:55.324623+0000 (oldest deadline 2023-07-13T22:27:21.224570+0000) 2023-07-13T22:27:36.434 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7[145155]: 2023-07-13T22:27:36.173+0000 7f1e2d4cd700 -1 osd.7 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:03.525947+0000 front 2023-07-13T22:27:03.525869+0000 (oldest deadline 2023-07-13T22:27:29.326401+0000) 2023-07-13T22:27:36.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:36.331+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:36.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:36.331+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:36.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:36.331+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:36.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:36.331+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:36.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:36.323+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:36.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:36.323+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:36.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:36 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:36.323+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:36.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:36.637+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:36.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:36.637+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:36.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:36.637+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:36.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:36 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:36.637+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:37.035 INFO:journalctl@ceph.osd.7.smithi117.stdout:Jul 13 22:27:36 smithi117 bash[160018]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-7 2023-07-13T22:27:37.035 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:36.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:37.036 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:36.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:37.036 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:36.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:37.036 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:36.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:37.036 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:36 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:36.831+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:37.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:37.315+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:37.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:37.315+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:37.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:37.315+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:37.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:37.315+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:37.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:37.338+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:37.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:37.338+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:37.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:37 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:37.338+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:37.929 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:37 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:37.837+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:37.929 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:37 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:37.837+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:37.929 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:37 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:37.837+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:37.929 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:37 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:37.837+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:37.929 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:37 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:37.837+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:37.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:37 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:37.616+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:37.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:37 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:37.616+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:37.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:37 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:37.616+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:37.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:37 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:37.616+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:38.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.7.service' 2023-07-13T22:27:38.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:38.240 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-13T22:27:38.240 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-13T22:27:38.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.8 2023-07-13T22:27:38.614 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:38.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:38.615 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:38.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:38.615 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:38.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:38.615 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:38.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:38.615 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 systemd[1]: Stopping Ceph osd.8 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:38.615 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.360+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:38.616 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.360+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:38.616 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.360+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:38.616 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.360+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:38.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:38 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:38.608+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:38.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:38 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:38.608+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:38.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:38 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:38.608+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:38.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:38 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:38.608+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:38.984 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.633+0000 7f2a5dd86700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:38.985 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.633+0000 7f2a5dd86700 -1 osd.8 88 *** Got signal Terminated *** 2023-07-13T22:27:38.985 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:38 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:38.633+0000 7f2a5dd86700 -1 osd.8 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:39.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:38 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:38.861+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:39.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:38 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:38.861+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:39.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:38 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:38.861+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:39.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:38 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:38.861+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:39.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:38 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:38.861+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:39.734 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:39.324+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:39.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:39.324+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:39.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:39.324+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:39.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:39.324+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:39.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:39.306+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:39.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:39.306+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:39.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:39.306+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:39.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:39.306+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:39.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:39 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:39.641+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:39.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:39 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:39.641+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:39.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:39 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:39.641+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:39.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:39 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:39.641+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:40.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:39 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:39.870+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:40.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:39 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:39.870+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:40.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:39 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:39.870+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:40.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:39 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:39.870+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:40.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:39 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:39.870+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:40.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:40.281+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:40.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:40.281+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:40.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:40.281+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:40.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:40.281+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:40.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:40.345+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:40.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:40.345+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:40.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:40.345+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:40.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:40 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:40.345+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:40.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:40 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:40.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:40.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:40 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:40.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:40.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:40 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:40.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:40.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:40 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:40.676+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:41.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:40 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:40.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:41.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:40 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:40.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:41.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:40 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:40.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:41.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:40 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:40.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:41.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:40 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:40.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:41.734 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:41.287+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:41.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:41.287+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:41.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:41.287+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:41.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:41.287+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:41.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:41.312+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:41.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:41.312+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:41.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:41.312+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:41.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:41 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:41.312+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:41.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:41.708+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:41.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:41.708+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:41.942 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:41.708+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:41.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:41.708+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:41.943 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:41 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:41.708+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:42.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:41 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:41.864+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:42.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:41 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:41.864+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:42.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:41 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:41.864+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:42.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:41 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:41.864+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:42.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:41 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:41.864+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:42.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:42.301+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:42.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:42.301+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:42.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:42.302+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:42.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:42.302+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:42.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:42.302+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.950902+0000 front 2023-07-13T22:27:16.950941+0000 (oldest deadline 2023-07-13T22:27:42.251046+0000) 2023-07-13T22:27:42.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:42.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:42.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:42.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:42.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:42.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:42.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:42 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:42.263+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:43.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:43.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:43.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:43.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:43.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:43.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:42 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:42.907+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:43.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:42.751+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:43.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:42.751+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:43.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:42.751+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:43.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:42.751+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:43.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:42 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:42.751+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:43.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:43.309+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:43.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:43.309+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:43.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:43.309+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:43.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:43.309+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:43.735 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:43.288+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:46.244509+0000 front 2023-07-13T22:26:46.244329+0000 (oldest deadline 2023-07-13T22:27:12.144365+0000) 2023-07-13T22:27:43.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:43.288+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:57.246672+0000 front 2023-07-13T22:26:57.246775+0000 (oldest deadline 2023-07-13T22:27:18.946837+0000) 2023-07-13T22:27:43.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:43.288+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:04.248004+0000 front 2023-07-13T22:27:04.248116+0000 (oldest deadline 2023-07-13T22:27:27.747857+0000) 2023-07-13T22:27:43.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:43.288+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:09.448801+0000 front 2023-07-13T22:27:09.448870+0000 (oldest deadline 2023-07-13T22:27:35.349518+0000) 2023-07-13T22:27:43.736 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8[145867]: 2023-07-13T22:27:43.288+0000 7f2a56b75700 -1 osd.8 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.950902+0000 front 2023-07-13T22:27:16.950941+0000 (oldest deadline 2023-07-13T22:27:42.251046+0000) 2023-07-13T22:27:44.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:44.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:44.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:44.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:44.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:44.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:43 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:43.896+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:44.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:43.750+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:44.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:43.750+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:44.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:43.750+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:44.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:43.750+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:44.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:43 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:43.750+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:44.479 INFO:journalctl@ceph.osd.8.smithi121.stdout:Jul 13 22:27:44 smithi121 bash[161145]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-8 2023-07-13T22:27:44.480 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:44 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:44.308+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:44.480 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:44 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:44.308+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:44.480 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:44 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:44.308+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:44.480 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:44 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:44.308+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:45.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:45.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:45.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:45.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:45.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:45.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:44 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:44.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:45.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:44.799+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:45.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:44.799+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:45.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:44.799+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:45.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:44.799+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:45.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:44 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:44.799+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:45.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:45 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:45.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:45.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:45 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:45.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:45.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:45 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:45.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:45.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:45 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:45.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:45.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:45 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:45.335+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:45.740 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.8.service' 2023-07-13T22:27:45.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:45.780 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-13T22:27:45.780 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-13T22:27:45.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.9 2023-07-13T22:27:46.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:46.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:46.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:46.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:46.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:46.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:45 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:45.903+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:46.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:45.760+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:46.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:45.760+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:46.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:45.760+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:46.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:45.760+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:46.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:45.760+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:46.194 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:45 smithi026 systemd[1]: Stopping Ceph osd.9 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:46.692 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.323+0000 7feae9b58700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:46.692 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.323+0000 7feae9b58700 -1 osd.9 88 *** Got signal Terminated *** 2023-07-13T22:27:46.692 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.323+0000 7feae9b58700 -1 osd.9 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:46.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:46 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:46.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:46.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:46 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:46.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:46.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:46 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:46.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:46.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:46 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:46.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:46.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:46 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:46.358+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:47.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:47.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:47.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:47.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:47.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:47.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:46 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:46.952+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:47.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.734+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:47.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.734+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:47.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.734+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:47.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.734+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:47.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:46 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:46.734+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:47.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:47.386+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:47.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:47.386+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:47.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:47.386+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:47.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:47.386+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:47.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:47.386+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:48.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:48.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:48.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:48.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:48.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:48.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:47 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:47.921+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:48.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:47.770+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:48.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:47.770+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:48.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:47.770+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:48.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:47.770+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:48.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:47 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:47.770+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:48.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:48 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:48.411+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:48.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:48 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:48.411+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:48.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:48 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:48.411+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:48.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:48 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:48.411+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:48.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:48 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:48.411+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:49.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:49.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:49.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:49.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:49.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:49.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:48 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:48.877+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:49.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:48.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:49.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:48.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:49.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:48.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:49.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:48.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:49.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:48 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:48.782+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:49.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:49 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:49.408+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:49.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:49 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:49.408+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:49.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:49 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:49.408+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:49.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:49 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:49.408+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:49.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:49 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:49.408+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:50.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:50.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:50.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:50.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:50.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:50.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:50.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:49 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:49.834+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:50.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:50.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:50.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:50.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:50.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:50.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:49 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:49.759+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.531838+0000 front 2023-07-13T22:27:26.531914+0000 (oldest deadline 2023-07-13T22:27:48.831777+0000) 2023-07-13T22:27:50.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:50 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:50.368+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:50.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:50 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:50.368+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:50.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:50 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:50.368+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:50.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:50 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:50.368+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:50.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:50 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:50.368+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:51.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:51.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:51.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:51.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:51.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:51.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:51.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:50 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:50.852+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:51.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:40.423922+0000 front 2023-07-13T22:26:40.423678+0000 (oldest deadline 2023-07-13T22:27:05.123521+0000) 2023-07-13T22:27:51.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:45.124459+0000 front 2023-07-13T22:26:45.124417+0000 (oldest deadline 2023-07-13T22:27:16.325109+0000) 2023-07-13T22:27:51.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:56.325843+0000 front 2023-07-13T22:26:56.325865+0000 (oldest deadline 2023-07-13T22:27:19.225869+0000) 2023-07-13T22:27:51.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:05.027690+0000 front 2023-07-13T22:27:05.027724+0000 (oldest deadline 2023-07-13T22:27:27.327872+0000) 2023-07-13T22:27:51.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.030688+0000 front 2023-07-13T22:27:16.030639+0000 (oldest deadline 2023-07-13T22:27:41.330448+0000) 2023-07-13T22:27:51.193 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:50 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9[125215]: 2023-07-13T22:27:50.733+0000 7feae3949700 -1 osd.9 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.531838+0000 front 2023-07-13T22:27:26.531914+0000 (oldest deadline 2023-07-13T22:27:48.831777+0000) 2023-07-13T22:27:51.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:51.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:51.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:51.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:51.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:51.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:51.363+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:52.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:52.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:52.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:52.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:52.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:52.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:52.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:51 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:51.884+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:52.192 INFO:journalctl@ceph.osd.9.smithi026.stdout:Jul 13 22:27:51 smithi026 bash[161279]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-9 2023-07-13T22:27:52.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:52.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:52.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:52.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:52.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:52.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:52 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:52.395+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:53.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:53.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:53.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:53.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:53.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:53.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:53.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:52 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:52.906+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:53.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:53.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:53.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:53.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:53.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:53.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:53 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:53.410+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:53.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.9.service' 2023-07-13T22:27:54.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:27:54.006 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-13T22:27:54.006 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-13T22:27:54.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.10 2023-07-13T22:27:54.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:54.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:54.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:54.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:54.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:54.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:54.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:53 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:53.862+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:54.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 systemd[1]: Stopping Ceph osd.10 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:27:54.684 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.381+0000 7f6bfb125700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:27:54.684 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.381+0000 7f6bfb125700 -1 osd.10 88 *** Got signal Terminated *** 2023-07-13T22:27:54.684 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.381+0000 7f6bfb125700 -1 osd.10 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:27:54.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:54.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:54.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:54.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:54.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:54.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:54 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:54.397+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:55.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:55.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:55.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:55.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:55.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:55.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:55.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:54 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:54.905+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:55.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:55.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:55.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:55.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:55.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:55.736 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:55 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:55.441+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:56.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:56.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:56.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:56.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:56.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:56.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:56.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:55 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:55.882+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:56.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:56.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:56.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:56.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:56.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:56.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:56 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:56.475+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:57.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:57.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:57.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:57.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:57.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:57.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:57.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:56 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:56.891+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:57.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:57.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:57.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:57.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:57.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:57.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:57 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:57.498+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:58.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:58.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:57 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:57.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:58.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:58.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:58.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:58.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:58.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:58.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:58 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:58.512+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:59.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.758127+0000 front 2023-07-13T22:26:39.757954+0000 (oldest deadline 2023-07-13T22:27:02.658164+0000) 2023-07-13T22:27:59.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.360548+0000 front 2023-07-13T22:26:48.360831+0000 (oldest deadline 2023-07-13T22:27:10.660703+0000) 2023-07-13T22:27:59.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:54.062530+0000 front 2023-07-13T22:26:54.062570+0000 (oldest deadline 2023-07-13T22:27:18.162608+0000) 2023-07-13T22:27:59.184 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6814 osd.3 since back 2023-07-13T22:27:01.063887+0000 front 2023-07-13T22:27:01.063873+0000 (oldest deadline 2023-07-13T22:27:26.363843+0000) 2023-07-13T22:27:59.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:11.664711+0000 front 2023-07-13T22:27:11.664663+0000 (oldest deadline 2023-07-13T22:27:36.364849+0000) 2023-07-13T22:27:59.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:16.365333+0000 front 2023-07-13T22:27:16.365390+0000 (oldest deadline 2023-07-13T22:27:42.265366+0000) 2023-07-13T22:27:59.185 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:27:58 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10[150613]: 2023-07-13T22:27:58.841+0000 7f6bf5f18700 -1 osd.10 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:26.167621+0000 front 2023-07-13T22:27:26.167737+0000 (oldest deadline 2023-07-13T22:27:49.667570+0000) 2023-07-13T22:27:59.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:27:59.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:27:59 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:27:59.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:00.143 INFO:journalctl@ceph.osd.10.smithi117.stdout:Jul 13 22:28:00 smithi117 bash[160280]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-10 2023-07-13T22:28:00.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:00.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:00.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:00.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:00.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:00.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:00.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:00 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:00.544+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:01.402 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.10.service' 2023-07-13T22:28:01.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:01.442 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-13T22:28:01.442 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-13T22:28:01.442 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.11 2023-07-13T22:28:01.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 systemd[1]: Stopping Ceph osd.11 for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:01.734 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:01.735 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.523+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:02.234 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.821+0000 7f8c5e90b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:02.234 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.821+0000 7f8c5e90b700 -1 osd.11 88 *** Got signal Terminated *** 2023-07-13T22:28:02.235 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:01 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:01.821+0000 7f8c5e90b700 -1 osd.11 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-13T22:28:02.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:02.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:02 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:02.553+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:03.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:03.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:03 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:03.575+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:04.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:04.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:04.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:04.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:04.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:04.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:04.986 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:04 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:04.615+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:05.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.572+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:05.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:05.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:05.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:05.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:05.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:05.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:05 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:05.573+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:06.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6806 osd.0 since back 2023-07-13T22:26:39.683664+0000 front 2023-07-13T22:26:39.683751+0000 (oldest deadline 2023-07-13T22:27:03.783629+0000) 2023-07-13T22:28:06.984 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2023-07-13T22:26:48.985836+0000 front 2023-07-13T22:26:48.985690+0000 (oldest deadline 2023-07-13T22:27:13.085788+0000) 2023-07-13T22:28:06.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6804 osd.2 since back 2023-07-13T22:26:53.086099+0000 front 2023-07-13T22:26:53.086171+0000 (oldest deadline 2023-07-13T22:27:18.386265+0000) 2023-07-13T22:28:06.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2023-07-13T22:27:12.989011+0000 front 2023-07-13T22:27:12.989098+0000 (oldest deadline 2023-07-13T22:27:37.589376+0000) 2023-07-13T22:28:06.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.121:6812 osd.5 since back 2023-07-13T22:27:18.690438+0000 front 2023-07-13T22:27:18.690406+0000 (oldest deadline 2023-07-13T22:27:44.590573+0000) 2023-07-13T22:28:06.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.26:6822 osd.6 since back 2023-07-13T22:27:24.591470+0000 front 2023-07-13T22:27:24.591375+0000 (oldest deadline 2023-07-13T22:27:50.491154+0000) 2023-07-13T22:28:06.985 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:06 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11[151422]: 2023-07-13T22:28:06.559+0000 7f8c596fe700 -1 osd.11 88 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2023-07-13T22:27:35.092565+0000 front 2023-07-13T22:27:35.092651+0000 (oldest deadline 2023-07-13T22:27:58.592469+0000) 2023-07-13T22:28:07.581 INFO:journalctl@ceph.osd.11.smithi121.stdout:Jul 13 22:28:07 smithi121 bash[161405]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-osd-11 2023-07-13T22:28:08.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@osd.11.service' 2023-07-13T22:28:08.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:08.883 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-13T22:28:08.884 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-13T22:28:08.884 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.a 2023-07-13T22:28:09.442 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:28:09 smithi026 systemd[1]: Stopping Ceph mds.a for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:09.942 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:28:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: 2023-07-13T22:28:09.469+0000 7f98d54a0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:09.942 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:28:09 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a[130614]: 2023-07-13T22:28:09.469+0000 7f98d54a0700 -1 mds.a *** got signal Terminated *** 2023-07-13T22:28:10.942 INFO:journalctl@ceph.mds.a.smithi026.stdout:Jul 13 22:28:10 smithi026 bash[161552]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-a 2023-07-13T22:28:11.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.a.service' 2023-07-13T22:28:11.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:11.269 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-13T22:28:11.269 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-13T22:28:11.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.d 2023-07-13T22:28:11.444 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:28:11 smithi026 systemd[1]: Stopping Ceph mds.d for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:12.192 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:28:11 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-d[129652]: 2023-07-13T22:28:11.823+0000 7f7a6a3d5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:12.192 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:28:11 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-d[129652]: 2023-07-13T22:28:11.823+0000 7f7a6a3d5700 -1 mds.d *** got signal Terminated *** 2023-07-13T22:28:13.191 INFO:journalctl@ceph.mds.d.smithi026.stdout:Jul 13 22:28:12 smithi026 bash[161696]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-d 2023-07-13T22:28:13.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.d.service' 2023-07-13T22:28:13.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:13.471 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-13T22:28:13.471 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-13T22:28:13.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.g 2023-07-13T22:28:13.691 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:28:13 smithi026 systemd[1]: Stopping Ceph mds.g for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:14.441 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:28:13 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: 2023-07-13T22:28:13.993+0000 7f4059448700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:14.442 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:28:13 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g[130223]: 2023-07-13T22:28:13.993+0000 7f4059448700 -1 mds.g *** got signal Terminated *** 2023-07-13T22:28:18.441 INFO:journalctl@ceph.mds.g.smithi026.stdout:Jul 13 22:28:18 smithi026 bash[161842]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-g 2023-07-13T22:28:18.633 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.g.service' 2023-07-13T22:28:18.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:18.717 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-13T22:28:18.718 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-13T22:28:18.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.j 2023-07-13T22:28:18.941 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:28:18 smithi026 systemd[1]: Stopping Ceph mds.j for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:19.691 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:28:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-j[129265]: 2023-07-13T22:28:19.242+0000 7ffa54200700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:19.692 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:28:19 smithi026 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-j[129265]: 2023-07-13T22:28:19.242+0000 7ffa54200700 -1 mds.j *** got signal Terminated *** 2023-07-13T22:28:22.191 INFO:journalctl@ceph.mds.j.smithi026.stdout:Jul 13 22:28:21 smithi026 bash[161987]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-j 2023-07-13T22:28:22.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.j.service' 2023-07-13T22:28:22.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:22.436 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-13T22:28:22.436 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-13T22:28:22.437 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.b 2023-07-13T22:28:22.789 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:28:22 smithi117 systemd[1]: Stopping Ceph mds.b for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:23.183 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:28:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: 2023-07-13T22:28:22.808+0000 7fa678fdd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:23.183 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:28:22 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b[154295]: 2023-07-13T22:28:22.808+0000 7fa678fdd700 -1 mds.b *** got signal Terminated *** 2023-07-13T22:28:26.935 INFO:journalctl@ceph.mds.b.smithi117.stdout:Jul 13 22:28:26 smithi117 bash[160542]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-b 2023-07-13T22:28:26.967 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.b.service' 2023-07-13T22:28:27.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:27.005 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-13T22:28:27.005 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-13T22:28:27.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.e 2023-07-13T22:28:27.378 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:28:27 smithi117 systemd[1]: Stopping Ceph mds.e for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:27.683 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:28:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: 2023-07-13T22:28:27.398+0000 7f014ecb1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:27.683 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:28:27 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e[153613]: 2023-07-13T22:28:27.398+0000 7f014ecb1700 -1 mds.e *** got signal Terminated *** 2023-07-13T22:28:30.933 INFO:journalctl@ceph.mds.e.smithi117.stdout:Jul 13 22:28:30 smithi117 bash[160692]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-e 2023-07-13T22:28:30.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.e.service' 2023-07-13T22:28:30.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:30.979 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-13T22:28:30.979 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-13T22:28:30.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.h 2023-07-13T22:28:31.386 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:28:31 smithi117 systemd[1]: Stopping Ceph mds.h for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:31.683 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:28:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: 2023-07-13T22:28:31.406+0000 7fac61b98700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:31.684 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:28:31 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h[153954]: 2023-07-13T22:28:31.406+0000 7fac61b98700 -1 mds.h *** got signal Terminated *** 2023-07-13T22:28:34.933 INFO:journalctl@ceph.mds.h.smithi117.stdout:Jul 13 22:28:34 smithi117 bash[160844]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-h 2023-07-13T22:28:35.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.h.service' 2023-07-13T22:28:35.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:35.062 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-13T22:28:35.062 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-13T22:28:35.063 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.k 2023-07-13T22:28:35.422 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:28:35 smithi117 systemd[1]: Stopping Ceph mds.k for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:35.683 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:28:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: 2023-07-13T22:28:35.443+0000 7fd2d3a52700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:35.683 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:28:35 smithi117 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k[153273]: 2023-07-13T22:28:35.443+0000 7fd2d3a52700 -1 mds.k *** got signal Terminated *** 2023-07-13T22:28:38.806 INFO:journalctl@ceph.mds.k.smithi117.stdout:Jul 13 22:28:38 smithi117 bash[160992]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-k 2023-07-13T22:28:38.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.k.service' 2023-07-13T22:28:39.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:39.030 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-13T22:28:39.031 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-13T22:28:39.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.c 2023-07-13T22:28:39.384 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:28:39 smithi121 systemd[1]: Stopping Ceph mds.c for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:39.734 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:28:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: 2023-07-13T22:28:39.403+0000 7f939cde8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:39.734 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:28:39 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c[154913]: 2023-07-13T22:28:39.403+0000 7f939cde8700 -1 mds.c *** got signal Terminated *** 2023-07-13T22:28:42.960 INFO:journalctl@ceph.mds.c.smithi121.stdout:Jul 13 22:28:42 smithi121 bash[161675]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-c 2023-07-13T22:28:42.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.c.service' 2023-07-13T22:28:43.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:43.004 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-13T22:28:43.004 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-13T22:28:43.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.f 2023-07-13T22:28:43.234 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:28:43 smithi121 systemd[1]: Stopping Ceph mds.f for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:43.734 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:28:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: 2023-07-13T22:28:43.402+0000 7f88e4903700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:43.734 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:28:43 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f[154574]: 2023-07-13T22:28:43.402+0000 7f88e4903700 -1 mds.f *** got signal Terminated *** 2023-07-13T22:28:46.981 INFO:journalctl@ceph.mds.f.smithi121.stdout:Jul 13 22:28:46 smithi121 bash[161820]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-f 2023-07-13T22:28:46.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.f.service' 2023-07-13T22:28:47.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:47.023 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-13T22:28:47.024 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-13T22:28:47.024 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.i 2023-07-13T22:28:47.398 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:28:47 smithi121 systemd[1]: Stopping Ceph mds.i for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:47.734 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:28:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: 2023-07-13T22:28:47.417+0000 7ff26b87d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:47.734 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:28:47 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i[155254]: 2023-07-13T22:28:47.417+0000 7ff26b87d700 -1 mds.i *** got signal Terminated *** 2023-07-13T22:28:50.984 INFO:journalctl@ceph.mds.i.smithi121.stdout:Jul 13 22:28:50 smithi121 bash[161964]: ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-i 2023-07-13T22:28:51.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.i.service' 2023-07-13T22:28:51.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:51.106 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-13T22:28:51.107 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-13T22:28:51.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.l 2023-07-13T22:28:51.480 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:28:51 smithi121 systemd[1]: Stopping Ceph mds.l for 23b26990-21c9-11ee-9b32-001a4aab830c... 2023-07-13T22:28:51.734 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:28:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: 2023-07-13T22:28:51.500+0000 7f082597a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-13T22:28:51.734 INFO:journalctl@ceph.mds.l.smithi121.stdout:Jul 13 22:28:51 smithi121 ceph-23b26990-21c9-11ee-9b32-001a4aab830c-mds-l[154236]: 2023-07-13T22:28:51.500+0000 7f082597a700 -1 mds.l *** got signal Terminated *** 2023-07-13T22:28:55.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-23b26990-21c9-11ee-9b32-001a4aab830c@mds.l.service' 2023-07-13T22:28:55.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-13T22:28:55.086 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-13T22:28:55.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-13T22:28:58.854 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-13T22:29:00.704 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force --keep-logs 2023-07-13T22:29:02.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:29:02.624 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:29:02.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-13T22:29:02.689 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-13T22:29:02.691 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi026/crash 2023-07-13T22:29:02.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash -- . 2023-07-13T22:29:02.764 INFO:teuthology.orchestra.run.smithi026.stderr:tar: /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-13T22:29:02.764 INFO:teuthology.orchestra.run.smithi026.stderr:tar: Error is not recoverable: exiting now 2023-07-13T22:29:02.768 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi117/crash 2023-07-13T22:29:02.769 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash -- . 2023-07-13T22:29:02.816 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-13T22:29:02.817 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2023-07-13T22:29:02.819 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi121/crash 2023-07-13T22:29:02.820 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash -- . 2023-07-13T22:29:02.859 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-13T22:29:02.859 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2023-07-13T22:29:02.860 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-13T22:29:02.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-13T22:29:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:grep: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph.log: No such file or directory 2023-07-13T22:29:02.946 INFO:tasks.cephadm:Compressing logs... 2023-07-13T22:29:02.946 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-13T22:29:02.951 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-13T22:29:02.953 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-13T22:29:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.5.log 2023-07-13T22:29:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:: No such file or directory 2023-07-13T22:29:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.2.log 2023-07-13T22:29:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:gzip /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.5.log: -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.11.log 2023-07-13T22:29:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.z.log 2023-07-13T22:29:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.8.log 2023-07-13T22:29:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.f.log 2023-07-13T22:29:02.981 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.c.log 2023-07-13T22:29:02.982 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip : No such file or directory 2023-07-13T22:29:02.982 INFO:teuthology.orchestra.run.smithi117.stderr:-5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.10.log 2023-07-13T22:29:02.983 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.h.log 2023-07-13T22:29:02.983 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.e.log 2023-07-13T22:29:02.983 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.4.log 2023-07-13T22:29:02.984 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.1.log 2023-07-13T22:29:02.984 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.k.log 2023-07-13T22:29:02.984 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-13T22:29:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.11.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.z.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.8.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.f.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.c.log: 92.9% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.z.log.gz 2023-07-13T22:29:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.i.log 2023-07-13T22:29:02.991 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.k.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.7.log 2023-07-13T22:29:02.993 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.c.log 2023-07-13T22:29:03.002 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.7.log: 91.5% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-13T22:29:03.002 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.b.log 2023-07-13T22:29:03.005 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-13T22:29:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.b.log 2023-07-13T22:29:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-13T22:29:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.0.log 2023-07-13T22:29:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.g.log 2023-07-13T22:29:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.a.log 2023-07-13T22:29:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.6.log 2023-07-13T22:29:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.3.log 2023-07-13T22:29:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.d.log 2023-07-13T22:29:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.x.log 2023-07-13T22:29:03.024 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log: 91.4% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-13T22:29:03.025 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.l.log 2023-07-13T22:29:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.9.log 2023-07-13T22:29:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.x.log: /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.9.log: 94.2% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.d.log.gz 2023-07-13T22:29:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log 2023-07-13T22:29:03.035 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.j.log 2023-07-13T22:29:03.037 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.b.log: 94.2% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.e.log.gz 2023-07-13T22:29:03.037 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.y.log 2023-07-13T22:29:03.044 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.l.log: 94.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.c.log.gz 2023-07-13T22:29:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-13T22:29:03.046 INFO:teuthology.orchestra.run.smithi026.stderr: 91.5% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-volume.log.gz 2023-07-13T22:29:03.047 INFO:teuthology.orchestra.run.smithi117.stderr: 92.7% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.k.log.gz 2023-07-13T22:29:03.055 INFO:teuthology.orchestra.run.smithi121.stderr: 92.4% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.f.log.gz 2023-07-13T22:29:03.056 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.y.log: 92.8% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.y.log.gz 2023-07-13T22:29:03.056 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-13T22:29:03.056 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-13T22:29:03.057 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.a.log 2023-07-13T22:29:03.057 INFO:teuthology.orchestra.run.smithi026.stderr: 94.3% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.j.log.gz 2023-07-13T22:29:03.058 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-13T22:29:03.068 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-13T22:29:03.076 INFO:teuthology.orchestra.run.smithi117.stderr: 92.3% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.h.log.gz 2023-07-13T22:29:03.078 INFO:teuthology.orchestra.run.smithi026.stderr: 94.1% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.g.log.gz 2023-07-13T22:29:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.a.log.gz 2023-07-13T22:29:03.086 INFO:teuthology.orchestra.run.smithi026.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-13T22:29:03.098 INFO:teuthology.orchestra.run.smithi121.stderr: 94.3% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.l.log.gz 2023-07-13T22:29:03.226 INFO:teuthology.orchestra.run.smithi121.stderr: 94.2% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.i.log.gz 2023-07-13T22:29:03.302 INFO:teuthology.orchestra.run.smithi026.stderr: 89.9% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mgr.x.log.gz 2023-07-13T22:29:03.364 INFO:teuthology.orchestra.run.smithi121.stderr: 91.8% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.c.log.gz 2023-07-13T22:29:03.382 INFO:teuthology.orchestra.run.smithi117.stderr: 91.9% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.b.log.gz 2023-07-13T22:29:04.386 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mds.b.log.gz 2023-07-13T22:29:04.399 INFO:teuthology.orchestra.run.smithi026.stderr: 90.4% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-mon.a.log.gz 2023-07-13T22:29:04.653 INFO:teuthology.orchestra.run.smithi026.stderr: 92.8% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.9.log.gz 2023-07-13T22:29:04.872 INFO:teuthology.orchestra.run.smithi121.stderr: 92.9% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.8.log.gz 2023-07-13T22:29:04.924 INFO:teuthology.orchestra.run.smithi121.stderr: 93.1% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.2.log.gz 2023-07-13T22:29:05.120 INFO:teuthology.orchestra.run.smithi121.stderr: 93.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.5.log.gz 2023-07-13T22:29:05.122 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.4.log.gz 2023-07-13T22:29:05.262 INFO:teuthology.orchestra.run.smithi121.stderr: 93.3% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.11.log.gz 2023-07-13T22:29:05.263 INFO:teuthology.orchestra.run.smithi121.stderr: 2023-07-13T22:29:05.264 INFO:teuthology.orchestra.run.smithi121.stderr:real 0m2.295s 2023-07-13T22:29:05.264 INFO:teuthology.orchestra.run.smithi121.stderr:user 0m8.722s 2023-07-13T22:29:05.264 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m0.351s 2023-07-13T22:29:05.346 INFO:teuthology.orchestra.run.smithi117.stderr: 93.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.10.log.gz 2023-07-13T22:29:05.425 INFO:teuthology.orchestra.run.smithi117.stderr: 93.2% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.7.log.gz 2023-07-13T22:29:05.546 INFO:teuthology.orchestra.run.smithi026.stderr: 93.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.6.log.gz 2023-07-13T22:29:05.590 INFO:teuthology.orchestra.run.smithi026.stderr: 93.0% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.3.log.gz 2023-07-13T22:29:05.749 INFO:teuthology.orchestra.run.smithi117.stderr: 93.2% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.1.log.gz 2023-07-13T22:29:05.750 INFO:teuthology.orchestra.run.smithi117.stderr: 2023-07-13T22:29:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m2.780s 2023-07-13T22:29:05.750 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m11.229s 2023-07-13T22:29:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.428s 2023-07-13T22:29:06.065 INFO:teuthology.orchestra.run.smithi026.stderr: 93.1% -- replaced with /var/log/ceph/23b26990-21c9-11ee-9b32-001a4aab830c/ceph-osd.0.log.gz 2023-07-13T22:29:06.068 INFO:teuthology.orchestra.run.smithi026.stderr: 2023-07-13T22:29:06.069 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m3.082s 2023-07-13T22:29:06.069 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m10.640s 2023-07-13T22:29:06.069 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m0.977s 2023-07-13T22:29:06.070 INFO:tasks.cephadm:Archiving logs... 2023-07-13T22:29:06.071 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi026/log 2023-07-13T22:29:06.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-13T22:29:07.368 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi117/log 2023-07-13T22:29:07.369 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-13T22:29:08.629 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi121/log 2023-07-13T22:29:08.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-13T22:29:09.708 INFO:tasks.cephadm:Removing cluster... 2023-07-13T22:29:09.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force 2023-07-13T22:29:10.863 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force 2023-07-13T22:29:11.635 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 23b26990-21c9-11ee-9b32-001a4aab830c --force 2023-07-13T22:29:12.401 INFO:tasks.cephadm:Removing cephadm ... 2023-07-13T22:29:12.402 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-13T22:29:12.451 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-13T22:29:12.473 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-13T22:29:12.491 INFO:tasks.cephadm:Teardown complete 2023-07-13T22:29:12.492 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-13T22:29:12.504 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-13T22:29:12.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-13T22:29:12.509 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-13T22:29:12.515 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-13T22:29:12.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-13T22:29:12.699 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-13T22:29:12.700 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:29:12.705 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:29:12.707 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-13T22:29:12.723 INFO:teuthology.orchestra.run.smithi117.stderr:bash: ntpq: command not found 2023-07-13T22:29:12.723 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-07-13T22:29:12.727 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:29:12.727 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2023-07-13T22:29:12.728 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 46 +369us[ +369us] +/- 86ms 2023-07-13T22:29:12.728 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -1500us[-1505us] +/- 72ms 2023-07-13T22:29:12.728 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +1548us[+1542us] +/- 86ms 2023-07-13T22:29:12.728 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:29:12.729 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:29:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-07-13T22:29:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 +370us[ +308us] +/- 86ms 2023-07-13T22:29:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -1515us[-1577us] +/- 72ms 2023-07-13T22:29:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +1535us[+1473us] +/- 86ms 2023-07-13T22:29:12.730 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:29:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:bash: ntpq: command not found 2023-07-13T22:29:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-13T22:29:12.752 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2023-07-13T22:29:12.753 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 +363us[ +363us] +/- 86ms 2023-07-13T22:29:12.753 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -1535us[-1510us] +/- 72ms 2023-07-13T22:29:12.753 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +1525us[+1525us] +/- 86ms 2023-07-13T22:29:12.753 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-13T22:29:12.754 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-13T22:29:12.766 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-13T22:29:12.767 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-13T22:29:12.794 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-13T22:29:12.976 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-13T22:29:13.079 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-13T22:29:13.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T22:29:13.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:13.248 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T22:29:13.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:13.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-13T22:29:13.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:13.314 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-13T22:29:13.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-13T22:29:13.354 INFO:teuthology.task.internal:Duration was 2498.751767 seconds 2023-07-13T22:29:13.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-13T22:29:13.388 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-13T22:29:13.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-13T22:29:13.393 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-13T22:29:13.396 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-13T22:29:13.441 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T22:29:13.442 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T22:29:13.509 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-13T22:29:13.745 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-13T22:29:13.746 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2023-07-13T22:29:13.746 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-13T22:29:13.805 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2023-07-13T22:29:13.806 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-13T22:29:13.828 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2023-07-13T22:29:13.828 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-13T22:29:13.850 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-13T22:29:13.851 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-13T22:29:13.855 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-13T22:29:13.870 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-13T22:29:13.936 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-13T22:29:13.937 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-13T22:29:13.944 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-13T22:29:13.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-13T22:29:14.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-13T22:29:14.119 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-13T22:29:14.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-13T22:29:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:mv: cannot stat '/etc/sudoers.orig.teuthology': No such file or directory 2023-07-13T22:29:14.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:14.191 ERROR:teuthology.run_tasks:Manager failed: internal.sudo Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi026 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' 2023-07-13T22:29:14.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-13T22:29:14.219 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:29:14.224 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:29:14.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-13T22:29:14.252 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2023-07-13T22:29:14.254 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2023-07-13T22:29:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2023-07-13T22:29:14.266 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2023-07-13T22:29:14.266 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2023-07-13T22:29:14.267 INFO:teuthology.orchestra.run.smithi121.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2023-07-13T22:29:14.288 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2023-07-13T22:29:14.320 INFO:teuthology.orchestra.run.smithi026.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2023-07-13T22:29:14.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-13T22:29:14.325 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2023-07-13T22:29:14.326 ERROR:teuthology.run_tasks:Manager failed: internal.coredump Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/task/internal/__init__.py", line 488, in coredump yield File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi026 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/task/internal/__init__.py", line 491, in coredump run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi026 with status 1: "sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump" 2023-07-13T22:29:14.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-13T22:29:14.337 INFO:teuthology.task.internal:Transferring archived files... 2023-07-13T22:29:14.338 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi026 2023-07-13T22:29:14.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-13T22:29:14.422 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi117 2023-07-13T22:29:14.422 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-13T22:29:14.459 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268/remote/smithi121 2023-07-13T22:29:14.460 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-13T22:29:14.502 INFO:teuthology.task.internal:Removing archive directory... 2023-07-13T22:29:14.503 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-13T22:29:14.509 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-13T22:29:14.512 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-13T22:29:14.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-13T22:29:14.573 INFO:teuthology.task.internal:Not uploading archives. 2023-07-13T22:29:14.574 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-13T22:29:14.620 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-13T22:29:14.620 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-13T22:29:14.625 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-13T22:29:14.628 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-13T22:29:14.641 INFO:teuthology.orchestra.run.smithi117.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 13 22:29 /home/ubuntu/cephtest 2023-07-13T22:29:14.645 INFO:teuthology.orchestra.run.smithi121.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 13 22:29 /home/ubuntu/cephtest 2023-07-13T22:29:14.664 INFO:teuthology.orchestra.run.smithi026.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 13 22:29 /home/ubuntu/cephtest 2023-07-13T22:29:14.667 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-13T22:29:14.679 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-13T22:29:14.709 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi026 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' 2023-07-13T22:29:14.710 INFO:teuthology.nuke:Checking targets against current locks 2023-07-13T22:29:14.739 DEBUG:teuthology.nuke:shortname: smithi026 2023-07-13T22:29:14.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T22:29:14.763 DEBUG:teuthology.nuke:shortname: smithi117 2023-07-13T22:29:14.763 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T22:29:14.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.079125', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-13T22:29:14.790 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.081778', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-13T22:29:14.792 DEBUG:teuthology.nuke:shortname: smithi121 2023-07-13T22:29:14.792 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-13T22:29:14.819 INFO:teuthology.orchestra.console:Power off smithi026 2023-07-13T22:29:14.820 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-13T22:29:14.834 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/7336268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-13 21:36:14.080855', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-13T22:29:14.844 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:14.869 INFO:teuthology.orchestra.console:Power off smithi117 2023-07-13T22:29:14.870 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-13T22:29:14.893 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:14.915 INFO:teuthology.orchestra.console:Power off smithi121 2023-07-13T22:29:14.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-13T22:29:14.938 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:18.884 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:18.928 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:18.963 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:23.000 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:23.042 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:23.075 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:23.213 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2023-07-13T22:29:23.287 INFO:teuthology.orchestra.console:Power off for smithi121 completed 2023-07-13T22:29:27.156 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-13T22:29:27.368 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2023-07-13T22:29:27.474 INFO:teuthology.run:Summary data: client.0-kernel-sha1: a440c79eee0258ae07e4988d8690fe6187b8b7a3 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} duration: 2498.751766681671 failure_reason: 'Command failed on smithi026 with status 1: ''sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers''' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-13T22:29:27.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-13T22:29:27.582 INFO:teuthology.run:FAIL